Fly Labs
Ask FlyBot

PicoMQ – Durable Streams over HTTP, on object storage

PicoMQ is a Rust server for Durable Streams, built on Object Store. Cheap, URL-addressable, granular streams (create/append/read/long-poll/SSE), with Pico Protocol or Durable Streams Protocol as the facade.S3Stream is the stream storage primitive, used in AutoMQ, shipped as a Rust library. Coordination is a command log in Postgres.

Hacker NewsToolDevSource
0Sign in to voteCopy link

FL score

62

out of 100

Verdict

VALIDATE

medium confidence

Competition

No competitor data yet

Trend

No signal yet

A Rust message queue that stores streams directly on S3 instead of managing its own infrastructure, targeting teams that want durability without operational overhead.

The pain

Teams running Kafka or RabbitMQ pay for always-on servers and operational complexity. Teams using managed queues (SQS, Kinesis) hit throughput limits or pay per-request pricing that scales poorly. Object storage is cheap and durable but has no queue abstraction.

The gap

No existing queue system treats object storage as the primary durability layer with HTTP as the access pattern. Kafka requires cluster management. SQS/Kinesis have throughput ceilings and pricing that doesn't match S3's cost curve. This sits in the middle, cheaper than managed queues at scale, simpler than Kafka.

Build angle

Start by validating that teams actually want to trade latency (object storage round-trips) for cost. Build a working prototype that handles 1000s of messages/sec over HTTP with long-poll and SSE. Get 3-5 paying customers using it for non-latency-critical workloads (batch processing, event archival, async tasks). Measure actual S3 costs vs. their current queue spend.

Strengths

  • Leverages existing S3 infrastructure, no new vendor lock-in, works with any S3-compatible storage
  • Cost structure aligns with actual usage, pay for storage and API calls, not reserved capacity
  • Rust implementation means low resource footprint and predictable performance
  • HTTP + standard protocols (Durable Streams) reduce client library friction
  • Postgres coordination is simple, well-understood, and doesn't add another moving part

Risks

  • Latency will be 100-500ms per operation due to S3 round-trips, unsuitable for real-time systems, which is most of the queue market
  • S3 request pricing ($0.0004 per PUT) becomes expensive at high throughput; break-even math unclear vs. managed queues
  • Competing against entrenched Kafka (free, battle-tested) and AWS SQS (integrated, simple). Hard to win on features.
  • Market for 'cheaper queues' is smaller than it appears, most teams optimize for latency/reliability first, cost second
  • Postgres coordination becomes a bottleneck if you need multi-region or high-availability setups
  • Unclear if AutoMQ's S3Stream library is production-ready or if licensing/support exists

Questions about this idea?

FlyBot reads the scoring and gives you a second opinion on “PicoMQ – Durable Streams over HTTP, on object storage”.

Open FlyBot