Soak · Queen 0.16 (pushser) · June 2026

24 hours. 10.4 billion messages. Zero loss.

A single continuous run of Queen 0.16 (the pushser architecture) under a balanced producer/consumer workload on a 32 vCPU / 62 GiB host with PostgreSQL. The goal isn't a peak number, it's stability: hold a high rate for a full day and prove nothing drifts, leaks, or loses a message. Raw data, the chart builder, and the full standalone report live in benchmark-queen/2026-06-07.

Queen 0.16 24-hour soak overview: throughput (push, pop, retention deletes all ~119k msg/s), CPU, memory and disk across 24 hours — **The whole run at a glance.** Push, pop & retention-delete throughput, CPU, memory and disk across the full 24 hours. Broker fusion coalesced ~12.3k HTTP requests/s into ~1.09k Postgres commits/s (≈11 requests & ≈220 row writes per commit).

Headline

Duration 24.4 h single continuous run

Throughput (avg) ~119k msg/s push and pop, balanced

Messages processed 10.42 B pushed = popped

Push errors 0 over 10.4 billion pushes

Broker memory ~400 MiB flat for 24h, no growth

Broker CPU ~5 / 32 cores never the bottleneck

Postgres CPU ~21 / 32 cores the real ceiling

PG cache hit 100 % across the whole run

Verdict. Queen 0.16 sustained ~119k msg/s of balanced push and pop for a full day, processing 10.42 billion messages with zero push errors and an in-flight gap of 0.001% (effectively zero loss). Broker memory stayed flat at ~400 MiB. The throughput ceiling is Postgres CPU (~21 of 32 cores), not the broker, there is headroom on the broker side the entire time.

The live dashboard at the 24-hour mark

This is Queen's own dashboard, not an external tool, a screenshot taken live at the end of the run. 10B+ messages, 301 partitions, throughput holding in its band, pending bounded.

Queen dashboard at the 24-hour mark showing throughput holding around 120k msg/s and pending bounded — **Queen dashboard, 24h in.** Throughput sits in its band with push ≈ pop ≈ ack; the pending series oscillates within a small in-flight window and never runs away.

Throughput: flat for 24 hours

Loader-side push/s and pop/s sampled every 10 s for the whole run. Thin lines are raw samples; thick lines are the 5-minute rolling mean.

Push and pop throughput tracking each other around 118k msg/s for 24 hours — **Push vs pop over 24h.** The two curves sit on top of each other in a ~118k msg/s band, no drift, no decay, no warm-up artifact after the first minute.

Average over the full run: ~118,700 msg/s push and ~118,700 msg/s pop. Pop tracks push the entire time, so the queue is drained as fast as it fills, the system runs at a stable operating point rather than building a backlog.

Correctness: 10.4 billion in, 10.4 billion out

Volume

10.42B Pushed10,422,886,560 Popped10,422,764,720 Avg rate~119k/s

Errors

0 Push errors0 Consumer errors172 Empty pops136

In-flight gap

0.001% Peak gap~0.3M vs total10.4B Lost0

Push minus pop gap staying a small bounded in-flight window across 24 hours — **Push − pop gap.** The difference between cumulative pushed and popped is just the instantaneous in-flight window, a few tens to a couple hundred thousand messages against 10.4 billion processed. It never trends upward.

Broker memory: a flat line

The single most important durability signal: does the broker leak over a long run?

Broker resident memory flat at around 400 MiB across the full 24 hours — **Queen broker RSS.** After a few-minute warm-up the broker holds ~400 MiB for the entire day. No sawtooth, no upward creep.

This is the headline of the pushser rework. An earlier architecture exhausted memory and crashed on a long soak; 0.16 runs a full day in a flat ~400 MiB footprint while doing 10.4 billion messages.

CPU: the broker is cheap, Postgres does the work

Postgres CPU around 21 cores and Queen broker around 5 cores across 24 hours — **Queen vs Postgres CPU (of 32 cores).** The broker holds ~5 cores; Postgres carries ~21. The throughput ceiling is Postgres, and there is broker headroom the whole run.

Storage & retention

Queen keeps two kinds of data: the live queue (messages), bounded by retention, and a durable record of every processed operation (messages_consumed), the operation history you can query, replay against, and prune on your own schedule.

Postgres rows inserted per second and deleted per second tracking each other near 120k — **Inserts vs retention deletes.** Postgres' own counters: ingestion and retention run neck-and-neck near 120k rows/s, so the live queue stays in steady state.

Disk free declining from 377 GB to 218 GB over 24 hours — **Disk free.** The steep early drop is WAL filling to its configured ceiling plus the tables warming; the gentle slope after is the operation history accumulating, by design.

Queen dashboard Postgres stats showing 100% database, table and index hit ratios — **Postgres cache, 24h in.** 100% database / table / index hit ratios on 24 GB shared_buffers, the working set stays resident; reads almost never touch disk.

The live queue (messages) holds flat at ~15 GB for the entire run, retention keeps the working set bounded. The messages_consumed history grows with total volume (that's what an operation log does); operators set a completed-retention window or prune it on demand. It is data you choose to keep, not a leak.

Message flow over time showing ingested and processed lines overlapping across 24 hours — **Message flow, ingested vs processed.** The two lines overlap for the whole run: everything ingested is processed, continuously.

Configuration

Broker (Queen 0.16 + Postgres)

Host	32 vCPU · 62 GiB · no swap
`NUM_WORKERS`	10
`DB_POOL_SIZE`	50 (+250 sidecar)
Pop concurrency	`static`, max 16
Push coalescing	hold 20 ms · pref 50 · max 500
Retention	parallelism 8 · 5 s interval
`shared_buffers`	24 GB

Loader (goload)

Partitions	300
Producers	650
Consumers	200
Push batch	10
Pop batch	300 · 10 partitions/pop · long-poll
Payload	256 B
Completed retention	120 s

For perspective. The previous long-running test (April, v0.14) did 1.5 billion messages in 14 hours at ~28k msg/s. This run did ~7× the messages at ~4× the sustained rate, with flat broker memory throughout.

Reproduce it

The loader (goload), the monitor, the chart builder, the full standalone HTML report, and every raw sample (loader log at 10 s, broker/PG monitor at 30 s, final database snapshot) are in benchmark-queen/2026-06-07.

Companion benchmark