How Queen works under the hood.

Queen is three layers stacked end-to-end. PostgreSQL stores everything. A C++ engine called libqueen fuses many HTTP requests into batched stored-procedure calls. A uWebSockets HTTP server distributes work across worker threads and routes responses back to the right connection. This page is the mental model, enough to predict how the system behaves under any workload, without reading the source.

The three layers

Read the system top-down. Each layer has one job.

The trick that makes this fast: HTTP threads never touch libpq directly. They drop a request into libqueen's per-type queue and return immediately. libqueen collects requests of the same type, fuses them into one batched stored-procedure call on a non-blocking PostgreSQL connection, then disperses the per-row results back to each waiting HTTP request via a cross-thread defer().

Data model, PostgreSQL is the source of truth

Everything the system knows about, every queue, every message, every consumer offset , lives in PostgreSQL rows. There is no in-memory queue state on the server that survives a restart. The schema is intentionally small:

Two indexes carry the hot path

• messages (partition_id, created_at, id), the cursor scan that pop walks to find the next batch of unconsumed messages on a claimed partition.
• UNIQUE (partition_id, transaction_id) on messages, enforces dedupe via ON CONFLICT DO NOTHING in push.

Update-heavy tables (partition_lookup, partition_consumers, stats) ship with FILLFACTOR=50 and tuned autovacuum knobs to keep them HOT-update-friendly under sustained write load.

Two coordination primitives

Both live inside the procedures, not the schema:

• POP wildcard claim, pg_try_advisory_xact_lock(hashtextextended(partition_id::text, …)). Non-blocking try; the first request to claim a partition wins it for the duration of its transaction.
• ACK serialization, pg_advisory_xact_lock(...md5(partition_id || consumer_group)...). Blocking lock; prevents two concurrent acks from racing on the same cursor row.

These are the entire concurrency story. There are no application-level mutexes, no external coordination services. PostgreSQL is the lock manager.

The __QUEUE_MODE__ trick

When a consumer pops without specifying a group, every code path internally uses the literal string '__QUEUE_MODE__' as the consumer-group value. Same partition_consumers row mechanism, same offset-tracking, same procedures , just a special name. This is why the behavioral difference between "queue mode" and "consumer-group mode" is small enough to coexist in one stored procedure.

Stored procedures, the hot-path API

Every hot-path operation is a named, versioned PostgreSQL procedure. They all take a JSON array (the batch from libqueen) and return a JSON result. This is what makes request fusion safe: many client requests collapse into one procedure call, one network round trip, one transaction.

How push actually works

Worth a closer look, push is the highest-throughput hot path. push_messages_v3 is three statements with no temp tables:

-- 1. Ensure all queues in this batch exist.
INSERT INTO queen.queues (name, …)
SELECT DISTINCT … FROM jsonb_array_elements(p_items)
ON CONFLICT (name) DO NOTHING;

-- 2. Ensure all partitions exist.
INSERT INTO queen.partitions (queue_id, name)
SELECT DISTINCT … FROM jsonb_array_elements(p_items) JOIN queues …
ON CONFLICT (queue_id, name) DO NOTHING;

-- 3. One CTE: parse → resolve partition_id → row_number for intra-batch
--    dedupe → INSERT … ON CONFLICT DO NOTHING → RETURNING.
WITH parsed AS MATERIALIZED ( … gen_random_uuid(), partition_id … ),
     deduped AS (SELECT *, row_number() OVER (PARTITION BY (partition_id, transaction_id)
                                              ORDER BY idx) AS dup_rank
                 FROM parsed),
     inserted AS (INSERT INTO queen.messages …
                  SELECT * FROM deduped WHERE dup_rank = 1
                  ON CONFLICT (partition_id, transaction_id) DO NOTHING
                  RETURNING …)
SELECT jsonb_build_object('items', …, 'partition_updates', …);

Three SQL statements, one network round trip from libqueen, hundreds of pushed messages per call under load. The partition_updates aggregate in the response is what the server replays into update_partition_lookup_v1 after commit, so the wildcard pop path always finds the latest writes.

How pop finds the next message

Pop is the most intricate procedure because it covers four shapes (queue / consumer group × fixed / wildcard partition) in one function. The wildcard-by-consumer-group path is the interesting one:

1. Read the last_empty_scan_at watermark from consumer_watermarks.
2. Scan partition_lookup for partitions where updated_at >= watermark - 2 minutes (caught-up consumers skip the rest), the lease is free, and the cursor is behind the head.
3. For each candidate, try pg_try_advisory_xact_lock on the partition id. The first successful claim wins; the loop exits.
4. UPSERT a row in partition_consumers, then UPDATE it to take the lease and read the cursor.
5. SELECT … FROM messages WHERE partition_id = $ AND (created_at, id) > (cursor) ORDER BY (created_at, id) LIMIT batch_size.

If no partition was claimed, the watermark is bumped and the procedure returns no_available_partition. libqueen turns this into a long-poll backoff rather than an error response.

libqueen, the request-fusion engine

libqueen is a per-HTTP-worker C++ library that owns its own libuv event loop and a pool of non-blocking PostgreSQL connections. Its job: take many HTTP requests of the same kind and turn them into one batched stored-procedure call.

The JobType enum

enum class JobType : uint8_t {
    PUSH = 0,        // batchable
    POP,             // batchable in groups
    ACK,             // batchable
    TRANSACTION,     // NOT batchable, atomic, serial
    RENEW_LEASE,     // batchable
    CUSTOM,          // NOT batchable, per-job SQL
};

// Every batchable type maps to exactly one stored procedure:
{ JobType::PUSH,        "SELECT queen.push_messages_v3($1::jsonb)" },
{ JobType::POP,         "SELECT queen.pop_unified_batch_v4($1::jsonb)" },
{ JobType::ACK,         "SELECT queen.ack_messages_v2($1::jsonb)" },
{ JobType::TRANSACTION, "SELECT queen.execute_transaction_v2($1::jsonb)" },
{ JobType::RENEW_LEASE, "SELECT queen.renew_lease_v2($1::jsonb)" },

Each JobType has its own queue, its own batch policy, and its own concurrency controller. They run in parallel, a high-load pop doesn't block push, and vice versa.

BatchPolicy, when to fire

Three knobs per type, settable via QUEEN_<TYPE>_* env vars:

FireDecision should_fire(size_t queue_size,
                         milliseconds oldest_age) const {
    if (queue_size == 0)                            return HOLD;
    if (queue_size >= preferred_batch_size)         return FIRE;  // throughput
    if (oldest_age >= max_hold_ms)                  return FIRE;  // latency cap
    return HOLD;
}

Fire when there are enough requests queued, or when the oldest one has waited too long. So latency is bounded by max_hold_ms when load is low; throughput compounds via batching when load is high. Defaults: PUSH 50/20ms, POP 20/5ms (latency-sensitive), ACK 50/20ms, TRANSACTION 1/0 (never fuse).

ConcurrencyController, adaptive in-flight cap

Two implementations, switched by QUEEN_CONCURRENCY_MODE:

• Static: a fixed cap. Atomic counter against max_concurrent. Predictable and boring.
• Vegas (default): TCP-Vegas-inspired adaptive limit. The controller computes queue_load = in_flight × (1 − rtt_min ⁄ rtt_recent). If queue_load < α (default 3) → grow the limit; if queue_load > β (default 12) → shrink. rtt_min is a 30-second sliding minimum (so a single fast outlier doesn't pin it). Adjusted at most once per second to prevent thrashing.

This is why QUEEN_PUSH_MAX_CONCURRENT ships at 24 even though Vegas typically converges to ~17 in steady state, Vegas needs headroom to grow into when load rises. The cap is a safety ceiling, not an operating point.

DrainOrchestrator, the heart

All scheduling runs on libqueen's dedicated libuv thread. The drain loop wakes on four sources:

One drain pass: process expired long-polls; round-robin over the six JobTypes (with a rotating start index so no type starves); for each type, while the BatchPolicy says fire and the ConcurrencyController has a slot and there's a free DB connection, take up to max_batch jobs, merge their JSON arrays, and call PQsendQueryParams on a non-blocking connection. Re-arm the safety-net timer.

libpq async, integrated with libuv

Each PostgreSQL connection is wrapped in a DBConnection with the libpq socket fd and a uv_poll_t. When a batch fires, libqueen starts polling for UV_WRITABLE | UV_READABLE:

1. On writable → PQflush until the query is sent.
2. On readable → PQconsumeInput + PQisBusy loop, then PQgetResult reads the JSON response.
3. The procedure returns one result row containing one JSONB array; libqueen splits it back into per-job results and invokes each PendingJob::callback(result_str).
4. Slot returns to the free pool, kicks the orchestrator again, which is why throughput stays high when many slots become free at once.

Long polling

When pop returns no_available_partition, libqueen tracks a per-request next_check with exponential backoff (initial 100ms, multiplier 2.0, cap 1000ms) and a wait_deadline from the client's timeout parameter. The safety-net timer fires the request again at next_check without holding any DB connection in between, this is what makes "30k idle long-pollers" actually fine.

HTTP shell, uWebSockets, acceptor + workers

The HTTP layer is small and uniform. main_acceptor.cpp is 76 lines; acceptor_server.cpp wires everything together.

Acceptor / worker pattern

• One acceptor thread listens on :6632. When a TCP connection arrives, it adoptSocket()s the socket onto one of the worker uWS::App event loops. The acceptor never serves a request itself.
• N worker threads (default 10), each running its own uWS::App and its own Queen instance. Each worker has its own ResponseRegistry, so cross-worker response delivery has no shared mutex.
• Worker 0 bootstraps schema + starts the background services: MetricsCollector, RetentionService, EvictionService, StatsService, PartitionLookupReconcileService, and SharedStateManager. Other workers wait on a condvar until worker 0 signals ready.

Cross-thread response delivery

This is the trickiest part of the C++ code, but the model is simple. When a route handler calls queen->submit(job, callback), the callback runs on libqueen's libuv thread, not the uWS thread that owns the HttpResponse*. Touching that pointer from the wrong thread is undefined behavior. So:

// On uWS thread: register the response, get a request_id back.
auto request_id = registry->register_response(res, worker_id, …);

// Submit to libqueen (runs immediately, callback fires later).
ctx.queen->submit(std::move(job_req),
    [worker_loop, worker_id, request_id](std::string result) {
        // We're on libqueen's libuv thread now.
        // Hop back to the uWS worker loop.
        worker_loop->defer([result, worker_id, request_id]() {
            // Safe to touch the response now.
            registry->send_response(request_id, parse(result), …);
        });
    });

uWS::Loop::defer() queues the lambda on the worker's event loop. If the client disconnected in the meantime, res->onAborted already invalidated the registry entry, late deliveries become no-ops.

End-to-end walkthrough

One push, in detail

1. Client sends POST /api/v1/push with {items: [...]}.
2. Acceptor adopts the socket onto worker N.
3. Auth middleware validates the JWT and stamps producerSub from the verified sub claim. Client-supplied producerSub is ignored, this is what closes the impersonation hole.
4. If the queue has encryption enabled, EncryptionService replaces each payload with {encrypted, iv, authTag} (AES-256-GCM).
5. The route registers the response in the per-worker ResponseRegistry, gets back a request_id.
6. It stashes the items in push_failover_storage[request_id] in case the DB call fails.
7. It builds a JobRequest{op_type=PUSH, params=[items_json]} and calls queen->submit(). The HTTP thread is unblocked.
8. libqueen's drain orchestrator collects this push along with others queued nearby. When BatchPolicy says fire, it merges their item arrays and calls PQsendQueryParams("SELECT queen.push_messages_v3($1::jsonb)", ...).
9. The procedure runs three SQL statements (queues upsert, partitions upsert, messages insert) and returns {items: [...], partition_updates: [...]}.
10. libqueen reads the response, splits it back into per-request results, invokes each callback.
11. The callback defer()s onto the uWS worker loop. There, the route writes the response. It also fires a UDP MESSAGE_AVAILABLE packet to peer instances and writes partition_updates via update_partition_lookup_v1.

A transactional pipeline

The path is the same up to queen->submit, but the JobType is TRANSACTION with max_concurrent=1, max_batch=1. libqueen runs the request alone, calling execute_transaction_v2. Inside the procedure, ack and push are inlined as one PL/pgSQL function, i.e. one PostgreSQL transaction. Any RAISE in either operation rolls back the whole bundle: the input message stays unacked, the output message never landed. On commit, both are durable in the same WAL record. That is exactly-once across queues.

Failover, multi-instance sync, encryption

Disk failover for push

If PostgreSQL is unreachable mid-push, the route handler's callback parses {success: false, error: …} and replays the items from push_failover_storage into the FileBufferManager, a rotating JSON-Lines file in FILE_BUFFER_DIR. A background scanner reads the file every FILE_BUFFER_FLUSH_MS (default 100ms) and replays buffered events into the database once it's reachable again. On startup, worker 0 runs a recovery pass over leftover files before accepting traffic.

Pop and ack do not have a file-buffer path. They require the database to be live. The disk buffer is about not losing inbound writes, not about serving reads from disk. For full HA the database itself must be HA (Patroni, RDS Multi-AZ, Cloud SQL, etc.), Queen just reconnects when the upstream reappears.

Multi-instance UDP sync

Queen servers are stateless above PostgreSQL, you can run as many as you want pointed at the same DB. Two optional UDP layers make horizontal scale fast:

• Peer notifications: when a server commits a push, it broadcasts a MESSAGE_AVAILABLE packet to peers. libqueen's POP backoff tracker hears this and resets the affected long-polls to fire immediately. Sub-millisecond cross-instance wake-up.
• Distributed cache (UDPSYNC): heartbeats + shared-state cache for queue config, partition-id LRU, and server-health tracking. Heartbeats authenticate via HMAC-SHA256 (QUEEN_SYNC_SECRET).

The cluster works fine without UDP, long-poll consumers just fall back to a slightly longer wake-up interval.

At-rest encryption

When a queue has encryptionEnabled=true and QUEEN_ENCRYPTION_KEY is set (32 bytes hex), the push route encrypts each payload with AES-256-GCM before sending it to PostgreSQL. The stored payload becomes {encrypted, iv, authTag}. Pop transparently decrypts on the way out. The key never leaves the server process; PostgreSQL never sees plaintext.

Authentication

Optional JWT, supporting HS256, RS256, and EdDSA. Routes are gated by access level (read-only / read-write / admin) derived from a configurable role claim. When auth is enabled the server stamps the validated sub claim onto every pushed message as producerSub, clients can't set this field, so even a compromised JWT cannot impersonate another producer in the message log.

Why this design

What we traded away

Every architectural choice has a counter-cost. Queen's choices buy strong durability, ordering at high partition cardinality, and operational simplicity, and they pay for it in four real ways. None of these are bugs; they're consequences of being a Postgres-backed message queue speaking HTTP/JSON. Knowing them upfront helps you decide whether Queen is the right tool for what you're building.

These trade-offs are real, but they're scoped: they only matter if your workload sits at the edges of Queen's envelope. For most business workloads (<100k msg/s, single-region, single-Postgres), none of this is a problem. The benchmarks page has the full picture with measured numbers and side-by-side comparisons against Kafka and RabbitMQ.

Where to read more