---
title: "Limits and non-goals"
description: "Queen's design decisions stated plainly, how to work with each one, and the operational boundaries to check before you build."
---

> Queen MQ documentation, for AI agents
> Complete self-contained summary of Queen MQ: https://queenmq.com/llms-brief.txt
> Fetch that first when the question is about the product rather than about this page.
> Index of all pages: https://queenmq.com/llms.txt

# Limits and non-goals

This page states Queen's opinions in one place: the design decisions, how to
work with each one, and the operational boundaries. Everything below is a
property of the build you can run today, not a roadmap item. Most of these are
how we believe a queue should behave; the point of collecting them here is that
you can check them against your requirements in one read.

## Per-tenant limits belong to the gateway

A bare broker enforces no per-caller quota: it is one namespace and one resource
pool. Put `queen_proxy` in front and every cluster carries a plan: request and
message rates with burst, queue and partition counts, payload and batch size,
parked long-poll slots, retained bytes and a retention ceiling, plus an optional
monthly message allowance. Enforcement is per cluster, and the wire contract is
one the SDKs already handle: `429` with `Retry-After` for a rate limit, which
every client retries on its own with backoff, and `403` with a code naming the
quota for a hard cap. What each number does at request time, in the order the
gateway evaluates them, is in
[Quotas and rate limits](/reference/multi-tenant/quotas).

## Ordering granularity is your partition key

A message's position is a per-partition `BIGINT` offset, allocated under that
partition's row lock, so the order is exactly as coarse or as fine as the key you
push with. One partition for a whole queue is total ordering across everything,
at one writer and one in-flight leased batch per consumer group: a correct
configuration, with one writer's worth of throughput. A partition per customer is
a million independent orders. Nothing about this is fixed when the queue is
created, because partitions are created by the push that names them.

What you cannot have is both at once. There is no cross-partition sequence
number, no global log, and no way to ask "what happened first" across two
partitions. Two consumers draining two partitions of the same queue observe two
independent orders. That is not a property of this implementation: a total order
has to be serialised through one writer, whatever the system underneath it.

The one extra guarantee you get *inside* a partition is that `created_at` is
monotone, because the timestamp is written under the same `FOR UPDATE` lock that
bumps the offset. Time-based retention and timestamp subscriptions depend on
that; it does not extend into a global clock.

## Delivery order is arrival order

There is no priority lane, no reordering at pop time, and no per-message
deadline: a message's position is fixed at push. If some work is more urgent than
other work, that is a different queue or a different partition, and you dedicate
consumers to it.

What you *can* shift is when a whole lane becomes visible. `delayedProcessing`
defers every message of a queue by N seconds after commit, and `windowBuffer`
holds a bursty partition until it goes quiet or the accumulated batch fills. Both
are per-queue options and neither reorders anything. `delayedProcessing` is
enforced in SQL on every pop path and is never skipped; `windowBuffer` is
enforced either by the same SQL debounce or, with `QUEEN_HOTLIST` on, by the
broker's timer wheel, which is what releases a continuously written partition at
first mark plus the window. Both are described in
[Queue options](/reference/queue-options).

## Parallelism inside a group is bounded by partition count

`queen.log_consumers` is keyed `(partition_id, consumer_group)` and carries one
`batch_end` and one lease. So exactly one in-flight leased batch exists per
(partition, group).

The rule that follows: **in-group parallelism equals partition count.** A
single-partition queue is drained by one worker at a time within a consumer
group, and further workers started against it get empty pops. Concurrency comes
from partitions. If you want more parallelism, choose a partition key with more
distinct values, or add a second consumer group, which is fan-out, not
work-sharing, because every group receives every message.

## One PostgreSQL is one failure domain and the ceiling

Brokers are stateless and scale horizontally. The database does not. Everything
Queen knows lives in one PostgreSQL, so that instance is simultaneously the
durability story, the coordination substrate, the throughput ceiling and the
single point of failure. High availability means PostgreSQL high availability.

While the database is unreachable, Queen degrades in a specific, asymmetric way.
Know which half keeps working before you rely on it:

- **Pushes are spooled.** When the transaction that would have stored a batch
  fails, the broker writes those messages to the on-disk spool
  (`FILE_BUFFER_DIR`) and answers HTTP 201 with per-item `status: "buffered"`
  instead of losing them. A background drain replays them oldest-first when the
  database returns, preserving each original `transactionId` so deduplication
  makes replay idempotent.
- **Reads need the database.** Pops, acks, transactions and every management
  route require PostgreSQL, so nothing is consumable while it is away. Buffered
  messages are not readable until they are drained.
- **The spool is bounded by your disk**, and by the fact that startup recovery
  stops after one hour of draining and hands the rest to the background loop.
  Size the volume for the longest outage you intend to absorb: a long outage is
  a large directory.

> **Danger**
>
> `/health` performs a real database round-trip and returns 503 when PostgreSQL is
> unreachable. Wiring it to a Kubernetes **liveness** probe therefore kills the
> broker during exactly the outage the disk spool exists to survive. The spool
> itself is not lost: a restarted broker finalizes any file it was mid-write on and
> replays the spool before it accepts a request, under the one-hour startup cap
> above, provided `FILE_BUFFER_DIR` sits on a volume that outlives the container.
> What the kill loop costs is every push arriving during it, refused at the socket
> instead of buffered. Use `/health` as a readiness signal.

## Retention deletes segments, not messages

Retention is **opt-in**: it requires `retentionEnabled` plus a positive window.
Without both, segments live forever and the disk grows monotonically. When it is
on, it runs as bounded autocommitting steps every `RETENTION_INTERVAL`
milliseconds (default 5000), on whichever replica claims the `retention` row in
`queen.maintenance_leases` for that period.

Granularity is a **whole segment**, one row holding many frames. So:

- "Delete exactly this one message" is not an operation Queen has. Retention
  drops the oldest whole segments of a partition, for every consumer group at
  once.
- `DELETE /api/v1/messages/:partitionId/:transactionId` removes a *dead-letter*
  row. On a queue served by the log engine it cannot remove a live message from a
  segment.
- Replay is likewise segment-granular. A `seek` to a timestamp lands on a segment
  boundary, not on the exact message.
- A partition is deleted only once it is empty *and* has been idle for
  `PARTITION_CLEANUP_DAYS` (30). Emptying it via retention is not enough, and the
  recreated lane is a new partition starting at offset 0. Never assume offset
  continuity across a month of silence.

### `maxWaitTimeSeconds` deletes data, including messages currently leased

Treat `maxWaitTimeSeconds` as an age-based shedding valve for data you are
willing to lose, never as a timeout mechanism. If a queue sets it, the
maintenance loop deletes **whole segments older than the cutoff, for every
consumer group, in-flight leases included**, regardless of whether
`retentionEnabled` is set. That is data loss by design. It does **not**
dead-letter what it drops, and a consumer whose cursor was inside the deleted
range resumes at the next surviving offset without an error: the pop scan
tolerates the gap. The eviction is counted in retention analytics; it is not
reported to the consumer that lost the messages.

## Dead letters are removed only when you remove them

`queen.log_dlq` grows until you clear it: no retention rule touches it, not the
retention window, not the completed-retention window, not
`maxWaitTimeSeconds`. Clearing is one address at a time, or wholesale by
deleting the queue, which deletes its dead-letter rows explicitly (the table has
no foreign key, so nothing cascades).

What you get for working through it is `GET /api/v1/dlq` to read, and per-address
routes to act on one message. A replay re-pushes the stored snapshot as a **new
message with a new identity at the tail of the partition**, then drops the
dead-letter row. It does not reinsert the message at its original position, and
redrive is per address, so reprocessing a large dead-letter backlog is a loop you
write, and a dead-letter table nobody works through keeps occupying disk.

## `/configure` is a full replace

`POST /api/v1/configure` writes every queue option from the body it receives, and
any option you omit is written back at its default. It is not a patch. Sending
`{"queue":"orders","options":{"leaseTime":600}}` to a queue that already had
retention configured switches retention off, and resets `dedupWindowSeconds` to
3600.

Always send the complete intended configuration. Related, because two defaults
circulate: `leaseTime` is 60 seconds on a queue created implicitly by a push, and
300 seconds once you have called `/configure` without overriding it.

## Transactions are one database transaction, not exactly-once

`POST /api/v1/transaction` bundles pushes and acks into one PostgreSQL
transaction, all-or-nothing. That is a real atomicity guarantee and it is not
end-to-end exactly-once delivery. If the response is lost and the client retries,
the work is done twice unless the `transactionId` is deterministic and still
inside the deduplication window.

Two related edges: deduplication is bounded by `dedupWindowSeconds` (default 3600)
and is exact only inside it, and `autoAck=true` commits the cursor inside the pop
transaction, which is at-most-once: a crash after the commit and before your
handler finishes loses the message.

## The broker expects a trusted network segment

It is built to run behind a terminator, on a network only your own services can
reach. The properties that follow from that:

- **No TLS listener.** The broker binds plain HTTP on `PORT`. Terminating TLS is
  the proxy's job (`QUEEN_PROXY_TLS_CERT` / `QUEEN_PROXY_TLS_KEY`) or a reverse
  proxy's. Broker-to-PostgreSQL TLS is separate and does exist (`PG_USE_SSL`,
  `PG_SSL_REJECT_UNAUTHORIZED`).
- **No CORS layer.** No middleware sets access-control headers, so a browser on
  another origin cannot call the broker directly.
- **JWT authentication is off by default** (`JWT_ENABLED=false`), and with it off
  the middleware passes every request through. Access levels are a role *set*,
  not a ladder: `WriteOnly` passes `/api/v1/push` and is rejected on every read
  and consume route.
- **The mesh port must be firewalled** if you run more than one broker. It is
  framed TCP; the HELLO handshake is HMAC'd, but the nonce is chosen by the
  dialer and never tracked, so a captured handshake is replayable, and
  post-handshake frames are unauthenticated JSON, including the frame that sets
  maintenance mode. This is a requirement, not a hardening tip.
- **`/metrics/prometheus` is not tenant-safe.** Per-queue series sum across
  tenants.
- **A malformed or wrong-length `QUEEN_ENCRYPTION_KEY` disables encryption** and
  stores plaintext on queues flagged for encryption, with only a warning in the
  log.

## The embedded engine is beta, and one per process

The broker can run inside a Rust application ([Embed the engine](/use/embed/)).
Its boundaries are its own: one `Broker` per process lifetime (the admission
arbiter is a process-wide singleton, and a start, shutdown, start cycle keeps
metering maintenance through the first instance's arbiter); shutdown is best
effort (engine loops without handles run until process exit, with the connection
pool closed under them); the outage spool defaults to a per-instance temp
directory, so `status: "buffered"` pushes do not survive a restart unless a
stable `spool_dir` is configured; and the surface is the data plane plus the
DLQ, with no consumer-group administration, listings, traces or streams. The
Rust API itself is beta: [Versions and compatibility](/reference/compatibility)
states what that means.

## Timers and KV state have their own boundaries

Both surfaces are part of the engine and every broker serves them: there is no variable that turns
them on, any more than there is one for push and pop, and the boundaries below always apply. What an
operator can do is pause a live surface with a runtime kill switch, which answers `503`, and bound a
tenant with grants and quotas, which answer `403`; both are on [KV and timers](/deploy/state). The
routes are on
[kv](/reference/http/kv) and [timers](/reference/http/timers); the mechanics are in
[Timer internals](/internals/timers), [KV internals](/internals/kv) and
[Sweeper internals](/internals/sweeper).

- **`deliverAt` is a floor, never an appointment.** A timer is delivered no earlier than the instant
  you name, and a wake-up that reaches only the broker that scheduled it makes the worst case on
  another broker one sleep ceiling away, `QUEEN_SWEEPER_MAX_SLEEP_MS`, which defaults to one second.
  There is no per-message deadline anywhere in Queen, and this is not one.
- **There are no recurring timers.** One timer is one delivery. A repeating schedule is a consumer
  that schedules the next one, which keeps the cancellation story to a single row.
- **A cancel after the fire answers `absent`.** The fire deletes the staging row in the same
  transaction that pushes the message, so there is no tombstone afterwards. `absent` means **no longer
  pending**. It never means "not delivered", and a compensation path has to check state rather than
  read a cancel result as a verdict.
- **Every KV write carries an expiry.** Exactly one of a TTL in seconds or an explicit `forever` is
  required on every write; zero or two of them is an error. A store with no natural retention whose
  TTL is optional grows in silence, and `forever` is deliberately something you have to type. This is
  also the difference from streams state, which is scoped to a partition and has no expiry at all.
- **The KV read boundary is a key or a prefix**, both capped, and prefix reads exist only on the
  batch endpoint. There is no query language, no secondary index over values, no scan of a whole
  namespace, and no cross-namespace read.
- **A quota on either surface is soft and late.** Enforcement is an in-process count refreshed from
  a rollup, because an exact count would mean one row per tenant locked by every write, in the
  outermost lock space, held for a whole transaction. Overshoot is bounded, not zero.

## Smaller edges

- **Payloads must be JSON.** The push path parses each payload as a JSON value.
  Opaque binary is not a payload type. Encode it yourself. The whole request
  body is capped by `QUEEN_MAX_BODY_BYTES` (default 64 MiB).
- **Every `204` has no body at all**, deliberately. An empty pop is a bare status
  code.
- **Consumers poll.** Long-polling parks a pop and wakes it, but there is no
  server-initiated push and no persistent subscription stream.
- **Retention cannot delete stream state, and a partition carrying open
  accumulators is never reclaimed.** `queen_streams.state` keys on
  `partition_id` and declines the foreign key on purpose, so partition cleanup
  checks it explicitly and refuses to delete: it is one of three vetoes, with
  dead-letter rows and a live lease ([Retention](/internals/retention)). The
  limitation that follows is that accumulators have no TTL. Nothing time-based
  purges `queen_streams.queries` or `queen_streams.state`: state goes only when
  its query is reset or deleted (the foreign key cascades), or when a tenant
  wipe removes the tenant's streams rows wholesale. All three `queen_streams`
  tables are `GRANT`ed to `PUBLIC`.
- **The bundled dashboard and JWT auth do not combine broker-direct.** With auth
  off the broker serves a fully working dashboard as an anonymous operator; with
  `JWT_ENABLED=true` it serves an explanation page instead, because the app
  sends no bearer token and the broker holds no sessions. A dashboard with
  logins and roles is `queen_proxy`'s.
- **The broker needs a session it keeps.** Schema apply (advisory lock
  `778120010`) and the retention leader election (`737001`) hold session-scoped
  advisory locks, which live on one backend session. **Transaction** pooling
  breaks that and is the only pooling mode that does; **session** pooling keeps
  the session intact and is supported, which is how PgBouncer is normally put in
  front. The residual risk in session mode is a pooler that recycles the server
  connection underneath a live session, dropping a lock the broker still holds:
  it surfaces as `pg_advisory_unlock reported not-held` in the retention log, and
  connecting directly is what rules it out.
- **The broker does not serve an OpenAPI document at runtime.** The
  [OpenAPI 3.1 documents](/reference/openapi) published with this site are
  generated from the router at documentation build time, not by the broker.
  There is also no data-migration tool: the binary's `migrate` subcommand is a
  retired stub that exits non-zero without touching the database. Backup is
  `pg_dump` and an ordinary PostgreSQL restore.
- **SDKs exist for JavaScript, Python, Go, Rust, PHP/Laravel and C++**, plus a Go
  operator CLI. There is no Java or .NET client. Those languages use the
  HTTP API directly.

## Where another tool fits better

| If you need | Where Queen stands |
| --- | --- |
| Total ordering across an entire topic or system | Ordering is per partition. One partition gives you total order and one writer's worth of throughput |
| Message priority, deadlines, or reordering | Delivery is arrival order within a partition, with no override. `delayedProcessing` and `windowBuffer` shift when a whole lane becomes visible, and reorder nothing |
| Many workers sharing one ordered stream | One in-flight leased batch per `(partition, group)` is what makes per-entity FIFO true. Parallelise by choosing a partition key with more distinct values: lanes are unbounded in the broker and created by the push that names them |
| A broker that keeps serving while the database is down for hours | Only pushes survive, into a disk spool bounded by your disk; nothing is consumable until PostgreSQL returns |
| A managed service with no database to operate | Queen is a binary you run against a PostgreSQL you run. Its ceiling and its failure domain are that instance |
| Per-tenant quotas, plans and usage metering | Shipped, in the Apache-2.0 gateway in `proxy/`, not in the broker |
| Opaque binary or very large payloads | Payloads are JSON, inside a request-body cap |
| A broker you can expose directly to untrusted clients | No TLS listener, no CORS, authentication off by default, and a mesh port that must be firewalled |
| A Java or .NET SDK today | Only six language SDKs exist; everything else is plain HTTP |
| Nothing but per-key ordering at very high cardinality, at a modest rate | Queen's per-lane cursor is what buys consumer groups, replay, per-group lag and a retry budget; a workload that needs none of those does not need the cursor, and pgmq serves it with no per-lane state at all |

The measured crossover behind that last row, in both directions, is in
[Cross-broker comparison](/benchmarks/comparison).

If none of those is disqualifying, the story behind the trade-offs is in
[History](/start/history), the architectural comparison is in
[Comparison](/start/compare), and the shortest path to a running broker is the
[Quickstart](/start/quickstart).

Source: https://queenmq.com/reference/limits/index.mdx