---
title: "Model"
description: "The whole model in one page: partitions created by the push that names them, one cursor per consumer group, leases, transactions, replay and retention."
---

> Queen MQ documentation, for AI agents
> Complete self-contained summary of Queen MQ: https://queenmq.com/llms-brief.txt
> Fetch that first when the question is about the product rather than about this page.
> Index of all pages: https://queenmq.com/llms.txt

# Model

Queen stores a queue as a set of ordered partitions, and a consumer group as one cursor per
partition. What a push creates, what an ack means and how far back you can replay all follow
from those two sentences.

## Queues and partitions

A queue is a named container with a configuration. A partition is an ordered lane inside it.
Neither is declared before use: a push to a queue and a partition that do not exist creates
both, in the transaction that stores the message.

```js title="clients/client-js/test-v2/docs.js"
const res = await client
  .queue('orders')
  .partition('customer-42')
  .push([{ data: { orderId: 9137, amount: 99.5 } }])
```

That call created the queue, the lane and the message; a push that names no partition lands in
`Default`. Partition by the entity whose order you care about, a customer or a device or a
document: one entity, one lane, and a slow lane holds up nothing outside it.

A partition costs one row, plus one row per group that reads it. Tens of thousands of lanes are
ordinary and [a million](/benchmarks/cardinality-1m) has been served at 200,000 messages a
second; a partition per message is not. A dotted queue name fills the `namespace` and `task`
labels a [discovery pop](/reference/http/pop) matches on.

## Producing and deduplication

Producing is one route, `POST /api/v1/push`, taking a list of items. Every item names its own
queue and partition, so one request can write to several of both.

```js title="clients/client-js/test-v2/docs.js"
const first = await client
  .queue('payments')
  .partition('customer-42')
  .push([{ transactionId: 'order-9137-paid', data: { orderId: 9137, amount: 99.5 } }])

const retry = await client
  .queue('payments')
  .partition('customer-42')
  .push([{ transactionId: 'order-9137-paid', data: { orderId: 9137, amount: 99.5 } }])
// retry[0].status is 'duplicate': the second push wrote nothing
// and answers with the first message's id.
```

Deduplication is on by default with a window of 3600 seconds, and it is exact, not
probabilistic: the probe runs inside PostgreSQL under the row lock that allocates offsets, so a
duplicate writes nothing. It is scoped to one partition, `dedupWindowSeconds: 0` turns it off,
and the key only works if you derive it from the work: an upstream event id, a primary key plus
a version. A fresh UUID per attempt deduplicates nothing.

The response is `201` with one verdict per item, and `201` does not mean stored: `queued` is
committed, `duplicate` wrote nothing and returns the original id, `buffered` means PostgreSQL
was unreachable and the item waits on the broker's disk spool, `failed` means neither. Read
every item's `status`.

## Consuming, groups and subscription modes

Consuming is a `GET` that claims a batch under a lease. No subscribe call, no broker-assigned
partitions, no membership protocol, so a restarting worker stalls nobody.

```js title="clients/client-js/test-v2/docs.js"
await client
  .queue('orders')
  .group('billing')
  .subscriptionMode('all')
  .limit(1)
  .each()
  .consume(async (message) => {
    console.log(message.data)
  })
```

A consumer group is a name with one cursor per partition, so two groups read the same messages
at their own pace. Omitting the group puts you in the implicit `__QUEUE_MODE__`, the
competing-consumers shape: one message, one consumer, and the backlog is never skipped.

Where a named group starts is decided on its first contact with the queue and stored: `new`, the
default, skips the backlog; `all` delivers everything retained. Passing the mode later changes
nothing; moving a group that has read is seek's job.

`wait=true` parks the pop instead of returning empty; a push wakes it in milliseconds. An empty
pop answers `204` with no body, so never parse it. Batches, timeouts and delayed visibility are
in [defaults](/reference/defaults) and [queue options](/reference/queue-options).

You do not have to size the pop yourself. Leave the batch and the partition width unset and the
broker chooses both, per pop, from what it can see and you cannot: how many partitions of your
group hold work right now, and how long the oldest has been waiting. The right width is a fact
about the broker rather than about your consumer, and it moves, because a queue's partition count
grows for years. What you do set stays yours: the choice is per knob, so pinning the width to one
partition and letting the broker size the batch is an ordinary thing to ask for, and the pinned
half is never adjusted. Every SDK exposes an off switch that restores its old client-side defaults
exactly. See [pop autopilot](/reference/http/pop#autopilot).

One rule shapes every deployment: one leased batch per (partition, group). Parallelism inside a
group is bounded by partitions, not workers: twenty workers on a single-partition queue is one
worker's throughput and nineteen idle pollers. Add lanes, not workers.

## Acknowledgement is an offset commit

Consumption state is one number per (partition, group): `committed`, the last offset that group
acknowledged; the next message it wants is `committed + 1`. Nothing per message exists anywhere,
so acking message N states that everything up to N is done, mentioned or not.

Acks are addressed by `transactionId` plus `partitionId`, and always answer `200`: the per-item
`success` is the result, not the status line.

```json
[
  { "transactionId": "m1", "status": "completed" },
  { "transactionId": "m2", "status": "failed" },
  { "transactionId": "m3", "status": "completed" }
]
```

**Figure.** A partition holding offsets 6 to 9. Offset 6 is already committed; 7 is m1, acknowledged completed; 8 is m2, acknowledged failed; 9 is m3, acknowledged completed. The cursor moves from 6 to 7 and stops there, below the failure at 8, even though 9 was acknowledged completed.

m3 was acknowledged completed and the cursor still does not reach it: the failure at m2 clamps it below. The group's next pop starts at m2.

That call commits past `m1` only. The lowest explicit `failed`, `retry` or `dlq` clamps the
cursor below itself and a completed ack above it does not survive; offsets you never mention are
completed silently. A duplicate delivery is recoverable, a swallowed failure is not.

Acking implicitly is two things. The SDK consume loops ack after your handler returns and nack
when it throws, still a leased at-least-once pop; the server-side `autoAck=true` commits the
cursor in the transaction that reads the messages, which is at-most-once.

## Leases, retries and the dead-letter queue

A leased pop removes nothing. It writes a holder, an expiry and the last delivered offset onto
the (partition, group) row; until an ack releases the claim or it expires, no other consumer in
the group takes that partition.

**Figure.** The same partition. Offsets 7, 8 and 9 are held under a lease, bracketed above the axis, while the committed cursor stays at offset 6.

A lease is a claim, not a consumption: it covers 7 to 9 while committed stays at 6. Nothing was removed, so an expiry redelivers the whole span from committed + 1.

```http
POST /api/v1/ack                     # completed, retry, failed or dlq
POST /api/v1/lease/:leaseId/extend
GET  /api/v1/dlq
```

Lease time comes from `leaseSeconds` on the pop, then the queue's `leaseTime`, then 60 seconds.
Nothing sweeps: the next pop finds the dead lease and redelivers the whole un-acked span from
`committed + 1`, messages your handler finished but never acked included. Expiry charges no
retry budget, so a crash-looping consumer redelivers forever without ever dead-lettering.

The budget is one counter per (partition, group) against `retryLimit`, default 3: a `failed` ack
spends one and redelivers, `retry` redelivers without spending, `dlq` files the message at once.
When it runs out the head message is copied into a real dead-letter table with its payload and
the reason, or dropped if you turned dead-lettering off.

Dead-letter rows survive retention and leave one address at a time, by delete or by replay under
a fresh transaction id, so an unattended one grows without bound
([messages and DLQ](/reference/http/messages-dlq)).

## Transactions

`POST /api/v1/transaction` bundles pushes and acks into one PostgreSQL transaction: everything
in the call commits together or nothing does. It exists for the pipeline handoff, an ack of the
input and a push of the output that cannot come apart.

**The bundle is N to M, not one to one.** One call may acknowledge batches leased from any number
of partitions, across any number of queues and consumer groups, and push to any number of queues
and partitions, all in the same commit: that is what makes a fan-in stage possible
([the bundle shape](/reference/http/transaction/)).

```js title="clients/client-js/test-v2/docs.js"
await client
  .queue('orders')
  .group('invoicing')
  .subscriptionMode('all')
  .each()
  .autoAck(false) // the acknowledgement belongs to the transaction, not to the loop
  .limit(1)
  .idleMillis(5000)
  .consume(async (message) => {
    // commit() throws when the broker rejects the bundle, so reaching the
    // line after it means the ack and the push are both durable.
    await client
      .transaction()
      .queue('invoices')
      .push([{ data: { orderId: message.data.orderId, invoiced: true } }])
      .ack(message, 'completed', { consumerGroup: 'invoicing' })
      .commit()
  })
```

There is no window in which the output exists without the input being consumed, or the reverse,
and no ordering of two calls buys that. Any failure in the call, a duplicate push included,
rolls back every operation and the input redelivers; the route answers `200` even then, so read
`success` in the body.

Atomicity covers the broker's state, not the network: a blind retry after a lost response
duplicates the pushes unless their transaction ids are deterministic and the deduplication
window covers your retry horizon.

**The bundle is not only pushes and acks.** Two more arrays ride the same commit when their surfaces
are switched on: `kv`, which writes [transactional state](/use/kv), and `timers`, which schedules or
cancels a [timer](/use/timers). Both are **top-level fields of the request body**, beside `operations`
and never inside it, and a bundle that carries neither is byte for byte the call it was before they
existed. That is what makes an idempotency marker, an external effect and the cursor advance a single
commit instead of three, and it is the one thing a key/value store or a scheduler standing beside the
broker cannot offer at any price.

## Replay

Consuming does not remove a message, so replay is arithmetic on a cursor. Seek moves one
group's cursor, per queue or per partition, backwards or forwards, to a timestamp or to the
end.

```js title="clients/client-js/test-v2/docs.js"
// Move the audit group's cursor back one hour. The seek also releases
// any live lease, so an in-flight batch is abandoned, not acked.
await client.admin.seekConsumerGroup('audit', 'orders', {
  timestamp: new Date(Date.now() - 3600 * 1000).toISOString(),
})
```

A seek releases any live lease, abandoning an in-flight batch instead of acking it, and resets
the retry budget. It touches exactly one group: everyone else keeps their position.

A timestamp lands on a segment boundary rather than an exact message, so you also get messages
committed slightly before the instant you asked for: seek a little early and filter in the
handler. Retention is the floor, and deleted data cannot be replayed by any means.

**Figure.** A partition whose offsets 2 and 3 have been deleted by retention and are drawn as dashed empty cells. A dashed vertical line at offset 4 marks log_start. The billing cursor sits at 7 before the seek and at the log_start boundary after it.

Seek is arithmetic on one group's cursor, and log_start is where the arithmetic stops: 2 and 3 are gone, so no seek reaches them.

## Retention

A queue has no retention until you configure it. Consuming does not delete and by default
nothing else does either: the retained log is the replay window, so an unattended queue grows
until you decide otherwise.

```json
{
  "queue": "orders",
  "options": {
    "retentionEnabled": true,
    "retentionSeconds": 604800,
    "completedRetentionSeconds": 3600,
    "maxWaitTimeSeconds": 0,
    "dedupWindowSeconds": 3600,
    "leaseTime": 300,
    "retryLimit": 3
  }
}
```

`retentionSeconds` deletes on age alone, consumed or not, so a consumer down longer than the
window comes back to a shorter log. `completedRetentionSeconds` never passes the slowest group's
cursor, so one abandoned group holds data for every other. Deletion is whole-segment: a window
is a floor on what is kept, not a ceiling on what is deleted.

One option deletes data out from under a running worker: `maxWaitTimeSeconds` drops segments on
age alone, leased or not. Use it where staleness makes a message worthless, never as a backstop
for slow consumers.

`POST /api/v1/configure` is a full replace: every omitted key returns to its default, the
deduplication window included. Send the whole set ([option table](/reference/queue-options)).

## Delivery guarantees

Delivery on a leased pop is at-least-once: the cursor moves only on ack, so nothing is lost and
some things arrive twice, on lease expiry, on any nack, after a broker restart, and on a
dead-letter replay. Ordering is total per partition in commit order and nothing more: no global
order, no priority. Durability is PostgreSQL's, which is why `buffered` deserves an alert. Make
handlers idempotent on `transactionId` and the duplicates stop mattering.

Two failures change the code you write. A `429` is backpressure and carries `Retry-After` in
seconds: the SDKs honour it and retry in place, unbounded on a long-poll pop and bounded
elsewhere, while 5xx and network failures get `retryAttempts` tries with backoff and any other
4xx stops the loop. A `403` or a `413` is terminal: fix the credential or reshape the request
([errors](/reference/errors)).

One lane per entity, one cursor per group, one integer of state for each pair. The rest is
arithmetic.

Source: https://queenmq.com/use/model/index.mdx