---
title: "Everything QueenMQ does, on one map"
description: "Every part of QueenMQ as a yellow box and every option as a dot: hover or click one to read its setting, its default, the code and when to use it."
---

> QueenMQ documentation, for AI agents
> Complete self-contained summary of QueenMQ: https://queenmq.com/llms-brief.txt
> Fetch that first when the question is about the product rather than about this page.
> Index of all pages: https://queenmq.com/llms.txt

# Everything QueenMQ does, on one map

## Push

Write messages. The first push that names a queue and a partition creates them: there is no partition count to choose.

**When to use it:** Every producer starts here.

**Other names:** Kafka: produce, producer; RabbitMQ: publish; Amazon SQS: SendMessage; Redis: XADD.

```js
await queen.queue('orders')
  .partition('customer-42')       // one ordered partition per customer, created by this push
  .push([{
    transactionId: 'order-1001',  // the dedup key: pushing the same id again is a no-op
    data: { orderId: 1001, customer: 42, total: 59.9 },
  }])
```

[Read it in the docs](/writing-applications/push/)

### Partition

A partition is one entity: a customer, an order, a conversation. Order holds inside it, and a slow entity holds up no other. A push with no partition goes to `Default`.

| Setting | Default |
| --- | --- |
| `.partition(name)` | Default |

**When to use it:** Name it after the entity whose events must stay in order, like `customer-42`.

**Other names:** Kafka: message key, partition key; Amazon SQS: FIFO queue, message group ID; Azure Service Bus: session; Google Pub/Sub: ordering key.

```js
await queen.queue('orders')
  .partition('customer-42')       // one ordered partition per customer, created by this push
  .push([{
    transactionId: 'order-1001',  // the dedup key: pushing the same id again is a no-op
    data: { orderId: 1001, customer: 42, total: 59.9 },
  }])
```

[Read it in the docs](/writing-applications/push/)

### Dedup id

A message's `transactionId`. The same id in the same partition within the dedup window is not written again: the push answers `duplicate`.

| Setting | Default |
| --- | --- |
| `dedupWindowSeconds` | 3600 s, set on the queue; at most 7 days, 0 turns it off |

**When to use it:** Derive it from the work (`receipt-${orderId}`), never a fresh UUID: then a retry after a timeout is safe.

**Other names:** Amazon SQS: message deduplication ID; Kafka: idempotent producer; NATS JetStream: Nats-Msg-Id; Azure Service Bus: duplicate detection.

```js
await queen.queue('orders')
  .partition('customer-42')       // one ordered partition per customer, created by this push
  .push([{
    transactionId: 'order-1001',  // the dedup key: pushing the same id again is a no-op
    data: { orderId: 1001, customer: 42, total: 59.9 },
  }])
```

[Read it in the docs](/writing-applications/push/)

### Per-message answer

One push carries up to 10,000 messages, and the broker answers each one: `queued`, `duplicate` or `error`. The HTTP status is 201 even when some are refused.

| Setting | Limit |
| --- | --- |
| `messages per push` | up to 10,000 |

**When to use it:** Read each message's status: retry only what was refused, and take a duplicate as an earlier attempt that got through.

**Other names:** RabbitMQ: publisher confirms; Kafka: delivery report.

```js
await queen.queue('emails')
  .partition('customer-42')
  .push([{ transactionId: 'welcome-42', data: { to: 'ada@example.com' } }])
  .onSuccess((items) => console.log('stored', items.length))
  .onDuplicate((items) => console.log('already stored', items[0].transactionId))
  .onError((items, error) => console.error('refused', error.message))
```

[Read it in the docs](/writing-applications/push/#every-message-gets-its-own-answer)

### Buffered push

The client collects messages and sends them in batches: when a buffer holds `messageCount` messages, or when `timeMillis` has passed.

| Setting | Default |
| --- | --- |
| `.buffer({ messageCount, timeMillis })` | off |
| `bufferMaxMessages` | 100,000 per client (JavaScript) |

**When to use it:** Thousands of small messages a second. A crash loses what is still in the buffer, so call `flushAllBuffers()` before the process exits.

**Other names:** Kafka: linger.ms, batch.size.

```js
for (let i = 0; i < 1000; i++) {
  await queen.queue('events')
    .partition(`device-${i % 10}`)
    .buffer({ messageCount: 100, timeMillis: 1000 })  // 100 at a time, or every second
    .push([{ transactionId: `reading-${i}`, data: { device: i % 10, value: i } }])
}
await queen.flushAllBuffers()                         // before exiting, send what is left
```

[Read it in the docs](/writing-applications/push/#buffering-for-many-small-messages)

### Binary push

A push travels as one binary body instead of JSON; with compression the client packs and zstd-compresses each partition's batch. A framed pop reads the records as stored.

| Setting | Default |
| --- | --- |
| `binaryPush` | false |
| `pushCompression` | false, needs binaryPush |
| `framedPop` | false |

**When to use it:** A high message rate, when the broker's CPU or the network is the limit. Otherwise leave them off: JSON is easier to read when you debug.

**Other names:** Kafka: compression.type.

```js
const fast = new Queen({
  url: 'http://localhost:6632',
  binaryPush: true,       // a push travels as one binary body, not JSON
  pushCompression: true,  // with binaryPush: each partition's batch is zstd-compressed here
  framedPop: true,        // a pop gets the records as stored, and unpacks them here
})

await fast.queue('invoices').partition('customer-42')
  .push([{ transactionId: 'invoice-2001', data: { orderId: 2001, total: 12.5 } }])

await fast.queue('invoices').each().limit(1).consume(async (msg) => {
  console.log(msg.data)           // { orderId: 2001, total: 12.5 }, as with JSON
})
```

[Read it in the docs](/writing-applications/push/#binary-push-compression-and-framed-pop)

### Headers (new in 2.9)

A small map of strings stored in clear beside the payload: the message's type, its trace context, its source. A `where` reads them without reading the payload.

| Setting | Limit |
| --- | --- |
| `headers` | up to 64 per message, 8 KiB in all |
| `names` | a-z 0-9 . _ - (queen- is reserved) |

**When to use it:** Route and trace on a header, even on an encrypted queue. No secret in a header: it is never encrypted.

**Other names:** Kafka: record headers; RabbitMQ: message properties; Amazon SQS: message attributes; Google Pub/Sub: attributes; Azure Service Bus: application properties.

```js
await queen.queue('orders').partition('order-9137').push([{
  transactionId: 'order-9137-paid',
  data: { orderId: 9137, total: 120 },
  headers: { type: 'order.paid', traceparent: '00-4bf92f3577b34da6a3ce929d0e0e4736-00f067aa0ba902b7-01' },
}])
```

[Read it in the docs](/writing-applications/headers/)

## Queue

A named set of partitions that share one configuration. A push creates it with the defaults; configure it when you want others.

**When to use it:** Always configure a queue before its first push: the defaults surprise. A queue a push creates leases for 60 s, one from `create()` for 300 s, and retention is off, so it keeps everything.

**Other names:** Kafka: topic; Google Pub/Sub: topic; NATS JetStream: stream; Redis: stream.

```js
import { Queen } from 'queen-mq'

const queen = new Queen({ url: 'http://localhost:6632' })

await queen.queue('orders')
  .namespace('shop')              // groups queues in the dashboard
  .config({
    leaseTime: 30,                // seconds a worker holds a message before it is redelivered
    retryLimit: 3,                // failures before the message goes to the dead-letter queue
    retentionEnabled: true,       // retention runs only when this is on
    retentionSeconds: 86400,      // then messages older than a day are removed
  })
  .create()
```

[Read it in the docs](/writing-applications/queues/)

### Lease

How long a worker holds the messages it took before they go to another worker. Long work renews the lease while it runs.

| Setting | Default |
| --- | --- |
| `leaseTime` | 60 s (300 s from create()) |
| `.renewLease(true, ms)` | off |

**When to use it:** Short enough that a crashed worker's messages move on quickly; renew it for slow but healthy work.

**Other names:** Amazon SQS: visibility timeout; Azure Service Bus: lock duration, peek-lock; Google Pub/Sub: ack deadline; NATS JetStream: AckWait; RabbitMQ: consumer timeout.

```js
const stop = new AbortController()
process.once('SIGTERM', () => stop.abort())     // a deploy stops the worker cleanly

await queen.queue('tasks')
  .group('workers')
  .subscriptionMode('all')
  .concurrency(4)                               // four handlers at once, on four partitions
  .renewLease(true, 10000)                      // long tasks keep their lease alive
  .idleMillis(60000)                            // or stop after a minute with nothing to do
  .each()
  .consume(async (task) => runTask(task.data), { signal: stop.signal })
```

[Read it in the docs](/writing-applications/pop/#leases-and-long-work)

### Delay

Every new message waits this long after its push before anyone can take it.

| Setting | Default |
| --- | --- |
| `delayedProcessing` | 0 s |

**When to use it:** The same delay for every message of a queue. For one message at a time, use a timer.

**Other names:** Amazon SQS: delay queue, DelaySeconds.

```js
await queen.queue('jobs')
  .config({
    leaseTime: 120,                   // 2 minutes per job, then it goes to another worker
    retryLimit: 5,                    // after 5 failures, the dead-letter queue
    retryDelay: 10000,                // a failed job comes back after 10 s, not at once
    delayedProcessing: 30,            // a new job waits 30 s before anyone can take it
    windowBuffer: 5,                  // a burst waits 5 s, then is handed over at once
    dedupWindowSeconds: 86400,        // a transactionId is remembered for a day
    retentionEnabled: true,           // retention runs only when this is on:
    completedRetentionSeconds: 3600,  //   jobs every group has finished go after an hour
    maxWaitTimeSeconds: 604800,       // anything older than a week goes, finished or not
  })
  .create()
```

[Read it in the docs](/writing-applications/queues/#every-option)

### Burst window

New messages wait in their partition this long, so a burst is handed over as one batch.

| Setting | Default |
| --- | --- |
| `windowBuffer` | 0 s |

**When to use it:** A guest sends three chat messages in a row, and the assistant answers all three at once.

**Other names:** Also called: debounce.

```js
// A burst waits 2 seconds, then goes to the handler as one batch.
await queen.queue('chat').config({ windowBuffer: 2 }).create()

// The guest sends three messages in a row.
for (const text of ['Hi!', 'Is breakfast included?', 'And is there parking?']) {
  await queen.queue('chat').partition('conv-42').push([{ data: { text } }])
}

// The assistant answers the whole burst once.
await queen.queue('chat')
  .group('assistant')
  .subscriptionMode('all')
  .batch(10)                          // the handler gets every message of the burst
  .consume(async (messages) => {
    question = messages.map((m) => m.data.text).join(' ')
    console.log(question)             // Hi! Is breakfast included? And is there parking?
  })
```

[Read it in the docs](/patterns/ordering-batching/)

### Retries

How many failed attempts a message gets, and how long it waits before each new one. Its partition waits with it, for that group.

| Setting | Default |
| --- | --- |
| `retryLimit` | 3 |
| `retryDelay` | 1000 ms |

**When to use it:** Temporary failures heal by themselves; the limit stops a message that never succeeds from blocking its partition.

**Other names:** Amazon SQS: maxReceiveCount, redrive policy; RabbitMQ: x-delivery-limit; Azure Service Bus: MaxDeliveryCount; Google Pub/Sub: max delivery attempts; NATS JetStream: MaxDeliver.

```js
await queen.queue('jobs')
  .config({
    leaseTime: 120,                   // 2 minutes per job, then it goes to another worker
    retryLimit: 5,                    // after 5 failures, the dead-letter queue
    retryDelay: 10000,                // a failed job comes back after 10 s, not at once
    delayedProcessing: 30,            // a new job waits 30 s before anyone can take it
    windowBuffer: 5,                  // a burst waits 5 s, then is handed over at once
    dedupWindowSeconds: 86400,        // a transactionId is remembered for a day
    retentionEnabled: true,           // retention runs only when this is on:
    completedRetentionSeconds: 3600,  //   jobs every group has finished go after an hour
    maxWaitTimeSeconds: 604800,       // anything older than a week goes, finished or not
  })
  .create()
```

[Read it in the docs](/writing-applications/retries/)

### Dead-letter queue

Where a message goes after its last retry, with its error, so its group moves on. With both settings off, the message is dropped instead.

| Setting | Default |
| --- | --- |
| `deadLetterQueue` | true |
| `dlqAfterMaxRetries` | true |

**When to use it:** Keep both on: a message that fails for good is parked, not lost, and you can replay it once the cause is fixed.

**Other names:** RabbitMQ: dead letter exchange, DLX; Amazon SQS: dead-letter queue; Google Pub/Sub: dead-letter topic; Kafka: dead letter topic; Also called: dead letter channel.

[Read it in the docs](/writing-applications/retries/)

### Dedup window

How long a partition remembers `transactionId`s: a second push with the same id in that time writes nothing and answers `duplicate`.

| Setting | Default |
| --- | --- |
| `dedupWindowSeconds` | 3600 s; at most 7 days; 0 turns it off |

**When to use it:** Cover the time in which your producers may retry a push.

**Other names:** Amazon SQS: deduplication interval; Azure Service Bus: duplicate detection window; NATS JetStream: duplicate window.

```js
await queen.queue('jobs')
  .config({
    leaseTime: 120,                   // 2 minutes per job, then it goes to another worker
    retryLimit: 5,                    // after 5 failures, the dead-letter queue
    retryDelay: 10000,                // a failed job comes back after 10 s, not at once
    delayedProcessing: 30,            // a new job waits 30 s before anyone can take it
    windowBuffer: 5,                  // a burst waits 5 s, then is handed over at once
    dedupWindowSeconds: 86400,        // a transactionId is remembered for a day
    retentionEnabled: true,           // retention runs only when this is on:
    completedRetentionSeconds: 3600,  //   jobs every group has finished go after an hour
    maxWaitTimeSeconds: 604800,       // anything older than a week goes, finished or not
  })
  .create()
```

[Read it in the docs](/writing-applications/queues/#every-option)

### Retention

How long messages are kept. Consuming does not delete: a queue is a log, kept for its retention.

| Setting | Default |
| --- | --- |
| `retentionEnabled` | false |
| `retentionSeconds` | 0: older than this goes, consumed or not |
| `completedRetentionSeconds` | 0: older than this goes, once every group acked it |
| `maxWaitTimeSeconds` | 0: works without retentionEnabled |

**When to use it:** Keep the history long enough to read it again, and no longer than the disk allows.

**Other names:** Kafka: retention.ms; Amazon SQS: MessageRetentionPeriod; Google Pub/Sub: message retention duration; NATS JetStream: MaxAge; RabbitMQ: max-age.

```js
await queen.queue('jobs')
  .config({
    leaseTime: 120,                   // 2 minutes per job, then it goes to another worker
    retryLimit: 5,                    // after 5 failures, the dead-letter queue
    retryDelay: 10000,                // a failed job comes back after 10 s, not at once
    delayedProcessing: 30,            // a new job waits 30 s before anyone can take it
    windowBuffer: 5,                  // a burst waits 5 s, then is handed over at once
    dedupWindowSeconds: 86400,        // a transactionId is remembered for a day
    retentionEnabled: true,           // retention runs only when this is on:
    completedRetentionSeconds: 3600,  //   jobs every group has finished go after an hour
    maxWaitTimeSeconds: 604800,       // anything older than a week goes, finished or not
  })
  .create()
```

[Read it in the docs](/writing-applications/queues/#every-option)

### Encryption

Payloads are encrypted on disk. Headers stay in clear, so routing still reads them.

| Setting | Default |
| --- | --- |
| `encryptionEnabled` | false |
| `QUEEN_ENCRYPTION_KEY` | required on every node |

**When to use it:** Queues that carry personal data.

**Other names:** Amazon SQS: server-side encryption, SSE; Also called: encryption at rest.

[Read it in the docs](/writing-applications/queues/#every-option)

### Labels

`namespace` and `task` group queues in the dashboard and in a discovery pop, one pop over every queue of a namespace or a task.

| Setting | Default |
| --- | --- |
| `.namespace(name)` | from the queue's name |
| `.task(name)` | from the queue's name |

**When to use it:** Find a team's queues at a glance, and pop from all of them in one call.

```js
import { Queen } from 'queen-mq'

const queen = new Queen({ url: 'http://localhost:6632' })

await queen.queue('orders')
  .namespace('shop')              // groups queues in the dashboard
  .config({
    leaseTime: 30,                // seconds a worker holds a message before it is redelivered
    retryLimit: 3,                // failures before the message goes to the dead-letter queue
    retentionEnabled: true,       // retention runs only when this is on
    retentionSeconds: 86400,      // then messages older than a day are removed
  })
  .create()
```

[Read it in the docs](/writing-applications/queues/#every-option)

### Reading queue (new in 2.9)

A queue whose messages come from other queues, with a `where` and partitions of its own. It holds links of about 20 bytes, not copies.

| Setting | Default |
| --- | --- |
| `from` | none: a queue, or a list of them |
| `where` | every message |
| `partitionBy` | same |
| `start` | now |

**When to use it:** A reader that needs settings of its own, partitions of its own or several sources.

**Other names:** RabbitMQ: exchange, binding, topic exchange; Amazon SNS: filter policy; Azure Service Bus: auto-forwarding, subscription filter; Also called: router, content-based router.

```js
await queen.queue('orders-it').config({
  from: 'orders',
  where: "data.country = 'IT'",
  partitionBy: 'same',
  leaseTime: 30,
  retryLimit: 5,
}).create()
```

[Read it in the docs](/writing-applications/routing/#reading-queues)

## Connectors

The broker copies data between QueenMQ and the systems around it: database tables into queues (sources), queues into tables and buckets (sinks), and Kafka clients through the Kafka port. No other process runs.

| Setting | What it needs |
| --- | --- |
| `PUT /api/v1/connectors/:name` | one call per source or sink |
| `QUEEN_ENCRYPTION_KEY` | required: a database password is stored sealed with it |

**When to use it:** Get a table's changes out without Debezium and Kafka Connect, keep a table or a bucket in step with a queue, keep your Kafka clients while you move.

**Other names:** Kafka: Kafka Connect, connector; Also called: CDC.

[Read it in the docs](/integrations/)

### PostgreSQL source

Reads committed changes through logical replication: every insert, update and delete becomes a message, with exactly-once effects.

| Setting | What it needs |
| --- | --- |
| `PostgreSQL` | 17 or later |
| `wal_level` | logical |

**When to use it:** Keep a cache, a search index or another service in step with a table.

**Other names:** Kafka: Debezium, source connector; Also called: CDC, change data capture.

```bash
curl -s -X PUT localhost:6632/api/v1/connectors/orders-src \
  -H 'content-type: application/json' -d '{
  "kind": "source",
  "connection": {"host": "pg", "database": "postgres", "user": "postgres",
                 "password": "secret", "sslMode": "disable"},
  "source": {"tables": [{"table": "public.orders", "queue": "orders"}]}
}'
```

[Read it in the docs](/integrations/postgres/#the-source)

### MySQL source (new in 2.9)

Reads the binlog the way a replica does: every committed row change becomes a message, with exactly-once effects.

| Setting | What it needs |
| --- | --- |
| `MySQL` | 8.0 or later, not MariaDB |
| `binlog_row_metadata` | FULL |
| `gtid_mode` | ON |

**When to use it:** The same as the PostgreSQL source, for a MySQL table.

**Other names:** Kafka: Debezium, source connector; Also called: CDC, change data capture.

[Read it in the docs](/integrations/mysql/#the-source)

### PostgreSQL sink

A consumer group the broker runs for you, writing every message into a table.

| Setting | How |
| --- | --- |
| `PUT /api/v1/connectors/:name` | one call per sink |

**When to use it:** Replace the consumer you would write to keep a table in step with a queue.

**Other names:** Kafka: JDBC sink connector, sink connector.

```bash
curl -s -X PUT localhost:6632/api/v1/connectors/orders-copy \
  -H 'content-type: application/json' -d '{
  "kind": "sink",
  "connection": {"host": "pg", "database": "postgres", "user": "postgres",
                 "password": "secret", "sslMode": "disable"},
  "sink": {"queue": "orders", "table": "public.orders_copy", "mode": "cdc"}
}'
```

[Read it in the docs](/integrations/postgres/#the-sink)

### S3 sink

Copies chosen queues into a bucket as JSONL or Parquet files that DuckDB, Spark or ClickHouse read. Exactly once: a retried upload rewrites the same file.

| Setting | Default |
| --- | --- |
| `QUEEN_S3_EMBEDDED` | off until set to true |
| `QUEEN_S3_QUEUES` | the queues to copy |
| `retentionSinkHold (on the queue)` | "default": retention waits for the sink, up to 7 days |

**When to use it:** History for analytics, audits and replays, on storage that costs less than the brokers' disks.

**Other names:** Kafka: S3 sink connector; AWS: Firehose.

[Read it in the docs](/integrations/s3/)

### Kafka port

Kafka producers, consumers and admin tools talk to QueenMQ through the Kafka port: a topic is a queue, a key is a partition, a Kafka consumer group is a QueenMQ group.

| Setting | Default |
| --- | --- |
| `QUEEN_KAFKA_EMBEDDED` | off until set to true |
| `QUEEN_KAFKA_ADVERTISED_ADDR` | required, no default |

**When to use it:** Move from Kafka one service at a time: change `bootstrap.servers` and nothing else.

**Other names:** Also called: Kafka protocol, Kafka API, Kafka-compatible.

[Read it in the docs](/integrations/kafka/)

## Pop

Take messages. Without a consumer group each message goes to one worker; with groups, every group gets every message.

**When to use it:** Workers that compete for jobs, and services that each want every event.

**Other names:** Kafka: poll, consumer; RabbitMQ: basic.consume, basic.get; Amazon SQS: ReceiveMessage; Google Pub/Sub: pull; Redis: XREADGROUP; Also called: competing consumers.

```js
await queen.queue('orders')
  .each()                         // the handler gets one message at a time
  .limit(1)                       // stop after one message; a worker leaves this out
  .consume(async (msg) => {       // acked when the handler returns, retried if it throws
    console.log(msg.data)
  })
```

[Read it in the docs](/writing-applications/pop/)

### Consumer groups

Each group reads the whole queue at its own pace, and the workers of one group share it. A group works a partition with one worker at a time.

| Setting | Default |
| --- | --- |
| `.group(name)` | none: queue mode, each message to one worker |

**When to use it:** Billing, emails and analytics each want every order: one group each, and none slows the others.

**Other names:** Kafka: consumer group; RabbitMQ: fanout exchange; Amazon SNS: subscription; Google Pub/Sub: subscription; Azure Service Bus: topic subscription; NATS JetStream: durable consumer; Redis: XGROUP.

```js
await queen.queue('orders')
  .group('billing')               // every group gets every message
  .subscriptionMode('all')        // a new group starts at the tail unless told otherwise
  .batch(100)                     // up to 100 messages per pop...
  .partitions(10)                 // ...from up to 10 partitions
  .renewLease(true, 2000)         // heartbeat every 2 s while the handler runs
  .autoAck(false)                 // the handler decides the outcome
  .each()
  .limit(1)
  .consume(async (msg) => {
    const ok = msg.data.total > 0
    await queen.ack(msg, ok ? 'completed' : 'failed', { group: 'billing' })
  })
```

[Read it in the docs](/writing-applications/pop/#queue-mode-and-consumer-groups)

### Where a group starts

Fixed at the group's first pop: `new` reads what comes after it, `all` everything still kept, `subscriptionFrom` from a time. A seek moves a group later.

| Setting | Default |
| --- | --- |
| `subscriptionMode` | new |
| `subscriptionFrom` | none |

**When to use it:** A new projection wants the history, a bug fix replays the last day. A new group skips everything pushed before its first pop unless it asks for `all`.

**Other names:** Kafka: auto.offset.reset, earliest, latest; NATS JetStream: DeliverPolicy; Google Pub/Sub: seek.

```js
await queen.queue('orders')
  .group('audit')                           // a new group, with its own cursor
  .subscriptionFrom('2026-10-22T00:00:00Z') // everything pushed from this time on
  .each()
  .limit(1)
  .consume(async (msg) => {
    console.log('again:', msg.data)
  })
```

[Read it in the docs](/writing-applications/pop/#where-a-new-group-starts)

### Pop parameters

By default the broker sizes every pop: 100 to 1,000 messages, from the partitions that have work. A value you set yourself is never changed.

| Setting | Default |
| --- | --- |
| `.batch(n)` | autopilot, 100 to 1,000 |
| `.partitions(n)` | autopilot, 1 to 64 |
| `.wait(bool)` | true: waits for messages, up to the timeout |
| `.timeoutMillis(ms)` | 30000, at most 60 s |
| `.autopilot(false)` | one message from one partition per pop |

**When to use it:** Leave it to the broker, unless the handler needs a batch of a fixed size.

**Other names:** Kafka: max.poll.records; Amazon SQS: MaxNumberOfMessages, WaitTimeSeconds, long polling; RabbitMQ: prefetch, basic.qos.

```js
await queen.queue('events')
  .group('archiver')
  .subscriptionMode('all')
  .batch(100)                         // up to 100 messages per call
  .limit(1000)
  .consume(async (messages) => {      // no .each(): the handler gets the whole batch
    console.log('archiving', messages.length)
  })                                  // all acked if it returns, all nacked if it throws
```

[Read it in the docs](/writing-applications/pop/#how-many-messages-a-pop-takes-autopilot)

### One or a batch

With `.each()` the handler gets one message at a time; without it, the whole batch. When it returns the client acks, when it throws it acks `failed`; `.autoAck(false)` leaves the ack to the handler.

| Setting | Default |
| --- | --- |
| `.each()` | off: the handler gets the batch |
| `.autoAck(bool)` | true |

**When to use it:** `each()` for one-at-a-time work, a batch for bulk writes, `autoAck(false)` when the handler acks in its own transaction.

**Other names:** Spring Kafka: batch listener.

```js
await queen.queue('events')
  .group('archiver')
  .subscriptionMode('all')
  .batch(100)                         // up to 100 messages per call
  .limit(1000)
  .consume(async (messages) => {      // no .each(): the handler gets the whole batch
    console.log('archiving', messages.length)
  })                                  // all acked if it returns, all nacked if it throws
```

[Read it in the docs](/writing-applications/pop/#one-message-at-a-time-or-a-whole-batch)

### Running a worker

`concurrency(n)` runs n handlers, each on its own partitions; `renewLease` keeps long work leased; `limit` and `idleMillis` end the loop; a `signal` stops it at once and gives back what it had not started.

| Setting | Default |
| --- | --- |
| `.concurrency(n)` | 1 |
| `.renewLease(true, ms)` | off |
| `.idleMillis(ms)` | none: run until stopped |
| `{ signal }` | none |

**When to use it:** A deployed worker: several handlers, renewal for long tasks, a clean stop on SIGTERM.

**Other names:** Also called: consumer threads, graceful shutdown.

```js
const stop = new AbortController()
process.once('SIGTERM', () => stop.abort())     // a deploy stops the worker cleanly

await queen.queue('tasks')
  .group('workers')
  .subscriptionMode('all')
  .concurrency(4)                               // four handlers at once, on four partitions
  .renewLease(true, 10000)                      // long tasks keep their lease alive
  .idleMillis(60000)                            // or stop after a minute with nothing to do
  .each()
  .consume(async (task) => runTask(task.data), { signal: stop.signal })
```

[Read it in the docs](/writing-applications/pop/#running-a-consumer-for-real)

### Conflation

A group with `.conflation()` gets only the newest message of each partition, and its cursor moves past the older ones.

| Setting | Default |
| --- | --- |
| `.conflation()` | off; needs a group, set at its first pop |

**When to use it:** Commands where only the latest counts, like "recompute the price of A-1": a worker that is behind does the work once.

**Other names:** Kafka: log compaction, compacted topic; RabbitMQ: last value queue; Also called: LVQ.

```js
const [latest] = await queen.queue('prices')
  .group('repricer')
  .subscriptionMode('all')
  .conflation()                       // only the newest message of each partition
  .wait(false)
  .pop()
console.log(latest.data)              // { sku: 'A-1', price: 12 }: 10 and 11 are skipped
await queen.ack(latest, 'completed', { group: 'repricer' })
```

[Read it in the docs](/writing-applications/pop/#only-the-newest-message-conflation)

### No ack

With `.commitOnDelivery()` the cursor moves as the messages are handed out: no lease, nothing to ack. Delivery is then at most once.

| Setting | Default |
| --- | --- |
| `.commitOnDelivery()` | off; pop() only |

**When to use it:** A live chart that can lose a reading now and then.

**Other names:** RabbitMQ: autoAck, auto-ack; Kafka: enable.auto.commit; Also called: at most once.

```js
const readings = await queen.queue('metrics')
  .group('live-chart')
  .subscriptionMode('all')
  .commitOnDelivery()                 // the cursor moves as they are handed out: no ack
  .batch(100)
  .wait(false)
  .pop()
```

[Read it in the docs](/writing-applications/pop/#faster-with-no-ack-commitondelivery)

### Filtered group (new in 2.9)

A group with a `where` gets only the messages it matches, and its lag counts only those. The `where` is set at the group's first pop.

| Setting | Default |
| --- | --- |
| `.where(field, op, value)` | every message |

**When to use it:** "Only some of these", in one line in the consumer, with no queue to create.

**Other names:** RabbitMQ: exchange, routing key, binding, topic exchange, headers exchange; Amazon SNS: filter policy; Azure Service Bus: subscription filter, SQL filter; Google Pub/Sub: subscription filter; NATS JetStream: filter subject; Also called: router, content-based router, message filter.

```js
await queen.queue('order-events')
  .group('billing')
  .where('data.kind', 'in', ['order.paid', 'order.refunded'])
  .consume(handler)
```

[Read it in the docs](/writing-applications/routing/#filtered-consumer-groups)

### Read by offset

`fetch` reads a partition by offset: no group, no lease, no ack.

**When to use it:** An audit, rebuilding one entity, or a sink that keeps its own position.

**Other names:** Kafka: offset, seek, assign; RabbitMQ: stream offset; Redis: XRANGE; Also called: replay.

```js
const read = await queen.queue('orders').fetch(['customer-42']) // no group, lease or ack
for (const record of read.get('customer-42').records) {
  console.log(record.offset, record.payload)
}
```

[Read it in the docs](/writing-applications/pop/#read-it-again)

### Ack: four answers

`completed` moves the cursor. `failed` spends a retry. `retry` gives the message back at once, spending none. `dlq` files it in the dead-letter queue now. `consume()` acks for you: `completed` when the handler returns, `failed` when it throws.

| Setting | What happens |
| --- | --- |
| `completed` | done, the cursor moves past it |
| `failed` | comes back after retryDelay |
| `retry` | comes back at once |
| `dlq` | dead-lettered now |

**When to use it:** `dlq` for input that can never work, like a payload that will not parse; `retry` for a worker that is shutting down.

**Other names:** RabbitMQ: basic.ack, basic.nack, basic.reject, requeue; Amazon SQS: DeleteMessage, ChangeMessageVisibility; Azure Service Bus: complete, abandon, dead-letter; NATS JetStream: Ack, Nak, Term; Kafka: commitSync.

```js
await queen.queue('emails')
  .group('mailer')
  .subscriptionMode('all')
  .each()
  .limit(1)
  .consume(async (msg) => sendEmail(msg.data))
  .onSuccess((msg) => queen.ack(msg, 'completed', { group: 'mailer' }))
  .onError((msg, error) => queen.ack(msg, error.permanent ? 'dlq' : 'failed',
    { group: 'mailer', error: error.message }))
```

[Read it in the docs](/writing-applications/ack/#four-answers)

### Replay dead letters

Read the dead-letter queue, newest first, with each letter's error. A replay puts a letter back at the end of its partition, once, and takes it out of the dead-letter queue.

| Setting | What it does |
| --- | --- |
| `.dlq(group).limit(n).get()` | reads it, newest first |
| `admin.retryMessage(partitionId, transactionId)` | replays one letter |

**When to use it:** Fix the cause, then send the parked messages through again, without anyone pushing them by hand.

**Other names:** Amazon SQS: DLQ redrive; RabbitMQ: shovel.

```js
const dead = await queen.queue('payments').dlq('charger').limit(20).get()
for (const letter of dead.messages) {
  console.log(letter.transactionId, letter.errorMessage, letter.retryCount)
}
```

[Read it in the docs](/writing-applications/retries/#replay-it)

## Stream

A stream reads a queue or the tenant feed, keeps its state in the broker, and commits its results with the acks of its inputs in one transaction.

**When to use it:** Counting, summing and grouping over time in a normal process, with no separate stream processor. In the JavaScript, Python, Go and Rust clients.

**Other names:** Kafka: Kafka Streams, ksqlDB; Also called: stream processing, Flink.

```js
import { Stream } from 'queen-mq'

const revenue = await Stream
  .from(queen.queue('orders'))                 // one window per partition: per customer
  .windowTumbling({ seconds: 60 })
  .aggregate({ count: () => 1, sum: (order) => order.total })
  .to(queen.queue('revenue-per-minute'))       // one result per customer per minute
  .run({ queryId: 'revenue-per-minute', url: 'http://localhost:6632' })
```

[Read it in the docs](/writing-applications/streams/)

### Windows

Four kinds: tumbling (fixed, one after the other), sliding (overlapping), session (closed after a quiet gap) and cron (aligned to the clock).

| Setting | Example |
| --- | --- |
| `windowTumbling` | { seconds: 60 } |
| `windowSliding` | { size: 600, slide: 60 } |
| `windowSession` | { gap: 1800 } |
| `windowCron` | { every: 'hour' } |

**When to use it:** Revenue per minute, logins in the last ten minutes, one window per visit, hourly reports.

**Other names:** Kafka: tumbling window, hopping window, session window; Also called: windowed aggregation.

```js
const visits = await Stream
  .from(queen.queue('clicks'))                   // one partition per visitor
  .windowSession({ gap: 1800 })                  // a visit ends after 30 minutes of quiet
  .aggregate({ count: () => 1 })
  .foreach(async (visit) => console.log('visit ended', visit))   // acked after this returns
  .run({ queryId: 'visits', url: 'http://localhost:6632' })
```

[Read it in the docs](/writing-applications/streams/#four-kinds-of-window)

### Event time

With `eventTime`, a window holds the events that happened during it, not the ones that arrived; `allowedLateness` says how late an event may still count.

| Setting | Default |
| --- | --- |
| `eventTime` | the arrival time |
| `allowedLateness` | 0 |
| `onLate` | 'drop' |

**When to use it:** A phone that was offline, a batch upload: count a sale in the hour it happened.

**Other names:** Kafka: TimestampExtractor; Also called: watermark, late events.

```js
const hourly = await Stream
  .from(queen.queue('sales'))
  .windowCron({
    every: 'hour',                     // on the clock: 10:00 to 11:00, 11:00 to 12:00
    eventTime: (msg) => msg.data.soldAt,  // when the sale happened, not when it arrived
    allowedLateness: 120,              // a sale up to 2 minutes late still counts
  })
  .aggregate({ count: () => 1, sum: (sale) => sale.amount, max: (sale) => sale.amount })
  .to(queen.queue('sales-per-hour'))
  .run({ queryId: 'sales-per-hour', url: 'http://localhost:6632', subscriptionMode: 'all' })
```

[Read it in the docs](/writing-applications/streams/#when-it-happened-or-when-it-arrived)

### Shape the data

`map`, `filter` and `flatMap` change, drop or split messages; `aggregate` folds a window into named numbers (`count`, `sum`, `min`, `max`, `avg`), `reduce` is the general fold; `keyBy` groups by something other than the partition.

**When to use it:** Most stream jobs are the same few steps: drop noise, reshape, group, fold.

**Other names:** Kafka: KStream, mapValues, aggregate.

```js
const perMinute = await Stream
  .fromFeed(queen, { queues: ['orders', 'payments'] })
  .windowTumbling({ seconds: 60 })
  .aggregate({ count: () => 1, sum: (e) => e.total })
  .map((totals, w) => ({ ...totals, customer: w.key, minute: w.windowKey }))
  .to(queen.queue('per-minute'))
  .run({ queryId: 'per-minute' })
```

[Read it in the docs](/writing-applications/streams/#shaping-the-data)

### Gate

`gate(fn)` decides, message by message, whether a message goes on now, with state kept per partition. A refused message stops its partition and comes back, in order, when its lease runs out.

| Setting | Gates |
| --- | --- |
| `tokenBucketGate({ capacity, refillPerSec })` | ready-made |
| `slidingWindowGate` | ready-made |

**When to use it:** An SMS provider that accepts five messages a second per customer.

**Other names:** Also called: rate limit, token bucket, throttle.

```js
import { tokenBucketGate } from 'queen-mq'

const limiter = await Stream
  .from(queen.queue('outgoing-sms'))             // one partition per customer
  .gate(tokenBucketGate({ capacity: 10, refillPerSec: 1 }))   // 10 at once, then 1 a second
  .foreach(async (sms) => sendSms(sms))
  .run({ queryId: 'sms-limiter', url: 'http://localhost:6632' })
```

[Read it in the docs](/writing-applications/streams/#rate-limits-with-a-gate)

### Outputs

`.to(queue)` pushes each result in the same commit as the state and the acks. `.foreach(fn)` calls your code and acks after it returns, so a crash calls it again. `.foreachBatch` hands results over in batches.

| Setting | Delivery |
| --- | --- |
| `.to(queue)` | exactly once, in the commit |
| `.foreach(fn)` | at least once |
| `.foreachBatch(fn, { maxItems, maxWaitMillis })` | 1000 items, 0 ms |

**When to use it:** Results that stay in QueenMQ go `to` a queue; a call to the outside world goes in `foreach`, with an idempotency key.

**Other names:** Kafka: to(), exactly-once, EOS.

```js
const toWarehouse = await Stream
  .fromFeed(queen, { queues: ['orders'] })
  .foreachBatch(async (items) => {
    // One write for the batch. An item's id is the same each time it is delivered.
    await warehouse.upsert(items.map((it) => ({ key: it.id, ...it.value })))
  }, { maxItems: 500, maxWaitMillis: 1000 })
  .run({ queryId: 'orders-to-warehouse' })
```

[Read it in the docs](/writing-applications/streams/#two-ways-to-end-a-stream)

### Tenant feed

`Stream.fromFeed` reads every push to the queues you name, in commit order, from the followers. Its `where` runs in the broker, so a message that fails it never crosses the network.

| Setting | Default |
| --- | --- |
| `queues` | every queue of the tenant |
| `from` | 'now' |

**When to use it:** Route, filter or window across several queues at once.

```js
const ordersIt = await Stream
  .fromFeed(queen, { queues: ['orders'] })
  .where('data.country', '=', 'IT')
  .filter((m) => m.data.total >= 10)
  .to(queen.queue('orders-it'))
  .run({ queryId: 'orders-it' })
```

[Read it in the docs](/writing-applications/streams/#streams-from-the-tenant-feed)

### Patterns and joins

Wait for one message after another (`pattern`), for several kinds (`join`), for n parts (`gather`), or join a stream with a table (`joinTable`), each within a time.

| Setting | Example |
| --- | --- |
| `within` | '15m', '2d', ... |

**When to use it:** An order not paid within 15 minutes; an order paid and shipped; a batch whose parts all arrived.

**Other names:** Kafka: KStream join; Also called: CEP, complex event processing.

```js
const followups = await Stream
  .fromFeed(queen, { queues: ['orders', 'payments'] })
  .pattern({
    start: (m) => m.queue === 'orders',
    then: (m) => m.queue === 'payments',
    within: '15m',
    onMatch: (order, payment) => ({ order: order.data.id, paid: payment.data.amount }),
    onTimeout: (order) => ({ order: order.data.id, remind: true })
  })
  .to(queen.queue('order-followups'))
  .run({ queryId: 'order-followups' })
```

[Read it in the docs](/writing-applications/streams/#patterns-and-joins)

### Run a stream

`run({ queryId })` names the stream: its position, snapshots and lock. One process runs it at a time; `shards` spreads it over several; `Stream.view` reads its state while it runs; `onPoison` says what a message that keeps failing does.

| Setting | Default |
| --- | --- |
| `queryId` | required |
| `exclusive` | true: one process at a time |
| `shards` | 1 |
| `onPoison, retries` | 'stop', 3 |
| `serveState` | true |

**When to use it:** A stream that restarts where it stopped, runs in one place, and can be read while it runs.

**Other names:** Kafka: application.id, state store.

```js
const perTenant = await Stream
  .fromFeed(queen, { queues: ['clicks'] })
  .keyBy((m) => m.data.tenant)
  .windowTumbling({ seconds: 60 }).aggregate({ clicks: () => 1 })
  .to(queen.queue('clicks-per-tenant'))
  .run({ queryId: 'clicks-per-tenant', shards: 4, shardBy: (m) => m.data.tenant })
```

[Read it in the docs](/writing-applications/streams/#run-options)

## Transaction

One call that commits acks, pushes, KV writes and timers together: one entry of the log, all or nothing.

| Part | Limit |
| --- | --- |
| `KV operations` | at most 64, over 256 keys, one write per key |
| `Timer operations` | one per queue and timer key |
| `Request body` | at most 64 MiB |

**When to use it:** Every step of a workflow: ack the input, push the next events, save the state, schedule the reminder.

**Other names:** Kafka: transactions, exactly-once; Also called: outbox, transactional outbox.

```js
await queen.queue('orders')
  .group('fulfillment')
  .subscriptionMode('all')
  .autoAck(false)
  .each()
  .limit(1)
  .consume(async (msg) => {
    const { orderId, customer } = msg.data
    await queen.transaction()
      .ack(msg, 'completed', { consumerGroup: 'fulfillment' })  // the input is done
      .kv.put('orders', `order-${orderId}`, { status: 'paid' }, { ttl: '7d' })
      .queue('shipments').partition(msg.partition)              // the next event
      .push([{ transactionId: `ship-${orderId}`, data: { orderId } }])
      .timer('reminders').key(`review-${orderId}`).delay('24h') // a message tomorrow
      .payload({ orderId, customer }).schedule()
      .commit()
  })
```

[Read it in the docs](/writing-applications/transactions/)

### Conditions

A transaction cannot read and then decide. Put a condition on a write instead: `expect` names the version a key must have, and `required: true` rolls the whole call back when it misses.

| Setting | Default |
| --- | --- |
| `expect` | none |
| `required` | false |

**When to use it:** Two workers read the same version and both decide to write: one wins, the other rolls back with nothing written.

**Other names:** Also called: compare-and-set, CAS, optimistic locking.

```js
await queen.queue('order-events')
  .group('workflow')
  .subscriptionMode('all')
  .autoAck(false)
  .each()
  .consume(async (msg) => {
    const { orderId } = msg.data
    const order = await queen.kv.get('orders', orderId)            // { status: 'paid' }
    if (order.value.status !== 'paid') {                   // already moved: just ack
      return queen.ack(msg, 'completed', { group: 'workflow' })
    }

    // Do the work of this step here: charge the card, book the courier, ...

    const result = await queen.transaction()
      .ack(msg, 'completed', { consumerGroup: 'workflow' })
      .kv.put('orders', orderId, { status: 'shipping' },
        { ttl: '30d', expect: order.version, required: true })  // only from what we read
      .queue('shipping').partition(orderId)
      .push([{ transactionId: `ship-${orderId}`, data: { orderId } }])
      .commit()
    if (result.success === false) return  // moved by someone else: nothing written
  })
```

[Read it in the docs](/writing-applications/transactions/#conditions-instead-of-reads)

### once()

A marker written with `putIfAbsent` and `required: true`, keyed on the business fact. A second delivery of the same fact rolls back whole, receipts and timers included.

| Setting | Where |
| --- | --- |
| `.once(ns, key, { ttl })` | JavaScript and Python; elsewhere putIfAbsent + required |

**When to use it:** Send the receipt once per payment: key the marker on the `paymentId`.

**Other names:** Also called: idempotency key, idempotent consumer, inbox.

[Read it in the docs](/writing-applications/transactions/#conditions-instead-of-reads)

### Link (new in 2.9)

Forward the message you hold into another queue without its payload: about 20 bytes, written in the same commit.

| Setting | Where |
| --- | --- |
| `.link([message], { headers })` | inside a transaction |

**When to use it:** Relays and pipelines: a hop costs about 20 bytes, whatever the payload weighs.

**Other names:** Azure Service Bus: auto-forwarding; Also called: forward.

```js
await queen.transaction()
  .ack(message)
  .queue('manual-review').partition(message.partition)
  .link([message], { headers: { 'review-reason': 'amount' } })
  .commit()
```

[Read it in the docs](/writing-applications/routing/#link-forward-a-message-without-its-payload)

### Rollback answers

A rolled-back call answers HTTP 200 with `success: false` and a reason: `rejected_ack`, `kv_precondition`, `duplicate` or `reserved`. The SDKs return it; they do not throw.

| Setting | In the answer |
| --- | --- |
| `success` | false on a rollback |
| `reason` | why it rolled back |

**When to use it:** Read `success` after every commit. `duplicate` means the step had already committed.

**Other names:** Kafka: abortTransaction; Also called: abort.

[Read it in the docs](/writing-applications/transactions/#why-a-transaction-rolls-back)

## KV / Locks

Keys and values in the same log as the messages, and the locks built on them. Every write names its lifetime and gives the key a new version; a lock is a lease with one holder, a semaphore one with n.

**When to use it:** An entity's state, the markers that make a step safe to repeat, counters, and the work no message triggers: a nightly job, a singleton, a pool of n. KV has no queries: the records you search stay in your database.

**Other names:** Redis: GET, SET; NATS JetStream: KV bucket; Also called: key-value store.

```js
// Every write names its lifetime: a ttl, or forever.
await queen.kv.put('carts', 'customer-42', { items: 3 }, { ttl: '7d' })
const cart = await queen.kv.get('carts', 'customer-42')   // { found, value, version, ... }

// Write only if nobody wrote since we read it: the version is the condition.
const saved = await queen.kv.put('carts', 'customer-42', { items: 4 },
  { ttl: '7d', expect: cart.version })
console.log(saved.applied)                                // false if the version moved

// A counter with a ceiling: past max, the increment does not apply.
const hit = await queen.kv.incr('limits', 'api:customer-42', 1, { max: 100, ttl: '1m' })
if (!hit.applied) console.log('over the limit')           // reason: 'limit'
```

[Read it in the docs](/writing-applications/kv-locks-timers/)

### Get and put

`get`, `getMany`, `put` and `delete`. A write names a TTL or `forever`; a read returns the value, its version and when it expires.

| Setting | Rule |
| --- | --- |
| `ttl` | every write names one, or forever |

**When to use it:** The state a step reads before it decides, written back in the step's transaction.

**Other names:** Redis: GET, SET, EXPIRE; NATS JetStream: KV get, KV put.

```js
// Every write names its lifetime: a ttl, or forever.
await queen.kv.put('carts', 'customer-42', { items: 3 }, { ttl: '7d' })
const cart = await queen.kv.get('carts', 'customer-42')   // { found, value, version, ... }

// Write only if nobody wrote since we read it: the version is the condition.
const saved = await queen.kv.put('carts', 'customer-42', { items: 4 },
  { ttl: '7d', expect: cart.version })
console.log(saved.applied)                                // false if the version moved

// A counter with a ceiling: past max, the increment does not apply.
const hit = await queen.kv.incr('limits', 'api:customer-42', 1, { max: 100, ttl: '1m' })
if (!hit.applied) console.log('over the limit')           // reason: 'limit'
```

[Read it in the docs](/writing-applications/kv-locks-timers/#kv)

### Put if absent

`putIfAbsent` writes only when the key does not exist: of several callers, one wins.

**When to use it:** A claim: the first worker to arrive does the work, the others see who has it.

**Other names:** Redis: SETNX, SET NX; NATS JetStream: KV create.

[Read it in the docs](/writing-applications/kv-locks-timers/#kv)

### Versions

`expect` makes a write depend on the version you read; `check` asserts a version and writes nothing. A missed condition answers `applied: false` with the current value.

| Setting | Default |
| --- | --- |
| `expect` | none |
| `check` | asserts, writes nothing |

**When to use it:** Two workers read the same cart and both write it: the second is refused instead of overwriting what it never saw.

**Other names:** NATS JetStream: revision; Also called: ETag, compare-and-set.

```js
// Every write names its lifetime: a ttl, or forever.
await queen.kv.put('carts', 'customer-42', { items: 3 }, { ttl: '7d' })
const cart = await queen.kv.get('carts', 'customer-42')   // { found, value, version, ... }

// Write only if nobody wrote since we read it: the version is the condition.
const saved = await queen.kv.put('carts', 'customer-42', { items: 4 },
  { ttl: '7d', expect: cart.version })
console.log(saved.applied)                                // false if the version moved

// A counter with a ceiling: past max, the increment does not apply.
const hit = await queen.kv.incr('limits', 'api:customer-42', 1, { max: 100, ttl: '1m' })
if (!hit.applied) console.log('over the limit')           // reason: 'limit'
```

[Read it in the docs](/writing-applications/kv-locks-timers/#kv)

### Counters

`incr` adds to a counter with no read first. With `max` or `min`, an increment past the bound does not apply.

| Setting | Default |
| --- | --- |
| `incr(ns, key, by, { max, min, ttl })` | no bound |

**When to use it:** A rate limit in one call: at most 100 calls a minute per customer.

**Other names:** Redis: INCR, INCRBY.

```js
// Every write names its lifetime: a ttl, or forever.
await queen.kv.put('carts', 'customer-42', { items: 3 }, { ttl: '7d' })
const cart = await queen.kv.get('carts', 'customer-42')   // { found, value, version, ... }

// Write only if nobody wrote since we read it: the version is the condition.
const saved = await queen.kv.put('carts', 'customer-42', { items: 4 },
  { ttl: '7d', expect: cart.version })
console.log(saved.applied)                                // false if the version moved

// A counter with a ceiling: past max, the increment does not apply.
const hit = await queen.kv.incr('limits', 'api:customer-42', 1, { max: 100, ttl: '1m' })
if (!hit.applied) console.log('over the limit')           // reason: 'limit'
```

[Read it in the docs](/writing-applications/kv-locks-timers/#kv)

### Prefix pages

`getPrefix` reads one page of keys under a prefix, with a cursor for the next. Not inside a transaction.

**When to use it:** List the keys of one customer, or of one job.

**Other names:** Redis: SCAN MATCH; Also called: list keys.

[Read it in the docs](/writing-applications/kv-locks-timers/#kv)

### Lock

`queen.lock(name, { ttl })`: `acquire()` answers false while someone else holds it. The JavaScript, Python, Go, Rust and C++ handles renew it in the background.

| Setting | Default |
| --- | --- |
| `ttl` | you choose; renewed every third of it |

**When to use it:** One worker at a time for a nightly report.

**Other names:** Redis: Redlock; Also called: distributed lock, leader election, mutex.

```js
const lock = queen.lock('nightly-report', { ttl: '30s' })
if (!(await lock.acquire({ wait: '1s' }))) return 'skipped'   // another worker has it
try {
  const report = { day: '2026-10-24', bookings: 412 }
  const result = await queen.transaction()
    .guard(lock)                                  // commits only while the lock is ours
    .queue('daily-reports').push([{ transactionId: `report-${report.day}`, data: report }])
    .commit()
  if (result.success === false) return 'lost the lock'
  return 'done'
} finally {
  await lock.release()
}
```

[Read it in the docs](/patterns/one-worker/)

### Semaphore

`queen.semaphore(name, n)`: at most n holders at once. One handle is one permit.

**When to use it:** At most four jobs at a time on a shared resource, across every worker.

**Other names:** Also called: distributed semaphore, concurrency limit.

[Read it in the docs](/writing-applications/kv-locks-timers/#locks-and-semaphores)

### Guard

`.guard(lock)` on a transaction: it commits only while the lock is still yours. A lock expires without telling its holder; the guard stops a paused process from doing the work twice.

**When to use it:** Fence the work of a lock holder inside QueenMQ.

**Other names:** Also called: fencing token.

```js
const lock = queen.lock('daily-report', { ttl: '30s' }) // a lease with a fencing token
if (await lock.acquire()) {                             // false: another worker has it
  try {
    await queen.transaction()
      .guard(lock)                                      // commits only while the lock is ours
      .queue('reports').push([{ transactionId: 'report-2026-10-22', data: { day: '2026-10-22' } }])
      .commit()
  } finally {
    await lock.release()
  }
}
```

[Read it in the docs](/writing-applications/kv-locks-timers/#locks-and-semaphores)

## Timers

A message scheduled for later, named by its queue and its key. When it fires, the broker pushes it through the log like any push.

**When to use it:** Reminders, timeouts and follow-ups, with no cron job and no table to scan.

**Other names:** Azure Service Bus: scheduled messages; Also called: cron, scheduler, reminder.

```js
// When an order is delivered, ask for a review in 24 hours.
await queen.timer('review-reminders').key('review-2001').delay('24h')
  .payload({ orderId: 2001 }).schedule()          // status: 'scheduled'

// The delivery was late: schedule the same key again, and the reminder moves.
await queen.timer('review-reminders').key('review-2001').delay('48h')
  .payload({ orderId: 2001 }).schedule()          // status: 'rescheduled'

// The guest wrote a review already: no reminder at all.
await queen.timer('review-reminders').key('review-2001').cancel()
```

[Read it in the docs](/writing-applications/kv-locks-timers/#timers)

### Schedule and move

Scheduling the same key again replaces the timer, so moving a reminder is the same call. A fire is never early, and a little late: the leader checks every 50 ms.

| Setting | How |
| --- | --- |
| `timer(queue).key(k).delay(d).schedule()` | replaces a timer with the same key |
| `.headers({ ... })` | the message it fires carries them (2.9) |

**When to use it:** Ask for a review 24 hours after delivery, and move it when the delivery was late.

**Other names:** Azure Service Bus: ScheduledEnqueueTime; RabbitMQ: delayed message exchange, x-delay; Amazon SQS: message timer; Pulsar: deliverAt.

```js
// When an order is delivered, ask for a review in 24 hours.
await queen.timer('review-reminders').key('review-2001').delay('24h')
  .payload({ orderId: 2001 }).schedule()          // status: 'scheduled'

// The delivery was late: schedule the same key again, and the reminder moves.
await queen.timer('review-reminders').key('review-2001').delay('48h')
  .payload({ orderId: 2001 }).schedule()          // status: 'rescheduled'

// The guest wrote a review already: no reminder at all.
await queen.timer('review-reminders').key('review-2001').cancel()
```

[Read it in the docs](/patterns/delayed-messages/)

### Cancel

A cancel answers `absent` when there is no timer, which may mean it already fired.

**When to use it:** The guest already wrote a review: no reminder. A handler that compensates checks the entity's state first.

**Other names:** Azure Service Bus: CancelScheduledMessage.

```js
// When an order is delivered, ask for a review in 24 hours.
await queen.timer('review-reminders').key('review-2001').delay('24h')
  .payload({ orderId: 2001 }).schedule()          // status: 'scheduled'

// The delivery was late: schedule the same key again, and the reminder moves.
await queen.timer('review-reminders').key('review-2001').delay('48h')
  .payload({ orderId: 2001 }).schedule()          // status: 'rescheduled'

// The guest wrote a review already: no reminder at all.
await queen.timer('review-reminders').key('review-2001').cancel()
```

[Read it in the docs](/writing-applications/kv-locks-timers/#timers)

### Entity's partition

Give a timer the entity's partition, and it arrives in order with the entity's other events.

| Setting | How |
| --- | --- |
| `.partition(name)` | the entity's partition |

**When to use it:** A timeout that must stay in step with the events of the same order.

[Read it in the docs](/writing-applications/kv-locks-timers/#timers)

## Ephemeral queues

Queues in memory, outside the log, for request and reply. Every node serves every queue. No transactions and no replay, and a crash loses what is in flight, by design.

**When to use it:** A request that waits for its answer now (what is the price of A-1?): speed matters more than history.

**Other names:** RabbitMQ: Direct reply-to, transient queue; NATS: request-reply; Also called: RPC.

```js
const inbox = `inbox-${crypto.randomUUID()}`           // a private reply queue
await queen.ephemeral.push('price-requests', [{ sku: 'A-1', replyTo: inbox }])
const reply = await queen.ephemeral.pop(inbox, { wait: true, timeout: 5000, commitOnDelivery: true })
console.log(reply.messages[0]?.payload)                // { sku: 'A-1', price: 59.9 }
```

[Read it in the docs](/writing-applications/ephemeral/)

### Reply inbox

The caller makes a private inbox, sends its name as `replyTo`, and waits on it with a timeout.

| Setting | What to give |
| --- | --- |
| `replyTo` | the caller's inbox |
| `timeout` | you choose |

**When to use it:** A quote with a fallback: wait at most half a second, then show "ask us".

**Other names:** RabbitMQ: Direct reply-to, reply_to; NATS: inbox; Also called: return address, correlation ID.

```js
// The pricing service answers quote requests, in memory.
async function pricing() {
  const { messages } = await queen.ephemeral.pop('quotes', { group: 'pricing', wait: true })
  for (const m of messages) {
    await queen.ephemeral.push(m.payload.replyTo, [{ room: m.payload.room, price: 129 }])
  }
  await queen.ephemeral.ack('quotes', messages, { group: 'pricing' })
}

// The booking page asks, and waits at most half a second for the answer.
async function quote(room) {
  const inbox = `inbox-${crypto.randomUUID()}`
  await queen.ephemeral.push('quotes', [{ room, replyTo: inbox }])
  const reply = await queen.ephemeral.pop(inbox,
    { wait: true, timeout: 500, commitOnDelivery: true })
  return reply.messages[0]?.payload ?? { room, price: null }    // too slow: show "ask us"
}

const [deluxe] = await Promise.all([quote('deluxe'), pricing()])
console.log(deluxe)                               // { room: 'deluxe', price: 129 }
```

[Read it in the docs](/patterns/request-reply/)

### Worker

A worker pops requests and answers on whatever inbox each one names. Groups, leases and acks work as on a regular queue.

**When to use it:** A pricing service, a lookup, any answer computed on demand.

**Other names:** Also called: RPC server, responder.

```js
const { messages } = await queen.ephemeral.pop('price-requests', { group: 'pricers', wait: true })
for (const m of messages) {
  await queen.ephemeral.push(m.payload.replyTo, [{ sku: m.payload.sku, price: 59.9 }])
}
await queen.ephemeral.ack('price-requests', messages, { group: 'pricers' })
```

[Read it in the docs](/writing-applications/ephemeral/)

### In memory

A partition keeps its order while one node owns it. When that node restarts, the partition moves, and a message pushed during the move can come out early. A crash loses that node's share.

**When to use it:** Anything that must survive a crash goes in a regular queue.

**Other names:** RabbitMQ: transient, non-durable; NATS: Core NATS.

[Read it in the docs](/writing-applications/ephemeral/)

## I want to

### Spread jobs over workers

Many workers share one stream of jobs, and each job is done by one of them.

1. Partition: Push each job into its user's partition: a partition is worked by one worker at a time.
2. Pop: Consume without a consumer group: each job goes to one worker.
3. Running a worker: Run a few handlers at once with `concurrency`; add workers to go faster.
4. Lease: A job not acked within its lease goes to another worker.
5. Ack: four answers: Ack each job when it is done; a failed one comes back.

[See it worked out](/patterns/work-queue/)

### Send one event to many services

One event, read by every service at its own pace.

1. Push: Push the event once.
2. Consumer groups: Each service reads with its own consumer group, and every group gets every event.
3. Where a group starts: A group added later asks for `subscriptionMode('all')` to start from the first event.

[See it worked out](/patterns/publish-subscribe/)

### Run a workflow step by step

Each step of the workflow is one transaction.

1. Versions: Read the order's state and its version.
2. Transaction: Open one transaction for the whole step.
3. Ack: four answers: Ack the event that started the step.
4. Conditions: Write the new state only if its version did not move.
5. Push: Push the event for the next step.
6. once(): Or gate the step on a key instead of a version.

[See it worked out](/patterns/workflows/)

### Ask and wait for an answer

A request with a deadline, answered through a private inbox in memory.

1. Reply inbox: Make a private inbox, send it as `replyTo`, and wait on it with a timeout.
2. Worker: Any instance of the service takes the request and answers to that inbox.
3. In memory: Nothing goes to disk: a node that crashes loses the requests it held.

[See it worked out](/patterns/request-reply/)

### Do something later

A timer is a message that waits, named by its queue and a key.

1. Schedule and move: Schedule a timer under a key; scheduling the key again moves it.
2. Cancel: Cancel the key when the reminder is no longer needed.
3. Timers: When it is due, the timer arrives in its queue as a normal message.
4. Delay: Or make every message of a queue wait a set time before anyone can take it.

[See it worked out](/patterns/delayed-messages/)

### Run a job on one worker

Every replica runs the same scheduler; one of them does the job.

1. Lock: Take the lock: the worker that gets it does the job, the others skip.
2. Transaction: Write the job's result in one transaction.
3. Guard: Put `.guard(lock)` on it: the commit rolls back if the lock was lost meanwhile.

[See it worked out](/patterns/one-worker/)

### Retry, then park what fails

A call that can fail is tried again, then kept aside for a person.

1. Retries: Give the queue a `retryLimit` and a `retryDelay`.
2. Ack: four answers: Throw, or ack it as failed: the message comes back after the delay.
3. Dead-letter queue: After the last try it goes to the dead-letter queue.
4. Replay dead letters: Read the dead letters, fix the cause, and send them back.

[See it worked out](/patterns/retries-dead-letters/)

### Keep only the newest update

For a value that changes, only the last one matters.

1. Partition: Push each thing's updates into its own partition: a hotel, a sensor.
2. Conflation: Read with `.conflation()`: the group gets only the newest message of each partition.

[See it worked out](/patterns/last-value/)

### Answer a burst at once

A guest sends three messages in a row; the assistant answers once.

1. Partition: Every conversation is a partition, so its messages stay in order.
2. Burst window: Set `windowBuffer`: new messages wait until the first is that old, then go together.
3. One or a batch: Consume without `.each()`: the handler gets the whole burst as one batch.

[See it worked out](/patterns/ordering-batching/)

### Rate-limit each customer

Each customer gets so many messages a second, all of them in order.

1. Partition: Push each customer's messages into the customer's partition.
2. Gate: Run a stream with a token bucket gate: so many messages a second, per partition.
3. Lease: A message the gate refuses comes back when its lease ends, still in order.

[See it worked out](/patterns/rate-limiting/)

### Count and sum over time

Events add up to one record per window: a trace, a session, a minute.

1. Partition: One partition per trace, session or visitor.
2. Windows: A stream closes a window after a gap of quiet, or every N seconds.
3. Shape the data: Add each window up: count, sum, the first and the last.
4. Outputs: Push one message per closed window with `.to(queue)`, once.
5. Event time: Or window by the time in the event, not by when it arrived.

[See it worked out](/patterns/windowed-aggregation/)

### Route by type

Each reader gets only the messages it wants, and each message is written once.

1. Headers: Push with headers, such as a `type`.
2. Filtered group: A consumer group reads only what matches its `where`.
3. Reading queue: Or a queue reads other queues through a `where`, as links of about 20 bytes.
4. Link: Or a transaction forwards the message it holds to another queue.

[See it worked out](/writing-applications/routing/)

### Stream a table into a queue

A table's changes arrive as messages, from inside the broker.

1. PostgreSQL source: Stream a PostgreSQL table into a queue through logical replication.
2. MySQL source: Or a MySQL table, from its binlog.

[See it worked out](/integrations/)

### Keep my Kafka clients

Kafka clients use QueenMQ unchanged; move one service at a time.

1. Kafka port: Turn on the Kafka port and point `bootstrap.servers` at it.
2. Consumer groups: A topic is a queue and a Kafka group is a consumer group: both clients share the data.

[See it worked out](/integrations/kafka/)

Source: https://queenmq.com/map/index.mdx
