Skip to content

POST /api/v1/push

Produce messages: the request body field by field, the per-item response array, all four item statuses, and exactly how deduplication decides.

Updated View as Markdown

One call writes one or more messages to one or more queues. Access level write-only, which is the only route a write-only token can reach. The response is always a top-level array with one element per request item, in request order.

Request

{
  "items": [
    {
      "queue": "orders.created",
      "partition": "customer-42",
      "transactionId": "order-8891-created",
      "payload": { "orderId": 8891, "total": 4200 }
    }
  ]
}
Field Type Required Default
items array of objects yes n/a
items[].queue string yes n/a
items[].partition string no "Default"
items[].payload any JSON value yes n/a
items[].transactionId string no the message id the broker mints for this item

Notes that change behaviour:

  • payload is the wire field name. The JavaScript and Python clients also accept data on an item and rename it to payload before sending; the Go client’s field is Payload. On raw HTTP, data is not recognised, and because unknown keys are ignored an item carrying data but no payload fails the body parse with a 400: payload is required.
  • payload may be any JSON value: object, array, string, number, boolean or null. It is stored verbatim and spliced back into the pop response without re-serialization.
  • traceId is silently ignored on this route. The push path always stores a null trace id, so every message produced through /api/v1/push comes back from a pop with traceId: null. A trace id can only be attached at write time through POST /api/v1/transaction.
  • Items in one call may target different queues and different partitions.
  • "items": [] is accepted and returns 201 with an empty array.
  • When authentication is enabled, the validated token’s sub is stamped onto every frame and echoed by pops as producerSub. A producerSub supplied in the request body is never honoured.

Queues and partitions are created by pushing

If the queue does not exist, the first push creates it, together with its configuration row and its first partition. namespace and task are derived from a dotted name (orders.created becomes namespace orders, task created), so a queue that was never explicitly configured is still discoverable by a namespace or task pop. Queue options keep their defaults, which is where the two-valued leaseTime comes from: a queue created implicitly by a push leases for 60 seconds, while a queue created or updated through /api/v1/configure leases for 300 seconds.

Partitions are created on first push to that partition name, and deleted automatically only once they are empty and have been idle for PARTITION_CLEANUP_DAYS (30). A push to a name whose partition was reclaimed simply creates it again, with a new id and offsets from 0.

Response

201 Created, Content-Type: application/json, a top-level array:

[
  {
    "index": 0,
    "message_id": "0198f2c1-4d3a-7c10-9f2b-6a1e5d0c7b83",
    "transaction_id": "order-8891-created",
    "queueName": "orders.created",
    "status": "queued"
  }
]
Field Type Meaning
index integer the item’s position in the request array
message_id string (UUIDv7) the stored message’s id; for a duplicate, the id of the original message
transaction_id string the deduplication key actually used: your transactionId, or the minted message id when you omitted it
queueName string the queue the item targeted
status string one of the four values below

The mixed casing (message_id and transaction_id snake_case, queueName camelCase) is the wire contract, not a typo, and clients depend on it.

There is no partition, no partitionId, no offset and no createdAt in a push result. The offset a message landed on is never returned to a producer; positions are internal to the storage engine and are only ever used through cursors.

Item statuses

status What happened What to do
queued stored and committed nothing
duplicate an earlier message with the same transactionId already exists in this partition, inside the deduplication window; nothing was written nothing. This is the idempotent outcome
buffered the message went to the broker’s on-disk spool instead of PostgreSQL, and a background drain will replay it: either because the database transaction failed, or because the broker is in maintenance mode, which diverts every push to the spool treat as accepted; if maintenance mode is off, investigate the database
failed the write did not reach PostgreSQL and the spool write also failed; the message exists nowhere retry it

Replay from the spool preserves the transactionId, so deduplication makes it idempotent: a message that was both buffered and (by your own retry) pushed again lands once.

Status codes

Code When
201 the normal path, whatever the per-item statuses; also the maintenance-mode path when every item spooled
400 the body is not JSON, or items / queue / payload is missing: {"error":"bad body: …"}
403 authentication is on and the token has no writer role
413 the body exceeds 64 MiB
500 maintenance mode is on and at least one spool write failed

Atomicity

Items are grouped by (queue, partition), and each group becomes one segment. The commit boundary is the group, not the request. A push spanning three partitions can have one group committed and another failed, which is exactly what the per-item statuses report. If you need several writes to land together, use POST /api/v1/transaction, which runs the whole bundle in one PostgreSQL transaction.

The handler does not answer until every group has committed. There is no batching delay in front of that commit: when the push lane is uncontended the segment dispatches on arrival, so a single low-rate push costs about one database round trip. Coalescing is emergent, not timed. While a partition has a flush in flight the next segment accumulates, so batches fatten only when load produces them, and QUEEN_V2_FUSION_HOLD_MS (default 3) is a liveness backstop tick, not a linger window.

Coalescing never reorders a partition either. At most one flush per (queue, partition) is in flight, and the next segment is dispatched only after the previous one commits, so a partition’s offsets are assigned in commit order no matter how many requests were fused into the same transaction. The commit timestamp is taken under the same partition row lock, so created_at is monotone per partition as well.

Bundling is the other half: one transaction can carry the segments of many partitions, which is what makes one commit and one fsync serve many messages. Life of a push walks the whole path, shard by shard.

Deduplication

Deduplication is on by default, exact, and scoped to one partition. The key is the item’s transactionId; the bound is the queue’s dedupWindowSeconds, whose default is 3600. Setting it to 0 disables deduplication for that queue.

Two layers run, in this order:

  1. Within one request. The first item for a given (queue, partition, transactionId) wins. Later items in the same body produce no frame and come back duplicate, carrying the winner’s message_id.
  2. In the database. Before allocating an offset, the push probes the partition’s transaction index for the incoming hashes while holding the partition’s row lock. A hit returns duplicate having written nothing at all: there is no insert to roll back, and no subtransaction. A clean probe proceeds to allocate.

Because the probe happens under the same row lock that serializes writes to the partition, two concurrent pushes of the same transactionId cannot both succeed.

clients/client-js/test-v2/docs.jsjs
const first = await client
  .queue('payments')
  .partition('customer-42')
  .push([{ transactionId: 'order-9137-paid', data: { orderId: 9137, amount: 99.5 } }])

const retry = await client
  .queue('payments')
  .partition('customer-42')
  .push([{ transactionId: 'order-9137-paid', data: { orderId: 9137, amount: 99.5 } }])
// retry[0].status is 'duplicate': the second push wrote nothing
// and answers with the first message's id.

What you get back for a duplicate is the original message’s id, resolved from the stored segment. If the original can no longer be read (retention removed its segment, for instance), the id comes back as the all-zeros UUID 00000000-0000-0000-0000-000000000000, the “original unknown” sentinel. The status is still duplicate.

Delayed delivery

A push has no per-message delay field. The delay is a queue option, delayedProcessing, an integer number of seconds defaulting to 0, set through /api/v1/configure and applied to every message the queue receives. A message pushed to a queue with delayedProcessing: 30 is stored and committed immediately, and becomes poppable 30 seconds after that commit.

The cut is enforced in SQL on the pop path, not in the producer: a pop delivers only segments whose created_at is at least delayedProcessing seconds old. The broker’s in-memory scheduler only schedules a revisit for the deadline, it never bypasses the cut, so a delayed message cannot be delivered early by a consumer that polls at the wrong moment.

Nothing is reordered by the delay. Commit timestamps are monotone per partition, so the deadline cuts a contiguous recent tail of the partition rather than holding individual messages back past their neighbours: the same messages come out in the same order, later. The in-flight backlog the delay creates is bounded by push rate times the delay.

Changing the option applies to messages that are already stored: raising it hides messages that were visible a moment ago, and lowering it exposes them. /api/v1/configure merges, so omitting delayedProcessing on a later call leaves it alone; send it as null, or send the whole configuration with "mode": "replace", to put it back to 0.

Examples

clients/client-js/test-v2/docs.jsjs
const res = await client
  .queue('orders')
  .partition('customer-42')
  .push([{ data: { orderId: 9137, amount: 99.5 } }])
clients/client-py/tests/test_docs.pypython
res = await client.queue("orders").partition("customer-42").push([
    {"data": {"orderId": 9137, "amount": 99.5}}
])
clients/client-go/tests/docs_test.gogo
res, err := client.Queue("orders").
	Partition("customer-42").
	Push(map[string]any{"orderId": 9137, "amount": 99.5}).
	Execute(ctx)
if err != nil {
	return err
}
// res[0].Status == "queued"
clients/client-rust/tests/docs.rsrust
let res = q
    .queue("orders")
    .partition("customer-42")
    .push(serde_json::json!({ "orderId": 9137, "amount": 99.5 }))
    .await
    .unwrap();

Raw HTTP, with an explicit deduplication key:

curl -sS -X POST http://localhost:6632/api/v1/push -H 'content-type: application/json' -d '{"items":[{"queue":"orders.created","partition":"customer-42","transactionId":"order-8891-created","payload":{"orderId":8891}}]}'

Encryption at rest

If the queue has encryptionEnabled and the broker has a valid QUEEN_ENCRYPTION_KEY, payloads are encrypted before they are packed into a segment, and pops decrypt them transparently. A malformed or wrong-length key disables encryption and stores plaintext with only a warning in the log. It is not a boot failure and not a push failure. Self-hosting covers key handling.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close