Skip to content

POST /api/v1/transaction

Pushes, acks and the key/value and timer riders in a single PostgreSQL transaction: the bundle shape, the six-step lock order, every rollback cause, the result array, and the one case that is exactly-once.

Updated View as Markdown

One call, one PostgreSQL transaction, all or nothing. This is how a pipeline stage hands work forward: acknowledge the message you just processed and write the next stage’s message so that either both are durable or neither is. Access level read-write.

The route exists because the alternative (ack then push, or push then ack) always has a window. Ack first and a crash loses the work; push first and a crash duplicates it. Here both operations are inside one commit, so there is no window.

clients/client-js/test-v2/docs.jsjs
await client
  .queue('orders')
  .group('invoicing')
  .subscriptionMode('all')
  .each()
  .autoAck(false) // the acknowledgement belongs to the transaction, not to the loop
  .limit(1)
  .idleMillis(5000)
  .consume(async (message) => {
    // commit() throws when the broker rejects the bundle, so reaching the
    // line after it means the ack and the push are both durable.
    await client
      .transaction()
      .queue('invoices')
      .push([{ data: { orderId: message.data.orderId, invoiced: true } }])
      .ack(message, 'completed', { consumerGroup: 'invoicing' })
      .commit()
  })

One transaction, many partitions, many queues

A bundle is not limited to one ack and one push. It may acknowledge batches leased from any number of partitions, across any number of queues, and push to any number of queues and partitions. All of it commits together.

That is what makes a fan-in stage possible. Consume from two input queues, merge, and write one result: either both inputs are acknowledged and the merged message is stored, or neither happened.

{
  "operations": [
    {
      "type": "ack",
      "transactionId": "order-8891-created",
      "partitionId": "b1c2d3e4-5f60-7180-9a0b-1c2d3e4f5061",
      "consumerGroup": "merge",
      "status": "completed"
    },
    {
      "type": "ack",
      "transactionId": "payment-7742-settled",
      "partitionId": "c2d3e4f5-6071-8290-ab1c-2d3e4f506172",
      "consumerGroup": "merge",
      "status": "completed"
    },
    {
      "type": "push",
      "items": [
        {
          "queue": "orders.enriched",
          "partition": "customer-42",
          "transactionId": "enriched-8891",
          "payload": { "orderId": 8891, "paidAt": "2026-08-17T09:12:03Z" }
        }
      ]
    }
  ],
  "requiredLeases": [
    "0198f2c1-9a7b-7d20-8e31-4c5d6e7f8a90",
    "0198f2c1-9a7b-7d20-8e31-4c5d6e7f8b01"
  ]
}

Inside the stored procedure this is a loop, not a special case. One set-based statement pre-locks every push partition in the bundle in ascending queen.log_partitions.id order, then a push loop runs over N push elements, then an ack loop runs over N ack elements in ascending (partitionId, consumerGroup) order. Nothing on that path assumes one of each.

The step that makes several leases work is lease resolution. Each ack group’s lease is read from its own queen.log_consumers row rather than from the request, because requiredLeases carries no lease-to-partition mapping and its single-hint fallback goes ambiguous the moment two distinct leases appear. Exactly one live lease exists per (partition, group), so the consumer row is authoritative and a bundle carrying several distinct leases resolves correctly. The full resolution order is spelled out under “How the lease for an ack is found” below.

Two contract tests in clients/client-js/test-v2/transaction.js cover exactly this shape: transactionWithPartitions acks two partitions of one queue in a single bundle, and transactionMultipleQueues acks from two queues and pushes the merged result to a third.

Request

{
  "operations": [
    {
      "type": "ack",
      "transactionId": "order-8891-created",
      "partitionId": "b1c2d3e4-5f60-7180-9a0b-1c2d3e4f5061",
      "consumerGroup": "billing",
      "status": "completed"
    },
    {
      "type": "push",
      "items": [
        {
          "queue": "invoices.requested",
          "partition": "customer-42",
          "transactionId": "invoice-for-order-8891",
          "payload": { "orderId": 8891 }
        }
      ]
    }
  ],
  "requiredLeases": ["0198f2c1-9a7b-7d20-8e31-4c5d6e7f8a90"]
}
Field Type Required Meaning
operations array yes the operations, in order; a missing or non-array value is a 400
requiredLeases array of strings no lease ids the acks belong to; this is where the SDK builders put the popped leaseId
kv array no key/value operations riding the bundle; see “kv and timers riders”
timers array no timer schedules and cancels riding the bundle; see “kv and timers riders”

Only two operation types exist: push and ack. Any other type fails the whole call with a 400 and nothing is written, including a type the broker does not recognise. The check runs before any database work.

kv and timers are the two riders, and they are separate arrays rather than new operations types on purpose: they commit in their own step of the lock order, before anything else in the bundle. A bundle that uses neither carries neither key, so its body is byte-identical to the one it sent before the riders existed. operations may be empty when a rider is present: a bundle that only writes a key is a legitimate bundle.

push operations

A push operation takes either the nested form with an items array, or a flat form where the operation object is the item:

{ "type": "push", "queue": "invoices.requested", "payload": { "orderId": 8891 } }
Field Type Default
queue string empty string. Supply it
partition string "Default"
payload any JSON value {} when neither payload nor data is present
data any JSON value an accepted alias for payload
transactionId string the message id the broker mints
traceId string (UUID) absent; this is the only write path that stores a trace id

Two differences from POST /api/v1/push are worth internalising: here data is accepted as an alias for payload and a missing payload defaults to an empty object rather than failing the parse, and here traceId is honoured instead of ignored.

ack operations

Field Type Default
transactionId string empty
partitionId string (UUID) empty
status string completed
consumerGroup string __QUEUE_MODE__
leaseId string absent; falls back to the resolution order below
error string absent; the reason recorded if this ack dead-letters

status is normalised exactly as on the ack route: completed, retry and dlq survive to SQL, anything unrecognised becomes failed. retry and dlq behave identically inside a transaction: they are not collapsed to a boolean.

How the lease for an ack is found

Acks are validated against the live lease, and the SDK builders put the lease id in requiredLeases rather than on each operation. The broker resolves each ack group’s lease in this order:

  1. a leaseId on the operation itself, if you sent one;
  2. the current lease holder for that (partition, consumerGroup), read from the consumer row, which is authoritative because exactly one live lease exists per pair;
  3. the single unambiguous entry in requiredLeases, if there is exactly one distinct value.

If no lease information appears anywhere in the request, every ack is treated as a lease-less ack: the lease check is skipped and the cursor still advances, matching the direct ack route. If you did supply a lease and it is expired or wrong, the ack fails and the transaction rolls back.

kv and timers riders

The riders are what let an application’s own state commit with the cursor advance. A key/value write, the pushed output and the ack are one commit, so there is no state in which the work is marked done and the message will be redelivered, or acknowledged and unmarked.

{
  "operations": [
    { "type": "ack", "transactionId": "order-8891-created", "partitionId": "b1c2…", "consumerGroup": "billing", "status": "completed" }
  ],
  "requiredLeases": ["0198f2c1-9a7b-7d20-8e31-4c5d6e7f8a90"],
  "kv": [
    { "op": "putIfAbsent", "ns": "charges", "key": "charge:8891", "value": { "chargeId": "ch_1" }, "ttlSeconds": 86400, "required": true }
  ],
  "timers": [
    { "op": "schedule", "queue": "reminders", "key": "order-8891", "delaySeconds": 86400, "payload": { "orderId": 8891 } }
  ]
}

required is an opt-in escalation, and it is the whole difference between a report and a gate. Without it, a putIfAbsent whose key already exists comes back applied: false and the bundle still commits, ack included. With it, the lost precondition rolls the whole bundle back, so a concurrent worker that got there first is the only one whose ack lands.

Two limits apply to the riders on this route and not to POST /api/v1/kv, because a bundle holds the outermost lock of the call while it runs: the per-call ceilings drop to 64 operations and 256 keys, and getPrefix is refused outright. An unbounded read under the outermost lock space is the one shape that cannot be allowed here; run it outside the bundle.

Atomicity

The whole bundle runs inside one call to one stored procedure, which is one transaction. Every failure path raises, so any single failure takes every other operation down with it.

A transaction rolls back when:

Cause Detail
a duplicate push any pushed transactionId that already exists in its partition inside the deduplication window aborts the bundle with a QDUP error. Unlike /api/v1/push, where a duplicate is a per-item duplicate status, here it is a rollback
an ack the broker cannot resolve every acked transactionId must resolve in its partition’s index. One that resolves nowhere (never pushed, or its index row has aged past the txns window) makes the stored procedure raise QTXN ack references unknown transactionId; transaction rolled back, and the rollback takes the pushes and riders with it, so nothing is written. An ack at or below the cursor is not this case: it resolves as already committed
a rejected ack an invalid or expired lease, or a position beyond the leased batch
a foreign partition multi-tenant deployments: an acked partitionId that belongs to another tenant
a lost KV precondition a kv op carrying required: true whose precondition did not hold. This is the expected outcome of a legitimate redelivery rather than a fault, and it has its own response shape below
a timer beyond the horizon a timers schedule further out than this cell allows, reported as timer_horizon_exceeded

Intra-bundle duplicate pushes are the one exception, and they are resolved before SQL: two push items with the same transactionId in the same (queue, partition) inside one transaction are first-wins. The second is echoed as a result with duplicate: true, produces no frame, and does not roll the transaction back.

Deadlock safety comes from a fixed total lock order shared with the ordinary push path, so concurrent transactions touching overlapping sets can never form a cycle. The six steps of that order are under “What the atomicity is made of” below.

Response

200 OK on success:

{
  "transactionId": "0198f2c1-b0c1-7e40-8a52-9d0e1f2a3b4c",
  "success": true,
  "results": [
    { "index": 0, "type": "ack", "success": true, "transactionId": "order-8891-created", "error": null, "dlq": false },
    { "index": 1, "type": "push", "success": true, "transactionId": "invoice-for-order-8891", "messageId": "0198f2c1-…", "queueName": "invoices.requested" }
  ]
}
Field Type Meaning
transactionId string an id the broker mints for this call; it is not any message’s deduplication key
success boolean whether the transaction committed
results array one element per flattened operation, indexed in request order; empty on failure

Push results carry index, type: "push", success, transactionId (the deduplication key used), messageId and queueName, plus duplicate: true for an intra-bundle duplicate. Ack results carry index, type: "ack", success, transactionId, error and dlq.

Because the transaction is all-or-nothing, every element of a successful results array says success: true. Per-operation failure does not exist here. Read the top-level success first, and treat results as an echo that tells you which message ids were assigned.

Failure keeps the same shape, with the reason at the top level:

{
  "transactionId": "0198f2c1-b0c1-7e40-8a52-9d0e1f2a3b4c",
  "success": false,
  "reason": "duplicate",
  "error": "QDUP duplicate messages in queue \"invoices.requested\" partition \"customer-42\"; transaction rolled back",
  "results": []
}

reason is a code from a closed taxonomy and is the only thing to branch on: bad_request, duplicate, ack_rejected, kv_precondition, timer_horizon_exceeded, payload_too_large, misaligned, db_error. Matching on the prose in error is what the code exists to end. The full table is in Errors and status codes.

A lost required precondition carries more, because the caller usually wants to know who won without a second round trip:

{
  "transactionId": "0198f2c1-b0c1-7e40-8a52-9d0e1f2a3b4c",
  "success": false,
  "reason": "kv_precondition",
  "failedIndex": 0,
  "kvReason": "exists",
  "version": 90101,
  "value": { "chargeId": "ch_1" },
  "results": []
}

failedIndex is in the flat index space of results. value and version are the winner’s, so a consumer that lost the gate can return the original result rather than recomputing it. Note that nothing in the bundle committed, including the ack: a consumer on this path still has to take the message off its own cursor with a separate ack, or it will be redelivered forever.

Status codes

Code When
200 the call was processed, including a rollback, which is success: false with an error
400 the body is not JSON, operations is missing or not an array, or an operation has an unrecognised type
403 authentication is on and the token has no read-write role
413 the body exceeds 64 MiB
500 no database connection was available

A rolled-back transaction is a 200. Check success, not the status code.

What the atomicity is made of

There is no coordinator. The bundle travels as one cached prepared statement in one round trip, a single SELECT over queen.log_transaction_wire_v1, executed on one pooled connection. The atomicity is PostgreSQL’s own, and the whole bundle is one server-side statement inside it.

Read it as four absences:

  • No producer epoch and no fencing token. Nothing has to be registered or leased before you are allowed to write.
  • No two-phase commit and no prepared transaction. Nothing is ever left in a prepared state, so a crash cannot strand a bundle that an operator then has to find and ROLLBACK PREPARED by hand.
  • No coordinator log to compact, replicate, or lose track of.
  • No half-applied state. The call either committed or it did not, so there is nothing for an operator to reconcile. Recovery is a client-side retry, and the next section is about what makes that retry safe.

What holds concurrent bundles apart is a lock order rather than a protocol, and it is one total order in six steps: the kv ops first, then the timers ops, then provisioning of any missing queue or partition rows, then one set-based pre-lock over every push partition in ascending queen.log_partitions.id order, then the pushes, then the acks in ascending (partition_id, consumer_group) order. Consumer-row locks are only ever taken after every partition-row lock, so the two lock spaces cannot form a cross-space cycle. The same partition pre-lock is used by the ordinary push path and by retention, which is why a transaction cannot deadlock against either.

The riders sit at the top of that order rather than the bottom, and the reason is that for them failure is the common path. An idempotency marker loses on every legitimate redelivery, and losing at the first step costs one insert and a raise before a single partition lock is taken. At the bottom it would cost the entire bundle written and then thrown away, which would make the most frequent outcome of the feature its most expensive one. Timers have no choice in any case: the sweeper’s fire takes timers before partitions, so timers at the bottom would deadlock against it immediately. The rule is that both riders sit at the top together, never one at each end: a split introduces arcs in both directions inside one protocol, and each half still looks correct on its own in review.

The boundary is worth stating plainly: the guarantee covers everything inside that one PostgreSQL, meaning the pushed messages, the committed cursors, any dead-letter rows and, since the riders exist, your own key/value state and timers. It covers nothing outside it. If the same handler also charged a card or called a third-party API, that side effect is not in the transaction and still needs its own idempotency key.

It is not exactly-once end to end

The commit is atomic; the round trip is not. If the response is lost (a dropped connection, a timeout on your side), you do not know whether the transaction committed. Retrying duplicates the pushes unless their transactionIds are deterministic and the retry arrives inside the queue’s deduplication window.

The pattern that makes this safe is to derive each pushed transactionId from the input you are processing rather than generating a fresh UUID: invoice-for-order-8891, not crypto.randomUUID(). Then a retried transaction either commits once or aborts on the duplicate, and both end states are the same one. With random ids, a retry after a lost response writes the message twice.

That does mean a retry of an already-committed transaction comes back success: false with a QDUP error. It is not a new failure to act on: the duplicate proves the message is already stored, and the acks of the original attempt committed with it. Distinguish QDUP from the other rollback causes before you retry again.

The ack side of the retry is safe on its own: a second attempt at the same ack resolves at or below the cursor and is reported as already committed rather than double-committing.

The one case where it is exactly-once

The heading above stays true for external effects, and it is worth naming the case it does not cover, because that case is most internal work.

When the effect is itself a row in this PostgreSQL, written through the kv rider, there is no second system and no second commit. The effect, the marker, the pushed output and the cursor advance are one COMMIT. Not “atomic enough”: one. There is no interleaving left to describe, and no reconciliation to write, because there is no state in which some of it happened.

What keeps the external case out of reach is not a limitation of this route. There is no atomic commit protocol between a database and a non-transactional remote service, so a crash between the charge and the commit is unclosable by any broker. What closes it is the remote service’s own idempotency key, and the natural value for that key is the one you already minted for the marker.

One property does carry into the external case, and it is the reason to put the marker here rather than beside the broker: the ack travels with the bundle, so if the lease expired while the work was running, the ack is refused and the marker is refused with it. A compare-and-swap in a second store cannot do that. It succeeds from a worker that no longer owns the message, because nothing in that store knows the message was taken away.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close