---
title: "Exactly-once charging"
description: "Charging N orders exactly N times under a forced redelivery, with the idempotency marker written in the same transaction as the acknowledgement, and an honest account of where that guarantee stops."
---

> Queen MQ documentation, for AI agents
> Complete self-contained summary of Queen MQ: https://queenmq.com/llms-brief.txt
> Fetch that first when the question is about the product rather than about this page.
> Index of all pages: https://queenmq.com/llms.txt

# Exactly-once charging

A billing run double-charged nineteen customers on a Tuesday afternoon. Nothing had crashed and
nothing was lost. A consumer took a batch, charged the cards, and was still writing its "done"
flag to a key/value store on the side when its lease expired. The broker did what a broker must
do and handed the batch to somebody else, and the somebody else found no flag.

The flag was not written too slowly. It was in the wrong place. Living in a second data system, it
could not commit with the acknowledgement, so there was always an interval in which one of the two
existed and the other did not, and that interval is where the work happens twice. The correction is
structural: put the marker in the database that already holds the queue, and write it in the same
transaction as the ack. A marker without its acknowledgement, and an acknowledgement without its
marker, then stop being representable.

The program builds that, five orders at a time. One consumer group charges them, with one order
scripted to fail before it charges anything so the redelivery is forced rather than hoped for, and a
second consumer group reads the same five orders from the beginning, which is a redelivery with the
cause removed. It measures three things: five orders produce exactly five charges, every order on
the second pass reports that it did not run, and the handler that failed before its commit left no
marker behind to block its own retry.

The last of those is the one that is easy to get wrong in the other direction. A marker written
early, before the work, turns a transient failure into a permanent skip: the order is marked, the
charge never happened, and nothing will ever try again. That is why the marker is not a flag you
set when you start.

### JavaScript

```js title="examples/apps/js/exactly-once.mjs"
//
// Charging an order exactly once, under redelivery.
//
// The war story is a billing run that double-charged nineteen customers on a
// Tuesday afternoon. Nothing had crashed and nothing was lost: a consumer took
// a batch, charged the cards, and was still writing its "done" flag to a side
// store when its lease expired. The broker did what a broker must do and gave
// the batch to somebody else, and the somebody else found no flag.
//
// The flag was in the wrong place. It was in a second data system, so it could
// not commit with the acknowledgement, and any window between the two is a
// window in which the work happens twice.
//
// Here the marker is a row in the same PostgreSQL as the queue, written in the
// same transaction as the ack. There is no window. Either the order is marked
// and acknowledged, or neither, and a redelivery finds the marker and does
// nothing.
//
//   orders
//     └── group "charger"   marker + ack in ONE transaction
//           └── group "replay"  reads the same orders again, charges nothing
//
// Run it:
//   QUEEN_URL=http://localhost:6632 node exactly-once.mjs

import { Queen } from 'queen-mq'

const QUEEN_URL = process.env.QUEEN_URL || 'http://localhost:6632'
const RUN = Date.now().toString(36)

// Two suffixes, not one. The queue name needs it because delete-then-recreate
// leaves stale partition state for up to 30 seconds; the KV namespace needs it
// for the same reason in reverse -- a marker outlives the run that wrote it, so
// a second run under the same namespace would find every order already charged
// and pass without charging anything.
const ORDERS = `app-js-exactly-once-${RUN}`
const NS = `app-js-exactly-once-${RUN}`
const GROUP = 'charger'
const REPLAY_GROUP = 'replay'

// Five orders. ORD-3 is scripted to fail once, before it charges anything and
// before it commits, which is the interesting failure: the one that must leave
// no trace at all.
const ORDER_IDS = ['ORD-1', 'ORD-2', 'ORD-3', 'ORD-4', 'ORD-5']
const CRASHING_ORDER = 'ORD-3'

// Six deliveries in the charging phase, not five: ORD-3 arrives twice, once to
// fail and once to succeed. The number is what ENDS the phase, and that is the
// rule this program is built on -- wait for a total, never for silence. A phase
// that stopped when nothing had arrived for a while would pass on a broker that
// had delivered nothing at all.
const CHARGE_DELIVERIES = ORDER_IDS.length + 1

// And the deadline behind the total, so a stall is a failure rather than a
// hang. Reaching it ends the phase early, short of the count, and the count
// check that follows is what reports it.
const PHASE_MS = 30000

let checks = 0
const assert = (condition, description) => {
  if (!condition) throw new Error(description)
  checks++
  console.log(`  ok: ${description}`)
}

// The external effect. Every call is a real charge against a real card, which
// is the whole reason this program exists: the ledger below is what the
// customer's statement would show.
const ledger = []
const chargeCard = (order) => {
  const chargeId = `ch_${order.orderId}_${ledger.length}`
  ledger.push({ orderId: order.orderId, chargeId, cents: order.cents })
  return chargeId
}

const attempts = new Map()
const markerKey = (orderId) => `charge:${orderId}`

const queen = new Queen({ url: QUEEN_URL, handleSignals: false })

try {
  console.log(`broker ${QUEEN_URL}`)

  await queen.queue(ORDERS).config({ leaseTime: 30, retryLimit: 5 }).create()

  // ------------------------------------------------------------------ queuing
  console.log('\nqueuing orders')
  for (const orderId of ORDER_IDS) {
    await queen.queue(ORDERS).push({
      transactionId: `order-${orderId}`,
      data: { orderId, cents: 1000 + ORDER_IDS.indexOf(orderId) },
    })
  }
  console.log(`  ${ORDER_IDS.length} orders queued`)

  // ------------------------------------------------------------------ charging
  //
  // handle() is the whole pattern, and it is four steps in a fixed order.
  //
  // It returns `{ran}`: true when THIS delivery performed the charge, false
  // when it found the marker and did nothing. A redelivery -- of any cause, a
  // lease that expired, a broker that restarted, an operator replaying a queue
  // -- must return false, and that is the property this program measures.
  const observed = []

  const handle = async (msg) => {
    const order = msg.data
    attempts.set(order.orderId, (attempts.get(order.orderId) ?? 0) + 1)

    // 1. Has this order already been charged? `found` is a separate field from
    //    `value` because null is a legal stored value, so absence is never
    //    inferred from the value being empty.
    const marker = await queen.kv.get(NS, markerKey(order.orderId))
    if (marker.found) {
      // Nothing to do, but the message still has to be taken off this group's
      // cursor, or it comes back forever.
      await queen.ack(msg, 'completed', { group: msg.consumerGroup })
      return { ran: false, chargeId: marker.value.chargeId }
    }

    // 2. The scripted failure. It happens BEFORE the charge and before the
    //    commit, which is the ordering a real handler should aim for: whatever
    //    can fail without an external effect should fail there.
    if (order.orderId === CRASHING_ORDER && attempts.get(order.orderId) === 1) {
      throw new Error(`${order.orderId}: card network timed out`)
    }

    // 3. The external effect.
    const chargeId = chargeCard(order)

    // 4. The marker and the acknowledgement, in ONE transaction.
    //
    //    `required: true` is what makes the putIfAbsent a GATE rather than a
    //    verdict. Without it a lost race would come back `applied:false` and
    //    the ack would still commit; with it, a lost race rolls the whole
    //    bundle back, ack included, so a concurrent worker that got there first
    //    is the only one whose ack lands.
    //
    //    The lease travels with the ack. If this worker's lease expired while
    //    it was charging -- the exact failure in the war story -- the ack is
    //    refused and the marker write is refused with it. That is the guarantee
    //    a compare-and-swap cannot give: an `expect` on a version that still
    //    matches succeeds even from a worker that no longer owns the message.
    const res = await queen
      .transaction()
      .kv.putIfAbsent(NS, markerKey(order.orderId), { chargeId, cents: order.cents }, { ttl: '1h', required: true })
      .ack(msg, 'completed', { consumerGroup: msg.consumerGroup })
      .commit()

    // A lost gate is RETURNED, not thrown: `success:false` with
    // `reason:'kv_precondition'` and HTTP 200. It is the most frequent
    // legitimate outcome of this shape, so it does not belong in an error path,
    // a retry policy or an error metric.
    if (res.success === false && res.reason === 'kv_precondition') {
      await queen.ack(msg, 'completed', { group: msg.consumerGroup })
      return { ran: false, chargeId: res.value?.chargeId }
    }

    return { ran: true, chargeId }
  }

  console.log('\ncharging')
  await queen
    .queue(ORDERS)
    .group(GROUP)
    .subscriptionMode('all')
    .autoAck(false)
    .each()
    .limit(CHARGE_DELIVERIES)
    .idleMillis(PHASE_MS)
    .consume(async (msg) => {
      try {
        const { ran } = await handle(msg)
        observed.push({ group: GROUP, orderId: msg.data.orderId, ran })
        console.log(`  ${msg.data.orderId}: ${ran ? 'charged' : 'already charged, skipped'}`)
      } catch (err) {
        // autoAck is off, so the negative acknowledgement is explicit. It
        // clamps the cursor below this message and charges one unit of the
        // retry budget, which is what brings the order back.
        console.log(`  ${msg.data.orderId}: ${err.message} (will be redelivered)`)
        await queen.ack(msg, 'failed', { group: msg.consumerGroup, error: err.message })

        // The claim this example exists to prove, checked at the only moment it
        // can be checked: right after a handler failed before its commit.
        const marker = await queen.kv.get(NS, markerKey(msg.data.orderId))
        assert(!marker.found, `${msg.data.orderId} failed before committing and left no marker behind`)
      }
    })

  assert(
    observed.filter(o => o.group === GROUP).length === ORDER_IDS.length,
    'the charger reached a decision on every order'
  )

  // ------------------------------------------------------------------- replay
  //
  // A second consumer group with subscriptionMode 'all' reads the same orders
  // from the beginning. This is a redelivery with the cause removed: the
  // messages are identical, the handler is identical, and the only thing
  // standing between them and a second charge is the marker.
  console.log('\nreplaying')
  await queen
    .queue(ORDERS)
    .group(REPLAY_GROUP)
    .subscriptionMode('all')
    .autoAck(false)
    .each()
    .limit(ORDER_IDS.length)
    .idleMillis(PHASE_MS)
    .consume(async (msg) => {
      const { ran } = await handle(msg)
      observed.push({ group: REPLAY_GROUP, orderId: msg.data.orderId, ran })
      console.log(`  ${msg.data.orderId}: ran === ${ran}`)
    })

  // ----------------------------------------------------------------- checking
  console.log('\nchecking')

  assert(
    ledger.length === ORDER_IDS.length,
    `${ORDER_IDS.length} orders produced exactly ${ORDER_IDS.length} charges (got ${ledger.length})`
  )

  const perOrder = new Map()
  for (const row of ledger) perOrder.set(row.orderId, (perOrder.get(row.orderId) ?? 0) + 1)
  assert(
    ORDER_IDS.every(id => perOrder.get(id) === 1),
    'every order was charged exactly once, none twice and none not at all'
  )

  assert(
    attempts.get(CRASHING_ORDER) >= 2,
    `${CRASHING_ORDER} was delivered again after it failed (${attempts.get(CRASHING_ORDER)} attempts)`
  )

  const replayed = observed.filter(o => o.group === REPLAY_GROUP)
  assert(
    replayed.length === ORDER_IDS.length,
    `the replay group received all ${ORDER_IDS.length} orders again (got ${replayed.length})`
  )
  assert(
    replayed.every(o => o.ran === false),
    'every order on the second pass reported ran === false'
  )
  assert(
    ledger.length === ORDER_IDS.length,
    `the replay charged nothing: the ledger is still ${ORDER_IDS.length} rows`
  )

  // The markers are readable state, not an internal detail: each one carries
  // the id of the charge it stands for, so a support engineer can answer "was
  // this order billed, and under which charge" without a second system.
  const markers = await queen.kv.getMany(NS, ORDER_IDS.map(markerKey))
  assert(markers.rows.length === ORDER_IDS.length, `all ${ORDER_IDS.length} markers exist`)
  assert(markers.missing.length === 0, 'no order is missing its marker')
  assert(
    markers.rows.every(r => ledger.some(l => l.chargeId === r.value.chargeId)),
    'each marker names the charge that was actually made'
  )

  console.log(`\n  ledger: ${ledger.map(l => `${l.orderId}=${l.chargeId}`).join(', ')}`)

  console.log(`\nPASS: ${checks} checks`)
} catch (err) {
  console.error(`\nFAIL: ${err.message}`)
  process.exitCode = 1
} finally {
  // ------------------------------------------------------------------- purge
  //
  // Two things to remove, and the second is the one that is easy to forget: the
  // markers are rows in their own table, so deleting the queue does not take
  // them with it. They would expire on their own -- that is what the mandatory
  // TTL bought -- but only once the sweeper gets to them, and an example that
  // needs a background task to tidy up after itself is not one.
  //
  // Unconditional, in a finally, because a run that FAILED is exactly the run
  // whose leftovers matter: markers surviving into the next run of the same
  // namespace would make the next run pass without charging anything.
  //
  // Best effort: a purge that threw would replace the real failure with its own.
  try {
    for (const orderId of ORDER_IDS) await queen.kv.delete(NS, markerKey(orderId))
    await queen.queue(ORDERS).delete()
  } catch (err) {
    console.error(`  (purge incomplete: ${err.message})`)
  }
  await queen.close()
}
```
### Python

```python title="examples/apps/py/exactly_once.py"
#
# Charging an order exactly once, under redelivery.
#
# The war story is a billing run that double-charged nineteen customers on a
# Tuesday afternoon. Nothing had crashed and nothing was lost: a consumer took
# a batch, charged the cards, and was still writing its "done" flag to a side
# store when its lease expired. The broker did what a broker must do and gave
# the batch to somebody else, and the somebody else found no flag.
#
# The flag was in the wrong place. It was in a second data system, so it could
# not commit with the acknowledgement, and any window between the two is a
# window in which the work happens twice.
#
# Here the marker is a row in the same PostgreSQL as the queue, written in the
# same transaction as the ack. There is no window. Either the order is marked
# and acknowledged, or neither, and a redelivery finds the marker and does
# nothing.
#
#   orders
#     `-- group "charger"   marker + ack in ONE transaction
#           `-- group "replay"  reads the same orders again, charges nothing
#
# Run it:
#   QUEEN_URL=http://localhost:6632 python3 exactly_once.py

import asyncio
import os
import sys
import time
from datetime import timedelta

from queen import Queen

QUEEN_URL = os.environ.get("QUEEN_URL", "http://localhost:6632")

# Two suffixes, not one. The queue name needs it because delete-then-recreate
# leaves stale partition state for up to 30 seconds; the KV namespace needs it
# for the same reason in reverse -- a marker outlives the run that wrote it, so
# a second run under the same namespace would find every order already charged
# and pass without charging anything.
RUN = f"{int(time.time() * 1000):x}"
ORDERS = f"app-py-exactly-once-{RUN}"
NS = f"app-py-exactly-once-{RUN}"
GROUP = "charger"
REPLAY_GROUP = "replay"

# Five orders. ORD-3 is scripted to fail once, before it charges anything and
# before it commits, which is the interesting failure: the one that must leave
# no trace at all.
ORDER_IDS = ["ORD-1", "ORD-2", "ORD-3", "ORD-4", "ORD-5"]
CRASHING_ORDER = "ORD-3"

# Six deliveries in the charging phase, not five: ORD-3 arrives twice, once to
# fail and once to succeed. The number is what ENDS the phase, and that is the
# rule this program is built on -- wait for a total, never for silence. A phase
# that stopped when nothing had arrived for a while would pass on a broker that
# had delivered nothing at all.
CHARGE_DELIVERIES = len(ORDER_IDS) + 1

# And the deadline behind the total, so a stall is a failure rather than a hang.
# Reaching it ends the phase early, short of the count, and the count check that
# follows is what reports it.
PHASE_MS = 30000

CHECKS = 0

# The external effect. Every entry is a real charge against a real card, which
# is the whole reason this program exists: the ledger is what the customer's
# statement would show.
LEDGER: list = []
ATTEMPTS: dict = {}


def check(condition: bool, description: str) -> None:
    """Record one verified fact, or abort the run.

    This raises instead of using the `assert` statement, because `python3 -O`
    removes `assert` and the checks are the whole point of the program.
    """
    global CHECKS
    if not condition:
        raise AssertionError(description)
    CHECKS += 1
    print(f"  ok: {description}")


def charge_card(order: dict) -> str:
    charge_id = f"ch_{order['orderId']}_{len(LEDGER)}"
    LEDGER.append({"orderId": order["orderId"], "chargeId": charge_id, "cents": order["cents"]})
    return charge_id


def marker_key(order_id: str) -> str:
    return f"charge:{order_id}"


async def main() -> int:
    queen = Queen(url=QUEEN_URL)
    verdict, failed = "", False

    try:
        print(f"broker {QUEEN_URL}")

        await queen.queue(ORDERS).config({"lease_time": 30, "retry_limit": 5}).create()

        # ------------------------------------------------------------ queuing
        print("\nqueuing orders")
        for index, order_id in enumerate(ORDER_IDS):
            await queen.queue(ORDERS).push(
                {"transactionId": f"order-{order_id}", "data": {"orderId": order_id, "cents": 1000 + index}}
            )
        print(f"  {len(ORDER_IDS)} orders queued")

        # ----------------------------------------------------------- charging
        #
        # handle() is the whole pattern, and it is four steps in a fixed order.
        #
        # It returns whether THIS delivery performed the charge: False when it
        # found the marker and did nothing. A redelivery -- of any cause, a
        # lease that expired, a broker that restarted, an operator replaying a
        # queue -- must return False, and that is the property this program
        # measures.
        observed: list = []

        async def handle(msg) -> bool:
            order = msg["data"]
            order_id = order["orderId"]
            ATTEMPTS[order_id] = ATTEMPTS.get(order_id, 0) + 1
            group = msg.get("consumerGroup")

            # 1. Has this order already been charged? "found" is a separate key
            #    from "value" because None is a legal stored value, so absence
            #    is never inferred from the value being empty.
            marker = await queen.kv.get(NS, marker_key(order_id))
            if marker["found"]:
                # Nothing to do, but the message still has to leave this
                # group's cursor, or it comes back forever.
                await queen.ack(msg, "completed", {"group": group})
                return False

            # 2. The scripted failure. It happens BEFORE the charge and before
            #    the commit, which is the ordering a real handler should aim
            #    for: whatever can fail without an external effect should fail
            #    there.
            if order_id == CRASHING_ORDER and ATTEMPTS[order_id] == 1:
                raise RuntimeError(f"{order_id}: card network timed out")

            # 3. The external effect.
            charge_id = charge_card(order)

            # 4. The marker and the acknowledgement, in ONE transaction.
            #
            #    required=True is what makes the put_if_absent a GATE rather
            #    than a verdict. Without it a lost race would come back
            #    applied=False and the ack would still commit; with it, a lost
            #    race rolls the whole bundle back, ack included, so a
            #    concurrent worker that got there first is the only one whose
            #    ack lands. (`tx.once(...)` is the same thing under a shorter
            #    name; it is spelled out here so `required` is visible.)
            #
            #    The lease travels with the ack. If this worker's lease expired
            #    while it was charging -- the exact failure in the war story --
            #    the ack is refused and the marker write is refused with it.
            #    That is the guarantee a compare-and-swap cannot give: an
            #    `expect` on a version that still matches succeeds even from a
            #    worker that no longer owns the message.
            res = await (
                queen.transaction()
                .kv.put_if_absent(
                    NS,
                    marker_key(order_id),
                    {"chargeId": charge_id, "cents": order["cents"]},
                    # The Python client takes a timedelta rather than the
                    # JavaScript client's "1h" string; both resolve to the one
                    # field the wire has, ttlSeconds.
                    ttl=timedelta(hours=1),
                    required=True,
                )
                .ack(msg, "completed", {"consumer_group": group})
                .commit()
            )

            # A lost gate is RETURNED, not raised: success=False with
            # reason="kv_precondition" and HTTP 200. It is the most frequent
            # legitimate outcome of this shape, so it does not belong in an
            # error path, a retry policy or an error metric.
            if res.get("success") is False and res.get("reason") == "kv_precondition":
                await queen.ack(msg, "completed", {"group": group})
                return False

            return True

        print("\ncharging")

        async def charge(msg) -> None:
            group = msg.get("consumerGroup")
            try:
                ran = await handle(msg)
                observed.append({"group": GROUP, "orderId": msg["data"]["orderId"], "ran": ran})
                print(f"  {msg['data']['orderId']}: {'charged' if ran else 'already charged, skipped'}")
            except RuntimeError as err:
                # auto_ack is off, so the negative acknowledgement is explicit.
                # It clamps the cursor below this message and charges one unit
                # of the retry budget, which is what brings the order back.
                print(f"  {msg['data']['orderId']}: {err} (will be redelivered)")
                await queen.ack(msg, "failed", {"group": group, "error": str(err)})

                # The claim this example exists to prove, checked at the only
                # moment it can be checked: right after a handler failed before
                # its commit.
                marker = await queen.kv.get(NS, marker_key(msg["data"]["orderId"]))
                check(
                    not marker["found"],
                    f"{msg['data']['orderId']} failed before committing and left no marker behind",
                )

        await (
            queen.queue(ORDERS)
            .group(GROUP)
            .subscription_mode("all")
            .auto_ack(False)
            .each()
            .limit(CHARGE_DELIVERIES)
            .idle_millis(PHASE_MS)
            .consume(charge)
        )

        check(
            len([o for o in observed if o["group"] == GROUP]) == len(ORDER_IDS),
            "the charger reached a decision on every order",
        )

        # ------------------------------------------------------------ replay
        #
        # A second consumer group with subscription_mode "all" reads the same
        # orders from the beginning. This is a redelivery with the cause
        # removed: the messages are identical, the handler is identical, and
        # the only thing standing between them and a second charge is the
        # marker.
        print("\nreplaying")

        async def replay(msg) -> None:
            ran = await handle(msg)
            observed.append({"group": REPLAY_GROUP, "orderId": msg["data"]["orderId"], "ran": ran})
            print(f"  {msg['data']['orderId']}: ran is {ran}")

        await (
            queen.queue(ORDERS)
            .group(REPLAY_GROUP)
            .subscription_mode("all")
            .auto_ack(False)
            .each()
            .limit(len(ORDER_IDS))
            .idle_millis(PHASE_MS)
            .consume(replay)
        )

        # ----------------------------------------------------------- checking
        print("\nchecking")

        check(
            len(LEDGER) == len(ORDER_IDS),
            f"{len(ORDER_IDS)} orders produced exactly {len(ORDER_IDS)} charges (got {len(LEDGER)})",
        )
        per_order = {order_id: sum(1 for row in LEDGER if row["orderId"] == order_id) for order_id in ORDER_IDS}
        check(
            all(count == 1 for count in per_order.values()),
            "every order was charged exactly once, none twice and none not at all",
        )
        check(
            ATTEMPTS.get(CRASHING_ORDER, 0) >= 2,
            f"{CRASHING_ORDER} was delivered again after it failed ({ATTEMPTS.get(CRASHING_ORDER, 0)} attempts)",
        )

        replayed = [o for o in observed if o["group"] == REPLAY_GROUP]
        check(
            len(replayed) == len(ORDER_IDS),
            f"the replay group received all {len(ORDER_IDS)} orders again (got {len(replayed)})",
        )
        check(
            all(o["ran"] is False for o in replayed),
            "every order on the second pass reported that it did not run",
        )
        check(
            len(LEDGER) == len(ORDER_IDS),
            f"the replay charged nothing: the ledger is still {len(ORDER_IDS)} rows",
        )

        # The markers are readable state, not an internal detail: each one
        # carries the id of the charge it stands for, so a support engineer can
        # answer "was this order billed, and under which charge" without a
        # second system.
        markers = await queen.kv.get_many(NS, [marker_key(o) for o in ORDER_IDS])
        check(len(markers["rows"]) == len(ORDER_IDS), f"all {len(ORDER_IDS)} markers exist")
        check(len(markers["missing"]) == 0, "no order is missing its marker")
        charge_ids = {row["chargeId"] for row in LEDGER}
        check(
            all(row["value"]["chargeId"] in charge_ids for row in markers["rows"]),
            "each marker names the charge that was actually made",
        )

        print("\n  ledger: " + ", ".join(f"{row['orderId']}={row['chargeId']}" for row in LEDGER))

        verdict = f"\nPASS: {CHECKS} checks"
    except Exception as err:
        verdict, failed = f"\nFAIL: {err}", True
    finally:
        # -------------------------------------------------------------- purge
        #
        # Two things to remove, and the second is the one that is easy to
        # forget: the markers are rows in their own table, so deleting the queue
        # does not take them with it. They would expire on their own -- that is
        # what the mandatory TTL bought -- but only once the sweeper gets to
        # them, and an example that needs a background task to tidy up after
        # itself is not one.
        #
        # Unconditional, in a finally, because a run that FAILED is exactly the
        # run whose leftovers matter: markers surviving into the next run of the
        # same namespace would make the next run pass without charging anything.
        #
        # Best effort: a purge that raised would replace the real failure with
        # its own.
        try:
            for order_id in ORDER_IDS:
                await queen.kv.delete(NS, marker_key(order_id))
            await queen.queue(ORDERS).delete()
        except Exception as err:  # noqa: BLE001 - the run's verdict outranks this
            print(f"  (purge incomplete: {err})")
        await queen.close()

    sys.stdout.flush()
    print(verdict, file=sys.stderr if failed else sys.stdout)
    return 1 if failed else 0


if __name__ == "__main__":
    sys.exit(asyncio.run(main()))
```
### HTTP

```bash title="examples/apps/http/exactly-once.sh"
#!/usr/bin/env bash
#
# Charging an order exactly once, under redelivery, with nothing but curl.
#
# The war story is a billing run that double-charged nineteen customers on a
# Tuesday afternoon. Nothing had crashed and nothing was lost: a consumer took a
# batch, charged the cards, and was still writing its "done" flag to a side
# store when its lease expired. The broker did what a broker must do and gave
# the batch to somebody else, and the somebody else found no flag.
#
# The flag was in the wrong place. It was in a second data system, so it could
# not commit with the acknowledgement, and any window between the two is a
# window in which the work happens twice.
#
# Here the marker is a row in the same PostgreSQL as the queue, written in the
# same transaction as the ack. There is no window. Either the order is marked
# and acknowledged, or neither, and a redelivery finds the marker and does
# nothing.
#
#   orders
#     ├── group "charger"  marker + ack in ONE transaction
#     └── group "replay"   reads the same orders again, charges nothing
#
# There is no client library here and none is needed, and this file is worth
# reading even if you use one: an SDK's `kv.putIfAbsent(...)` inside a
# transaction is the `kv` array below, and the array is a key of the ROOT of the
# request beside `operations`, never an element of it. Everything an SDK hides
# is written out.
#
# Run it:
#   QUEEN_URL=http://localhost:6632 bash exactly-once.sh

set -euo pipefail

QUEEN_URL="${QUEEN_URL:-http://localhost:6632}"

# Two suffixes, not one. The queue name needs it because delete-then-recreate
# leaves stale partition state for up to 30 seconds; the KV namespace needs it
# for the same reason in reverse -- a marker outlives the run that wrote it, so a
# second run under the same namespace would find every order already charged and
# pass without charging anything. $$ is the process id, which keeps two runs in
# the same second apart.
RUN="$(date +%s)-$$"
ORDERS="app-http-exactly-once-$RUN"
NS="app-http-exactly-once-$RUN"

# The consumer groups. A group's cursor lives on the queue, and the queue name is
# already unique per run, so these need no suffix.
CHARGER=app-http-charger
REPLAY=app-http-replay

# Five orders. ORD-3 is scripted to fail once, before it charges anything and
# before it commits, which is the interesting failure: the one that must leave
# no trace at all.
ORDER_IDS="ORD-1 ORD-2 ORD-3 ORD-4 ORD-5"
ORDER_COUNT=5
CRASHING_ORDER=ORD-3

# Every pop long-polls for this many milliseconds and no longer.
POLL_MS=1000

# The bound that keeps a stall from becoming a hang. A phase that has not made
# every decision by then stops, and the count check that follows reports what was
# missing. Never wait for silence; wait for a total, with a deadline.
PHASE_MS=30000

command -v jq >/dev/null 2>&1 || { echo "FAIL: jq is not installed"; exit 1; }

CHECKS=0
TMP="$(mktemp -d)"

# The external effect. Every line is a real charge against a real card, which is
# the whole reason this program exists: the ledger is what the customer's
# statement would show.
LEDGER="$TMP/ledger"
: > "$LEDGER"

# One line per delivery handled: "<group> <order> <ran>". `ran` is the whole
# point -- true when THIS delivery performed the charge, false when it found the
# marker and did nothing.
OBSERVED="$TMP/observed"
: > "$OBSERVED"

# One line per delivery attempt, so the redelivery of the failed order can be
# counted rather than assumed.
ATTEMPTS="$TMP/attempts"
: > "$ATTEMPTS"

# One exit path for everything. A failed check calls fail(), which records the
# reason and exits 1; any other command that fails under `set -e` arrives here
# too, with its own status. FAIL is printed exactly once, and only on failure.
#
# It also purges, and there are two things to remove: the queue, and the markers.
# The second is the one that is easy to forget, because the markers are rows in
# their own table and deleting the queue does not take them with it. They would
# expire on their own -- that is what the mandatory ttlSeconds below bought --
# but only once the sweeper gets to them, and an example that needs a background
# task to tidy up after itself is not one.
#
# The purge is UNCONDITIONAL, because a run that failed is exactly the run whose
# leftovers matter: markers surviving into the next run of the same namespace
# would make the next run pass without charging anything. And it is best effort,
# with `|| true` throughout, so a purge that fails cannot overwrite the verdict.
cleanup() {
  local status=$?
  purge || true
  rm -rf "$TMP"
  if [ "$status" -ne 0 ]; then
    echo
    echo "FAIL: ${FAILURE:-a command exited with status $status}"
  fi
  exit "$status"
}

purge() {
  local keys body
  keys="$(printf '%s\n' $ORDER_IDS | jq -R 'sub("^"; "charge:")' | jq -sc .)" || return 0
  body="$(jq -cn --arg ns "$NS" --argjson keys "$keys" \
    '{operations: [$keys[] | {op: "delete", ns: $ns, key: .}]}')" || return 0
  request POST /api/v1/kv "$body" || true
  request DELETE "/api/v1/resources/queues/$ORDERS" || true
}
trap cleanup EXIT

fail() { FAILURE="$*"; exit 1; }

# check <actual> <expected> <description>
check() {
  [ "$1" = "$2" ] || fail "$3 (expected [$2], got [$1])"
  CHECKS=$((CHECKS + 1))
  echo "  ok: $3"
}

# ok <description>: records a check whose condition was already tested, for the
# assertions that are not an equality.
ok() {
  CHECKS=$((CHECKS + 1))
  echo "  ok: $1"
}

# A millisecond clock. GNU date spells it %3N; BSD date (macOS) has no %N and
# leaves the unconverted tail in the output, so a probe for anything that is not
# a digit tells the two apart, and perl, whose Time::HiRes is core, is the
# fallback.
if [ -z "$(date +%s%3N 2>/dev/null | tr -d '0-9')" ]; then
  now_ms() { date +%s%3N; }
else
  command -v perl >/dev/null 2>&1 \
    || { echo "FAIL: need GNU date or perl for a millisecond clock"; exit 1; }
  now_ms() { perl -MTime::HiRes -e 'printf "%d", Time::HiRes::time() * 1000'; }
fi

# Sets $STATUS to the HTTP status code and writes the response body to $OUT.
# There is no --fail: Queen reports outcomes in the body and several of the
# interesting ones arrive as 200, so read the status, then the body.
OUT="$TMP/body"
request() {
  local method="$1" path="$2" body="${3:-}"
  if [ -n "$body" ]; then
    STATUS="$(curl -sS -o "$OUT" -w '%{http_code}' \
      -X "$method" "$QUEEN_URL$path" \
      -H 'content-type: application/json' -d "$body")"
  else
    STATUS="$(curl -sS -o "$OUT" -w '%{http_code}' -X "$method" "$QUEEN_URL$path")"
  fi
}

# ---------------------------------------------------------------------------
# The KV surface, in the two shapes this program needs.
#
# Everything goes through POST /api/v1/kv, including the single-key read. The
# path routes (GET|PUT|DELETE /api/v1/kv/:ns/*key) exist and are correct, but the
# batch route keeps the key out of the access log, the proxy's samples and any
# tracing span, which is the same reason a prefix may not travel in a query
# string. A marker names a customer's order; it is not URL material.
# ---------------------------------------------------------------------------

# kv_get <key>: prints the whole row as JSON, `{"found":false,...}` when absent.
#
# `found` is a field of its own because null is a legal stored value: absence is
# never inferred from the value being empty.
kv_get() {
  local body
  body="$(jq -cn --arg ns "$NS" --arg key "$1" \
    '{operations: [{op: "get", ns: $ns, key: $key}]}')"
  request POST /api/v1/kv "$body"
  [ "$STATUS" = 200 ] || fail "kv get returned HTTP $STATUS"
  jq -c '.results[0]' "$OUT"
}

# marker_key <order>: the name of the row that says this order has been charged.
marker_key() { printf 'charge:%s' "$1"; }

echo "broker $QUEEN_URL"

# Every broker serves /api/v1/kv: there is no flag that turns it on. What can
# still refuse is an operator's runtime kill switch (503) or a quota (403), so
# probe once here and name that, rather than letting the first real call fail
# with something that reads like a bug.
request POST /api/v1/kv '{"operations":[{"op":"get","ns":"probe","key":"probe"}]}'
[ "$STATUS" = 200 ] \
  || fail "the kv probe returned HTTP $STATUS: $(cat "$OUT") (503 is an operator's kill switch, 403 a quota; see /deploy/state)"

# ---------------------------------------------------------------------------
# Leases are what make a crashed worker safe: a message whose handler dies is
# redelivered once the lease expires. retryLimit bounds how many times that can
# happen before the message is dead-lettered instead.
#
# /configure is a full replace rather than a patch, so what is not named here is
# reset to its default.
# ---------------------------------------------------------------------------
configure_body="$(jq -n --arg queue "$ORDERS" \
  '{queue: $queue, options: {leaseTime: 30, retryLimit: 5}}')"
request POST /api/v1/configure "$configure_body"
[ "$STATUS" = 200 ] || fail "configure returned HTTP $STATUS"
check "$(jq -r .configured "$OUT")" true 'the queue was created with a 30 second lease'

# ---------------------------------------------------------------------- queuing
echo
echo "queuing orders"
cents=1000
for order in $ORDER_IDS; do
  body="$(jq -n --arg queue "$ORDERS" --arg order "$order" --argjson cents "$cents" \
    '{items: [{queue: $queue, transactionId: ("order-" + $order),
               payload: {orderId: $order, cents: $cents}}]}')"
  request POST /api/v1/push "$body"
  [ "$STATUS" = 201 ] || fail "push of $order returned HTTP $STATUS"
  # HTTP 201 is not proof the message was stored: "buffered" and "failed" also
  # come back 201. The per-item status is the only answer.
  [ "$(jq -r '.[0].status' "$OUT")" = queued ] \
    || fail "push of $order came back $(jq -r '.[0].status' "$OUT")"
  cents=$((cents + 1))
done
echo "  $ORDER_COUNT orders queued"

# ---------------------------------------------------------------------------
# handle <group>: one delivery, from the pop response in $TMP/pop.
#
# Four steps in a fixed order, and the order is the design.
# ---------------------------------------------------------------------------
handle() {
  local group="$1" order cents txn partition lease marker charge_id body

  order="$(jq -r '.messages[0].data.orderId' "$TMP/pop")"
  cents="$(jq -r '.messages[0].data.cents' "$TMP/pop")"
  txn="$(jq -r '.messages[0].transactionId' "$TMP/pop")"
  partition="$(jq -r '.messages[0].partitionId' "$TMP/pop")"
  # The lease minted for THIS pop. It is what says the worker still owns the
  # message, and it is the reason the acknowledgement below can refuse.
  lease="$(jq -r '.leaseId' "$TMP/pop")"

  printf '%s\n' "$order" >> "$ATTEMPTS"

  # 1. Has this order already been charged?
  marker="$(kv_get "$(marker_key "$order")")"
  if [ "$(printf '%s' "$marker" | jq -r '.found')" = true ]; then
    # Nothing to do, but the message still has to be taken off this group's
    # cursor, or it comes back forever.
    ack_body="$(jq -cn --arg txn "$txn" --arg pid "$partition" --arg grp "$group" --arg lease "$lease" \
      '{transactionId: $txn, partitionId: $pid, consumerGroup: $grp, leaseId: $lease, status: "completed"}')"
    request POST /api/v1/ack "$ack_body"
    [ "$STATUS" = 200 ] || fail "ack returned HTTP $STATUS"
    printf '%s %s false\n' "$group" "$order" >> "$OBSERVED"
    echo "  $order: already charged, skipped"
    return 0
  fi

  # 2. The scripted failure. It happens BEFORE the charge and before the commit,
  #    which is the ordering a real handler should aim for: whatever can fail
  #    without an external effect should fail there.
  if [ "$order" = "$CRASHING_ORDER" ] \
     && [ "$(grep -c "^$CRASHING_ORDER$" "$ATTEMPTS")" -eq 1 ]; then
    # The negative acknowledgement is explicit. It clamps the cursor below this
    # message and charges one unit of the retry budget, which is what brings the
    # order back.
    ack_body="$(jq -cn --arg txn "$txn" --arg pid "$partition" --arg grp "$group" --arg lease "$lease" \
      '{transactionId: $txn, partitionId: $pid, consumerGroup: $grp, leaseId: $lease,
        status: "failed", error: "card network timed out"}')"
    request POST /api/v1/ack "$ack_body"
    [ "$STATUS" = 200 ] || fail "the failing ack returned HTTP $STATUS"
    echo "  $order: card network timed out (will be redelivered)"

    # The claim this example exists to prove, checked at the only moment it can
    # be checked: right after a handler failed before its commit.
    marker="$(kv_get "$(marker_key "$order")")"
    check "$(printf '%s' "$marker" | jq -r '.found')" false \
      "$order failed before committing and left no marker behind"
    return 0
  fi

  # 3. The external effect.
  charge_id="ch_${order}_$(wc -l < "$LEDGER" | tr -d ' ')"
  printf '%s %s %s\n' "$order" "$charge_id" "$cents" >> "$LEDGER"

  # 4. The marker and the acknowledgement, in ONE transaction.
  #
  #    `kv` is a key of the ROOT of this body, beside `operations` and not inside
  #    it. That is not a style choice: the two arrays are separate top-level
  #    fields precisely so that no client can send them under one key by
  #    accident.
  #
  #    `required: true` is what makes putIfAbsent a GATE rather than a verdict.
  #    Without it a lost race would come back applied:false and the ack would
  #    still commit; with it, a lost race rolls the whole bundle back, ack
  #    included, so a concurrent worker that got there first is the only one
  #    whose ack lands.
  #
  #    ttlSeconds is mandatory on every KV write. A marker with no expiry is a
  #    row nothing will ever delete.
  #
  #    The lease travels with the ack. If this worker's lease had expired while
  #    it was charging -- the exact failure in the war story -- the ack is
  #    refused and the marker write is refused with it. That is the guarantee a
  #    compare-and-swap cannot give: an `expect` on a version that still matches
  #    succeeds even from a worker that no longer owns the message.
  body="$(jq -cn --arg ns "$NS" --arg key "$(marker_key "$order")" \
    --arg charge "$charge_id" --argjson cents "$cents" \
    --arg txn "$txn" --arg pid "$partition" --arg grp "$group" --arg lease "$lease" '
    {operations: [{type: "ack", transactionId: $txn, partitionId: $pid,
                   consumerGroup: $grp, leaseId: $lease, status: "completed"}],
     kv: [{op: "putIfAbsent", ns: $ns, key: $key,
           value: {chargeId: $charge, cents: $cents},
           ttlSeconds: 3600, required: true}]}')"
  request POST /api/v1/transaction "$body"

  # A lost gate is an HTTP 200 with success:false and reason "kv_precondition".
  # It is the most frequent legitimate outcome of this shape, so it does not
  # belong in an error path, a retry policy or an error metric -- which is
  # exactly why it is not a 409.
  [ "$STATUS" = 200 ] || fail "the commit for $order returned HTTP $STATUS: $(cat "$OUT")"
  if [ "$(jq -r '.success' "$OUT")" != true ]; then
    [ "$(jq -r '.reason' "$OUT")" = kv_precondition ] \
      || fail "the commit for $order failed: $(jq -r '.error' "$OUT")"
    printf '%s %s false\n' "$group" "$order" >> "$OBSERVED"
    echo "  $order: lost the gate to a concurrent worker"
    return 0
  fi

  printf '%s %s true\n' "$group" "$order" >> "$OBSERVED"
  echo "  $order: charged"
}

# ---------------------------------------------------------------------------
# drain <group> <decisions>: pop and handle until this group has made that many
# decisions, or the phase deadline passes. The count is the bound and the
# deadline is the net; neither is a wait for silence.
# ---------------------------------------------------------------------------
drain() {
  local group="$1" wanted="$2" deadline
  deadline=$(( $(now_ms) + PHASE_MS ))

  while [ "$(grep -c "^$group " "$OBSERVED" || true)" -lt "$wanted" ]; do
    [ "$(now_ms)" -lt "$deadline" ] || break

    # subscriptionMode=all is what makes a group created now read what was
    # pushed before it existed: a new cursor is seeded at the TAIL unless you say
    # otherwise. batch=1 keeps one message in flight, so the failure below is a
    # single message's failure and not a batch's.
    request GET "/api/v1/pop/queue/$ORDERS?consumerGroup=$group&subscriptionMode=all&batch=1&wait=true&timeout=$POLL_MS"
    # 204 is an empty pop, with no body at all. Go round again until the
    # deadline.
    [ "$STATUS" != 204 ] || continue
    [ "$STATUS" = 200 ] || fail "pop returned HTTP $STATUS"
    cp "$OUT" "$TMP/pop"
    handle "$group"
  done
}

# --------------------------------------------------------------------- charging
echo
echo "charging"
drain "$CHARGER" "$ORDER_COUNT"

check "$(grep -c "^$CHARGER " "$OBSERVED" || true)" "$ORDER_COUNT" \
  'the charger reached a decision on every order'

# ---------------------------------------------------------------------- replay
#
# A second consumer group with subscriptionMode=all reads the same orders from
# the beginning. This is a redelivery with the cause removed: the messages are
# identical, the handler is identical, and the only thing standing between them
# and a second charge is the marker.
echo
echo "replaying"
drain "$REPLAY" "$ORDER_COUNT"

# --------------------------------------------------------------------- checking
echo
echo "checking"

check "$(wc -l < "$LEDGER" | tr -d ' ')" "$ORDER_COUNT" \
  "$ORDER_COUNT orders produced exactly $ORDER_COUNT charges"
check "$(awk '{print $1}' "$LEDGER" | sort -u | wc -l | tr -d ' ')" "$ORDER_COUNT" \
  'every order was charged exactly once, none twice and none not at all'

[ "$(grep -c "^$CRASHING_ORDER$" "$ATTEMPTS")" -ge 2 ] \
  || fail "$CRASHING_ORDER was never redelivered after it failed"
ok "$CRASHING_ORDER was delivered again after it failed"

check "$(grep -c "^$REPLAY " "$OBSERVED" || true)" "$ORDER_COUNT" \
  "the replay group received all $ORDER_COUNT orders again"
check "$(awk -v g="$REPLAY" '$1 == g && $3 == "false"' "$OBSERVED" | wc -l | tr -d ' ')" \
  "$ORDER_COUNT" 'every order on the second pass reported that it did not run'
check "$(wc -l < "$LEDGER" | tr -d ' ')" "$ORDER_COUNT" \
  "the replay charged nothing: the ledger is still $ORDER_COUNT rows"

# The markers are readable state, not an internal detail: each one carries the id
# of the charge it stands for, so a support engineer can answer "was this order
# billed, and under which charge" without a second system. getMany reports
# `missing` explicitly, because absence is a datum and not a hole computed by
# difference.
keys="$(printf '%s\n' $ORDER_IDS | jq -R 'sub("^"; "charge:")' | jq -sc .)"
many_body="$(jq -cn --arg ns "$NS" --argjson keys "$keys" \
  '{operations: [{op: "getMany", ns: $ns, keys: $keys}]}')"
request POST /api/v1/kv "$many_body"
[ "$STATUS" = 200 ] || fail "kv getMany returned HTTP $STATUS"
check "$(jq -r '.results[0].rows | length' "$OUT")" "$ORDER_COUNT" \
  "all $ORDER_COUNT markers exist"
check "$(jq -r '.results[0].missing | length' "$OUT")" 0 \
  'no order is missing its marker'
check "$(jq -r '[.results[0].rows[].value.chargeId] | sort | join(",")' "$OUT")" \
  "$(awk '{print $2}' "$LEDGER" | sort | paste -sd, -)" \
  'each marker names the charge that was actually made'

echo
echo "  ledger: $(awk '{printf "%s=%s ", $1, $2}' "$LEDGER")"

echo
echo "PASS: $CHECKS checks"
```

## Four steps, and the order is the design

Read the marker first. If it is there, this delivery has nothing to do, and the only remaining job
is to take the message off this group's cursor so it stops coming back. The read reports `found` as
a field of its own rather than returning an empty value, because `null` is a legal stored value and
absence has to be distinguishable from it.

Then do everything that can fail without touching the outside world. The scripted failure in the
program sits exactly here, and that placement is the advice: a handler that validates, resolves and
prepares before it charges is a handler whose failures cost nothing.

Then the external effect, once.

Then the marker and the acknowledgement, together, in one call to
[`/api/v1/transaction`](/reference/http/transaction). `kv` is a top-level field of that request,
beside `operations` and never inside it, and the two arrays commit or roll back as one thing. The
write is a `putIfAbsent` carrying `required: true`, which is what turns it from a verdict into a
gate: without it a lost race would come back `applied: false` and the acknowledgement would still
commit, which is precisely the divergence this page exists to remove.

Two properties of that call are worth stating on their own, because they are the ones people are
surprised by.

A lost gate is **returned, not thrown**. It arrives as HTTP 200 with `success: false` and
`reason: "kv_precondition"`, and the clients hand it back rather than raising. It is the expected
outcome of every legitimate redelivery, which makes it one of the most frequent answers this
product gives, and an outcome that frequent does not belong inside a catch block where the natural
reflex is to retry.

And the expiry is **mandatory**. Every KV write declares exactly one of `ttlSeconds` and
`forever: true`, with zero being the same error as two, because a marker with no expiry is a row
nothing will ever delete. Pick a TTL comfortably longer than the window in which a redelivery is
still possible, which means longer than the lease times the retry budget, and long enough to answer
a support question afterwards. The programs here use an hour. They never use `forever`: an example
that runs in CI and writes an immortal row is one bad assertion away from leaving state in a shared
database that nothing removes.

## Where the guarantee stops

The atomic pair is the marker and the acknowledgement. It is not the marker and the card.

If the process dies after the charge and before the commit, the money moved and no marker exists,
so the redelivery charges again. Nothing in a broker can close that window, because the effect is
outside the database that would have to roll it back. What the pattern does is make the window as
small as one commit, and make every failure on either side of it consistent: before the commit
nothing happened, after the commit both things happened.

That leaves one job for you, and it is one line: give the payment provider the same idempotency key
you used for the marker. The order id is already the natural one. Then the residual window closes
against the provider's own ledger rather than against a hope.

The exception, and it is the only one, is the case where the effect is itself a row in this
PostgreSQL: a push onto another queue, an acknowledgement, another KV write. Those go inside the
same bundle and inherit its atomicity exactly, and for them the phrase "exactly once" is literal
rather than aspirational. The related caveat about a lost **response** rather than a lost commit,
and the deterministic `transactionId` that answers it, is on
[the transaction reference](/reference/http/transaction#it-is-not-exactly-once-end-to-end).

## What each failure does

| what happens | what the program does about it |
|---|---|
| the handler fails before the commit | nothing was written, so no marker exists and the retry is not blocked. The program asserts this at the only moment it can be observed, right after the failing acknowledgement |
| the handler fails after the commit | the marker is there, so the redelivery finds it, does nothing, and acknowledges. This is the second pass, and it is what `ran === false` reports |
| the lease expired while the handler worked | the acknowledgement is refused and the KV write is refused with it, so the state stays consistent at "not done" and the order is charged by whoever holds the lease now. A compare-and-swap cannot give this: an `expect` on a version that still matches succeeds even from a worker that no longer owns the message |
| two workers race for the same order | one wins the `putIfAbsent`, the other gets `success: false` with `reason: "kv_precondition"` and its whole bundle rolls back, acknowledgement included, so only the winner's acknowledgement lands |
| the response is lost after a successful commit | the redelivery reads the marker and skips, which is the same end state |
| the marker expires while a redelivery is still possible | the order is charged twice, and it is the TTL that was wrong. This is the failure the mandatory expiry makes you think about instead of discovering |

## Run it

The key/value surface is served by every broker, so there is nothing to turn on and no flag to pass.
Each program probes once at the start anyway, because a `503` from [an operator's kill
switch](/deploy/state) or a `403` from a quota is worth naming before the first real call fails with
something that reads like a bug.

Against a broker from [the quickstart](/start/quickstart):

```bash
QUEEN_URL=http://localhost:6632 examples/apps/run.sh
```

Every program on this page asserts the property it exists to demonstrate and exits
non-zero if it does not hold. The runner takes a language name to run one of them on its
own, for example `examples/apps/run.sh js`. Each phase ends on a **count** with a deadline behind
it rather than on a quiet interval, because a program that stops when nothing has arrived for a
while passes on a broker that delivered nothing at all. And each run purges what it wrote, markers
included: they are rows in their own table, so deleting the queue does not take them along.

Source: https://queenmq.com/use/full-examples/exactly-once/index.mdx