Skip to content

Ephemeral queues for fast request and reply

Ephemeral queues live in memory, outside the log: a requester sends a replyTo inbox and waits on it with a timeout. Fast, and lost on a crash by design.

Updated View as Markdown

Ephemeral queues live in memory, outside the log, for request/reply. The worker answers on whatever inbox a request names; the caller makes a private inbox and waits on it. There is no transaction or replay here, and a crash loses what is in flight.

The worker:

const { messages } = await queen.ephemeral.pop('price-requests', { group: 'pricers', wait: true })
for (const m of messages) {
  await queen.ephemeral.push(m.payload.replyTo, [{ sku: m.payload.sku, price: 59.9 }])
}
await queen.ephemeral.ack('price-requests', messages, { group: 'pricers' })

The caller:

const inbox = `inbox-${crypto.randomUUID()}`           // a private reply queue
await queen.ephemeral.push('price-requests', [{ sku: 'A-1', replyTo: inbox }])
const reply = await queen.ephemeral.pop(inbox, { wait: true, timeout: 5000, commitOnDelivery: true })
console.log(reply.messages[0]?.payload)                // { sku: 'A-1', price: 59.9 }
Why
a request that waits for an answer (“what is the price of A-1, now?”) needs speed more than history. Writing it to disk, replicating it and keeping it for days would cost time for nothing. Ephemeral queues keep the same groups, leases and acks, and skip the log. Use a regular queue for anything that must survive a crash.
The caller mints a private inbox queue, inbox-<uuid>, and pushes a request to price-requests with replyTo set to it. A worker of the group pricers pops the request, pushes the answer to the inbox and acks. The caller's pop on its inbox, waiting with a timeout, returns the answer. Everything stays in memory, outside the log.The caller mints a private inbox queue, inbox-<uuid>, and pushes a request to price-requests with replyTo set to it. A worker of the group pricers pops the request, pushes the answer to the inbox and acks. The caller's pop on its inbox, waiting with a timeout, returns the answer. Everything stays in memory, outside the log.
Request and reply over ephemeral queues, with a private inbox.

See also

Navigation

Type to search…

↑↓ navigate↵ selectEsc close