Skip to content

Exactly-once transaction benchmark vs Kafka

Exactly-once consume-transform-produce on QueenMQ, Kafka, Redpanda and Pulsar on the same machines: commit and end-to-end latency, each system's ceiling, what partition count does to it, and what moved QueenMQ's limit.

Updated View as Markdown

A transaction is the step QueenMQ is built around: take a message, do the work, then ack it and write what comes next, all of it or none of it. So we built a transactional benchmark and ran it on all four systems. A pipeline reads an input, transforms each batch and commits the ack and the output together, and at the end a verifier reads every output and checks that every input appears in it exactly once. It passed on every point that ran to the end, on all four systems; one Pulsar point needed a second run because our verifier stopped reading too early. At 9,000 messages a second in transactions of 10, a QueenMQ commit took 3 ms at p50 and 4 ms at p99, Kafka’s 12 and 34 ms and Pulsar’s 22 and 32 ms (2.0.0-beta.2, 2026-10-01). 2.3.0 was measured against 2.2.0, not against the other systems again: its binary transaction body takes about a third off the commit time at 12,000 transactions a second (below). Compare shows what one commit covers in each system.

The pipeline

A feeder pushes 256-byte messages into an input queue (or topic) of 200 partitions. Workers, one per partition at a time, take a batch of up to 10 messages, transform it, and commit one transaction that acknowledges the inputs and writes the outputs to the same partition of an output queue. Readers consume the output, and only what committed. When the run ends, the verifier reads the whole output and what is left of the input and checks every input id: in the output exactly once, or still waiting in the input.

Each system commits its own way. A QueenMQ 2.0.0-beta.2 worker pops with a lease, then sends one POST /api/v1/transaction that acks every input with its lease and pushes the outputs; that is one log entry, and the lease fences it, so a worker whose lease ran out commits nothing. Kafka 4.3.1 and Redpanda 26.2.3 run franz-go’s GroupTransactSession: one transactional producer per worker with the consumer offsets inside the transaction, transactions v2, read_committed readers and classic consumer groups. Pulsar 4.2.4 runs with its transaction coordinator on (16 coordinators) and batched writes of the transaction log and the pending acks.

All four ran on 2026-10-01 on the rig of the other pages, with the durability of each system unchanged: QueenMQ, Redpanda and Pulsar acknowledge after two of three copies are fsynced, Kafka after three replicas have the data in page cache. Runs: benchmark-queen/2026-09-30-kafka-pulsar/runs/txn/.

Latency at 9,000 messages a second

Ten messages per transaction, 200 partitions in and out, 198 workers. Commit is from the transaction’s start to its acknowledged commit; e2e is from the input’s scheduled send to the consumption of its committed output, so it holds two hops and a commit.

QueenMQ Kafka Redpanda Pulsar
Commit p50 / p99 3 / 4 ms 12 / 34 ms 15 / 151 ms 22 / 32 ms
e2e p50 / p99 12 / 19 ms 21 / 51 ms 241 / 338 ms 29 / 50 ms
Cores, three brokers 4.3 7.2 5.7 4.4
p99 latencies of the transaction pipeline at 9,000 messages a second on three nodes. Commit p99: QueenMQ 4 ms, Kafka 34 ms, Redpanda 151 ms, Pulsar 32 ms. End-to-end p99: QueenMQ 19 ms, Kafka 51 ms, Redpanda 338 ms, Pulsar 50 ms.QueenMQKafkaRedpandaPulsar0 ms100 ms200 ms300 msmillisecondscommit p99end-to-end p994 ms34 ms151 ms32 ms19 ms51 ms338 ms50 ms
Ten messages per transaction. A QueenMQ transaction is one log entry, so it costs about what one write costs; Kafka, Redpanda and Pulsar each go through a transaction coordinator. Source: benchmark-queen/2026-09-30-kafka-pulsar

A QueenMQ transaction is one entry in the log, so it costs about what one write costs. A Kafka transaction ends with a call to its transaction coordinator, which writes a commit marker into every partition the transaction touched, and a Pulsar transaction goes through a coordinator and a transaction log of its own.

The ramp

Output delivered and e2e p99, ten messages per transaction, 200 partitions. “Behind” means output under 97% of the offered rate, or an e2e p99 over 5 seconds.

Offered msg/s QueenMQ Kafka Redpanda Pulsar
9,000 9.0k, 19 ms 9.0k, 51 ms 9.0k, 338 ms 9.0k, 50 ms
18,000 18.0k, 34 ms behind: 17.5k, p99 26 s behind: 17.8k, p99 22 s 18.0k, 55 ms
36,000 36.0k, 22 ms behind: 13.2k behind: 13.7k 36.0k, 64 ms
72,000 72.0k, 33 ms not run not run behind: 71.2k, p99 24 s
144,000 behind: 66.7k not run not run behind: 69.8k
Messages delivered per second against messages offered through the transaction pipeline, both on log scales. QueenMQ keeps up to 72,000 msg/s and delivers 66,700 of 144,000. Pulsar keeps up to 36,000, delivers 71,200 of 72,000 behind schedule and 69,800 of 144,000. Kafka and Redpanda keep up at 9,000 and fall behind from 18,000, delivering about 13,000 of 36,000.delivered msg/s10k20k50k100k10k20k50k100koffered msg/sQueenMQKafkaRedpandaPulsar
Where a line leaves the diagonal, the system fell behind. QueenMQ's ceiling is the one thread that plans transactions, near 7,000 transactions of ten messages a second here. Source: benchmark-queen/2026-09-30-kafka-pulsar

At 200 partitions QueenMQ and Pulsar top out close together, near 7,000 transactions of ten messages a second, and QueenMQ gets there with less than half of Pulsar’s e2e latency. With feeder batches of 100 and one partition per pop, QueenMQ delivered 8,900 transactions a second at 144,000 offered, still behind. Kafka and Redpanda fell behind at 18,000 msg/s, about 1,750 transactions a second, with this client and one transaction per partition at a time; another client or layout may do better.

With one message per transaction at 9,000 msg/s, QueenMQ committed all 9,000 transactions a second (commit p50 / p99 4 / 6 ms), while Kafka managed 1,644, Redpanda 1,313 and Pulsar 7,511, each falling behind.

2.3.0 against 2.2.0

The same pipeline ran on 2026-10-12 on the machines of the 2.3.0 pre-release runs, which sync faster than September’s: 200 partitions in and out, 198 workers, 40-second steps, release 2.2.0 next to a pre-release build of 2.3.0 with its transactions sent as JSON and as the binary body the release adds (the pushes of a transaction go as frames the client built and compressed; a client sends it when binary push and push compression are both on). No step had an error or an abort, and a verifier found every message once on both bodies. Commit and e2e are p50 / p99 in milliseconds over all load processes. Runs: txn-* in benchmark-queen/2026-10-12-three-node-new-forms/runs/.

Transactions of 10 messages 2.2.0 2.3.0, JSON body 2.3.0, binary body
6,000 a second: commit 2.8 / 5.0 2.7 / 4.5 2.6 / 4.2
12,000 a second: commit 5.3 / 12.4 4.9 / 11.1 3.6 / 8.6
12,000 a second: e2e 78 / 195 72 / 176 62 / 143
Committed a second when 18,000 are offered 11,320 11,770 12,890
Transactions of 100 messages 2.2.0 2.3.0, JSON body 2.3.0, binary body
3,000 a second: commit 6.1 / 19.2 5.6 / 9.7 4.1 / 7.0
3,000 a second: e2e 25 / 60 21 / 30 19 / 26
3,000 a second: cores, three brokers 13.2 11.8 11.2
Committed a second when 6,000 are offered 2,020 2,290 2,450

The binary body takes about a third off the commit time at the highest rate all three carry, and moves the ceiling by 14% with 10 messages and 21% with 100. The ceilings themselves belong to the design and are the same in the three: the next section. Past its limit the cluster does less than at it (3,000 transactions of 100 a second are carried, 2,450 are reached when 6,000 are offered), and plain pushes then wait behind the transactions for 0.6 to 0.7 s.

These machines commit more than September’s did: 2.2.0 reached 11,320 transactions of 10 a second here, where 2.0.0-beta.2 topped out between 7,000 and 9,000 on the other rig. The machines differ, so do not read that difference as the code’s.

The ceiling, and what moves it

Up to 2.8.1, QueenMQ planned pushes on its lanes in parallel (eight by default, sixteen in these runs), but a transaction that acked, wrote KV or scheduled a timer was planned on one thread, the control step, because those parts are checked against state that every partition shares. That thread was QueenMQ’s transaction ceiling, and more workers only queued more transactions for it: with 900 partitions and 900 workers QueenMQ committed 5,100 transactions a second, at a commit p50 of 155 ms (2.0.0-beta.2). Pulsar spreads its transactions over the broker’s threads, and the same change doubled its rate to 14,400 a second (143,700 msg/s, e2e p99 799 ms). With 1,800 partitions and 1,800 workers Pulsar reached 17,600 a second while falling behind.

Bigger transactions raise QueenMQ’s ceiling in messages, because much of that thread’s work is paid once per transaction:

Offered msg/s, 100 per transaction QueenMQ Pulsar
144,000 144k; commit 5 / 9 ms, e2e 19 / 29 ms 144k; commit 25 / 35 ms, e2e 32 / 56 ms
288,000 behind: 250.5k, commit 61 / 80 ms 288k; commit 26 / 45 ms, e2e 33 / 98 ms
576,000 not run behind: 572.4k, e2e p99 22 s
1,152,000 not run behind: 548.8k

Below about 250,000 msg/s QueenMQ commits faster; above it, Pulsar carries more (2.0.0-beta.2).

The footprint scheduler against 2.8.1

The next release replaces the lanes and the control step with one scheduler: commands that share no partition, queue setting, KV key or timer plan at the same time on eight threads, a transaction that acks and writes KV included, and the log entry is the one a single thread would have planned (the write path). We measured it against 2.8.1 on 2026-11-12: both built from source the same way, three brokers of 22 vCPU (c3-standard-22-lssd, four local NVMe disks), one loader of 24 vCPU, the pipeline above with 200 workers (440 at 900 partitions), a fresh cluster for each ladder, 40-second steps. The highest rate each build carried:

Transactions 2.8.1 Footprint scheduler
10 messages, 200 partitions 160,000 msg/s (16,000 txn/s) 240,000 msg/s (24,000 txn/s)
10 messages and one KV write, 200 partitions 120,000 msg/s (12,000 txn/s) 200,000 msg/s (20,000 txn/s)
10 messages, 900 partitions 160,000 msg/s (16,000 txn/s) 280,000 msg/s (28,000 txn/s)
100 messages, 200 partitions 450,000 msg/s (4,500 txn/s) 750,000 msg/s (7,500 txn/s), the top of the ladder

Below 2.8.1’s ceiling both builds commit in the same time: 10 messages at 120,000 msg/s, p50 1.5 ms and p99 2.8 ms on both. Past it, 2.8.1 falls behind and, at 900 partitions or 100 messages a transaction, commits less than at the rate below (10,467 and 2,842 txn/s). At its own ceiling the scheduler’s leader used 12 to 13 of its 22 cores, and its busiest thread was apply’s.

On brokers of 8 vCPU the two builds stop at about the same rate, 13,000 transactions of 10 messages a second: the leader runs out of CPU before its planning thread is the limit. There the scheduler carries 3% more with acks and pushes, 11% more with a KV write, and 51% more at 900 partitions under overload. Runs: benchmark-queen/2026-11-12-ab-scheduler/ in the repository.

Bigger transactions still raise the ceiling in messages, because much of the work is paid once per transaction. Tenants spread over several raft groups each get a planner of their own, which we have not measured.

Partition count

e2e p50 / p99 in milliseconds at 9,000 msg/s, ten messages per transaction, with that many partitions in the input and as many in the output.

Partitions QueenMQ Kafka Redpanda Pulsar
200 12 / 19 21 / 51 241 / 338 29 / 50
10,000 12 / 32 behind: 1.8k delivered 375 / 758 30 / 42
100,000 12 / 43 did not start did not start did not start

At 100,000 partitions Kafka created the two topics in 15 minutes, then a load process exited before it was ready; Redpanda refused to create the output partitions; Pulsar’s producers timed out during setup for an hour. Pulsar’s first run at 10,000 failed its check because our verifier gave up after 10 seconds without a new record, before it had read every partition; with 120 seconds it passed. Both runs are in the archive.

What this page does not show

Kafka, Redpanda and Pulsar against 2.3.0 or the footprint scheduler: the comparison tables are September’s, with QueenMQ at 2.0.0-beta.2; the scheduler was measured against 2.8.1 only. Kafka and Redpanda were not run with more workers, or with 100 messages per transaction above 9,000 msg/s. No run used transactions through QueenMQ’s Kafka facade, and the QueenMQ transactions carried acks and pushes only, with no KV writes or timers. Each point offered load for 70 seconds, a 10-second ramp included.

Navigation

Type to search…

↑↓ navigate↵ selectEsc close