Skip to content

Laravel queue benchmark: QueenMQ vs Horizon on Redis

Horizon on Redis against QueenMQ's Laravel driver and supervisors on the 2.0 Raft broker: jobs per second, latency, memory and CPU, 15 failures, a 45-minute soak and 32 Laravel features, each with its run and conditions.

Updated View as Markdown

The same Laravel application ran on Horizon with Redis and on QueenMQ’s queue driver and supervisor with the 2.0 Raft broker. These runs measure what an application team lives with: how many jobs a pool of workers completes, how soon a job runs after its dispatch, how much memory the workers take, and what happens when a worker, a master or the backend dies in the middle of a job. We wrote the protocol ourselves, so read every result as diagnostic: one host per run, one broker node, and the conditions next to each number. The guides that use these numbers start at Laravel.

The runs

Every run used one DigitalOcean droplet that ran nothing else (16 vCPUs, Intel Xeon Gold 6548N, 31 GiB, Ubuntu 24.04, Docker 29), whose virtual disk set the fsync cost, and one node of the Raft broker, the only storage 2.0 has. Each archive under benchmark-queen/ holds the scripts, the raw data and a README.

Date Broker What it measured Archive
2026-10-01 2.0.0-beta.2 throughput, latency, memory and CPU, 16 to 64 workers, 119 runs 2026-10-01-linux-vm-horizon-raft
2026-10-02 2.0.0-beta.3 15 failures, 2 replica scenarios, a 45-minute soak, 32 Laravel features, the crash hand-back 2026-10-02-laravel-failure-matrix
2026-10-02 2.0.0-beta.3 64, 128 and 256 partition stripes, 36 runs 2026-10-02-laravel-partition-stripes
2026-10-05 2.0.0-beta.3 worker memory: prefork and the opcache apart, jobs that allocate, 20 minutes of work, 38 runs 2026-10-05-laravel-worker-memory
2026-10-05 2.0.0-beta.3 the safe, balanced and fast profiles, prefetch 'auto', 100 runs 2026-10-05-laravel-auto-prefetch
2026-10-06 2.0.0-beta.3 64 forked workers for 23 hours, 4.1 million jobs, one run 2026-10-06-laravel-worker-memory-24h

Every run used the Laravel 12 application in benchmark-queen/laravel-supervisors and its job App\Jobs\BenchmarkJob, which sleeps 10 ms unless a lane says otherwise, on PHP 8.3. Horizon ran on Redis 7.4 with its append-only file. Whenever both stacks ran, Redis used appendfsync always unless a lane says otherwise, so that Redis and the Raft log both fsync every write they acknowledge. A Raft node fsyncs every acknowledged write in a group commit, in every lane.

Worker capacity on a Linux server

2026-10-01. The application had 8 CPUs and 8 GiB, the broker and Redis 4 CPUs and 4 GiB each. QueenMQ ran the Rust supervisor with prefork workers, the command-line opcache, lease renewal in the supervisor, ack_async, pop_ahead, prefetch 4 and the cURL transport. Horizon ran its own supervisor with block_for 1 s. Each lane ran 3 times per engine (the 15-minute lane once), engines alternating, each run on a fresh backend, and all 119 runs completed the exact job set with no duplicate and no failure.

When the jobs are dispatched in one burst, Horizon’s rate is its producer’s: for every job it dispatches, Horizon writes its monitoring records to Redis too, and one producer sends about 860 jobs/s with appendfsync always. So the capacity lanes hold every worker until the whole batch is enqueued (run.sh --backlog-first 1); in none of their 24 runs did a job start before the release.

Lane, medians of 3 Horizon QueenMQ QueenMQ / Horizon
32 workers, 10 ms jobs, fsync on every write 1,124 jobs/s 2,753 jobs/s 2.4
32 workers, Redis everysec 1,247 jobs/s 2,752 jobs/s 2.2
32 workers, Redis appendfsync no 1,313 jobs/s 2,714 jobs/s 2.1
32 workers, empty jobs 1,090 jobs/s 5,995 jobs/s 5.5
64 workers, 10 ms jobs 1,090 jobs/s 4,395 jobs/s 4.0
Laravel jobs per second on one 16-vCPU server, QueenMQ's driver and supervisor on one QueenMQ node against Horizon on Redis, medians of three runs. 32 workers with 10 ms jobs and Redis fsyncing every write: QueenMQ 2,753, Horizon 1,124. With Redis everysec: 2,752 and 1,247. With Redis appendfsync no: 2,714 and 1,313. 32 workers with empty jobs: 5,995 and 1,090. 64 workers with 10 ms jobs: 4,395 and 1,090.QueenMQHorizon02,0004,000jobs per second32 workers, 10 ms jobsRedis everysecRedis appendfsync no32 workers, empty jobs64 workers, 10 ms jobs2,7531,1242,7521,2472,7141,3135,9951,0904,3951,090
QueenMQ fsyncs every write in every lane; only Redis's setting changes. Horizon stays near 1,100 to 1,300 jobs a second however it is tuned. Source: benchmark-queen/2026-10-01-linux-vm-horizon-raft

Horizon stays between 1,090 and 1,313 jobs/s whatever the worker count, the job length or the fsync mode of Redis, and relaxing that fsync moved it by less than a fifth. Each Horizon job cost about 33 Redis commands, against 1.3 HTTP requests to the QueenMQ broker, and doubling the pool from 32 to 64 workers took QueenMQ from 2,753 to 4,395 jobs/s.

With the producer included, as an application sees it:

Lane Horizon QueenMQ
50,000 jobs, 32 workers, all done after 58.5 s 18.0 s
100,000 empty jobs, 32 workers 889 jobs/s 5,989 jobs/s
30,000 jobs, autoscaling from 1 to 64 workers, all done after 37.0 s 9.5 s
500 jobs/s asked, one dispatch per job 440 jobs/s reached 500 jobs/s

In the autoscaling lane QueenMQ ran event_driven and fast_scale_up and reached 64 workers 8.2 s after the burst began; Horizon’s producer sent its jobs too slowly for it to need more than about 34 workers (33 to 37 in its three runs).

Latency at a steady rate

The paced lanes dispatch one job at a time at a fixed rate below capacity, with 16 workers, and measure from the dispatch to the end of the job (2026-10-01, Linux server).

Lane Horizon p50 / p95 QueenMQ p50 / p95
500 jobs/s asked, median of 3 75 / 227 ms 15 / 23 ms
400 jobs/s for 15 minutes, one run 71 / 249 ms 14 / 22 ms
Latency from dispatch to the end of a job with 16 workers on one 16-vCPU server. At 500 jobs/s asked, median of three runs: QueenMQ p50 15 ms and p95 23 ms, Horizon p50 75 ms and p95 227 ms. At 400 jobs/s for 15 minutes, one run: QueenMQ p50 14 ms and p95 22 ms, Horizon p50 71 ms and p95 249 ms.QueenMQHorizon0100200ms, dispatch to completion500 jobs/s, p50500 jobs/s, p95400 jobs/s for 15 min, p50400 jobs/s for 15 min, p95157523227147122249
Below capacity QueenMQ answers in about a fifth of Horizon's median time and a tenth of its p95. Horizon's producer reached 440 of the 500 jobs/s asked. Source: benchmark-queen/2026-10-01-linux-vm-horizon-raft

Horizon’s producer reached 440 jobs/s of the 500 asked, because each dispatch also writes its monitoring records.

Memory

Workers Horizon, application / workers QueenMQ, application / workers
16 525 / 469 MiB 83 / 58 MiB
32 988 / 922 MiB 120 / 81 MiB
64 1,892 / 1,825 MiB 183 / 122 MiB
Memory of the Laravel application processes by worker count. 16 workers: Horizon 525 MiB, QueenMQ 83 MiB. 32 workers: 988 and 120 MiB. 64 workers: 1,892 and 183 MiB.QueenMQHorizon05001,0001,500application memory, MiB16 workers32 workers64 workers835251209881831,892
A QueenMQ worker is forked from one booted Laravel and shares its pages; a Horizon worker boots its own. Prefork saves the booted framework, about 27 MiB per worker; the memory a job allocates is private on both, so a real application's ratio is smaller. Source: benchmark-queen/2026-10-01-linux-vm-horizon-raft

2026-10-01. Application is the median of the application container’s cgroup memory (memory.current), read every second from the dispatch to the end of the run: every process in the container, the page cache of the benchmark’s own result files and kernel memory such as page tables. Workers is the workers’ summed proportional set size (PSS), which divides each shared page among the processes that map it; Horizon’s horizon:supervisor and QueenMQ’s fork server count with the supervisor. The 16- and 32-worker rows come from the capacity lanes, the 64-worker row from the 64-worker drain. Horizon ran with the command-line opcache off, as PHP ships and as is smaller for it, and QueenMQ with prefork and the opcache on.

Memory of workers drawn to one scale, five of 64 shown. Each Horizon worker boots its own Laravel: 48.0 MiB resident, of which 28.2 MiB is private to it and 19.8 MiB shared with other processes; the 64 workers' proportional set size is 1,825 MiB. Each QueenMQ worker is forked from one Laravel booted in the fork server and shares its pages until it writes them: 32.7 MiB resident, of which 1.4 MiB is private and 31.3 MiB shared; the 64 workers' proportional set size is 121 MiB.Each worker boots LaravelHorizon, 64 workers48.0 MiB resident each28.2 MiBprivate to each19.8 MiBshared, held once64 workers: 1,825 MiBproportional set size, five of 64 drawnForked from one booted LaravelQueenMQ supervisor with prefork, 64 workers32.7 MiB resident each1.4 MiBprivate to each31.3 MiBshared, held once64 workers: 121 MiBproportional set size, five of 64 drawn
Values from benchmark-queen/2026-10-01-linux-vm-horizon-raft/raw/process-memory.csv.

Where the difference comes from

2026-10-05, on the same droplet with the same budgets and the broker image of the stripe runs. 64 fixed workers ran 60,000 jobs of 10 ms, dispatched in one burst, three runs per case, each on a fresh broker, and the sampler read every process’s smaps_rollup and the container’s cgroup for 20 s after the last job, with every worker alive and idle. The values are medians of those samples; the three runs of each case agree within 0.1 MiB per worker and 2 MiB per container.

Workers Opcache Private per worker, MiB PSS per worker, MiB Application, MiB
Horizon, spawned off 28.9 29.2 1,967
Horizon, spawned on 35.9 36.2 2,436
QueenMQ, spawned off 31.1 31.5 2,062
QueenMQ, spawned on 38.1 38.4 2,512
QueenMQ, forked off 6.2 6.7 496
QueenMQ, forked on 1.7 2.2 225

A spawned worker starts PHP and boots Laravel on its own; a forked one is forked from the fork server. Private memory is the pages no other process maps. A spawned worker holds about 29 MiB of it: Laravel’s compiled code and the booted application, in its heap. The opcache moves the compiled code into opcache memory that each such process creates for itself, 22.6 MiB in a Horizon worker, and costs 7 MiB more per worker. A QueenMQ worker started on its own held 2.2 MiB more than a Horizon worker, so the difference is prefork, not the queue driver.

A forked worker maps the fork server’s pages instead. With the opcache, its 32.7 MiB resident set was 12.9 MiB of the booted heap, shared copy-on-write, 11.6 MiB of the fork server’s opcache memory, which is shared outright, and 8.2 MiB of the PHP binary and its libraries; 1.7 MiB of it was its own. Without the opcache, the compiled code sits in the fork server’s heap (27.4 MiB instead of 12.8), and each worker had copied 6.2 MiB of it. The fork server held 24 MiB of PSS with the opcache and 14 MiB without, the Rust master 6.6 MiB; Horizon’s master and horizon:supervisor held 60 MiB. In the container, the forked case held 133 MiB of anonymous memory, 25 MiB of opcache memory, 36 MiB of page cache for the result files and 31 MiB of kernel memory, 14 MiB of it page tables; Horizon’s held 1,906 MiB of anonymous memory.

Jobs that allocate

The same lanes with a job that built 8,192 strings of 1 KiB and released them at its end, as a job builds models and collections. PHP keeps freed memory for the next job, so each worker grew by 7.9 to 8.3 MiB in every case and stayed there:

Private memory per worker, 64 workers, with the 10 ms job and with a job that holds 8 MiB. Horizon, spawned, opcache off: 28.9 and 37.1 MiB. Horizon, spawned, opcache on: 35.9 and 43.9. QueenMQ, spawned, opcache off: 31.1 and 39.4. QueenMQ, spawned, opcache on: 38.1 and 46.0. QueenMQ, forked, opcache off: 6.2 and 14.5. QueenMQ, forked, opcache on: 1.7 and 10.0.10 ms jobjob holding 8 MiB02040private memory per worker, MiBHorizon, spawned, opcache offHorizon, spawned, opcache onQueenMQ, spawned, opcache offQueenMQ, spawned, opcache onQueenMQ, forked, opcache offQueenMQ, forked, opcache on28.937.135.943.931.139.438.1466.214.51.710
A spawned worker holds its own booted Laravel; a forked one shares the fork server's. The job's 8 MiB lands on every worker alike, so forking still saves about 27 MiB per worker. Source: benchmark-queen/2026-10-05-laravel-worker-memory
Workers, opcache Application, 10 ms job Holding 8 MiB
Horizon, spawned, off 1,967 MiB 2,494 MiB
Horizon, spawned, on 2,436 MiB 2,949 MiB
QueenMQ, spawned, off 2,062 MiB 2,590 MiB
QueenMQ, spawned, on 2,512 MiB 3,019 MiB
QueenMQ, forked, off 496 MiB 1,026 MiB
QueenMQ, forked, on 225 MiB 755 MiB

Over 20 minutes at 500 jobs/s of the job that holds 8 MiB, about 9,400 jobs per worker, the private memory of a worker stayed at 37.1 MiB for Horizon and 10.1 MiB for a forked one; the 64 forked workers copied 7 MiB more of their shared pages in all, 0.1 MiB each.

Over 23 hours at 50 jobs/s of the same job, about 64,000 jobs per worker, with the PHP client 2.3.0 and no worker restarted, the median forked worker went from 9.66 to 10.04 MiB of private memory and stayed there; the largest never passed 11.3 MiB. The step is about 0.35 MiB, once per worker: 51 of the 64 had taken it by the end of the dispatch. The application’s anonymous memory went from 638 to 658 MiB (the 23-hour run). Four of the 4.1 million jobs failed without running, with tries 1: the broker counted a delivery that no worker received, and the next worker got the job as its second attempt. The broker fix takes that attempt back (#95).

Against Horizon with the opcache off, forked workers with the opcache saved 27 MiB per worker with either job, the booted framework, and the ratio fell from 8.7 to 3.3. A real application’s jobs allocate more than the benchmark’s, so expect the saving per worker, not the ratio. The saving is the size of the booted application, which is small here: an application with more code and providers boots bigger, and forking saves more per worker.

Over the 15-minute soak, at 16 workers, the application’s processes stayed flat for both engines: 529 MiB of proportional set size for Horizon and 81 MiB for QueenMQ, supervisor and workers together. Redis grew from 67 to 611 MiB with Horizon’s job records; the broker grew from 59 to 117 MiB, plus the page cache of its log. In the 23-hour worker run, with retention off, the broker’s anonymous memory grew with the messages it kept, from 59 to 741 MiB for 4.1 million of them, about 175 bytes each, and stopped growing when the dispatch ended.

Delivery profiles and prefetch auto

2026-10-05, QueenMQ only: the Rust supervisor with prefork, the command-line opcache and lease renewal in the master, three runs per lane (five at 500 jobs/s), each on a fresh broker. Every worker long-polls for 1 s. The profiles are the Laravel guide’s: safe is prefetch 1 with synchronous ACKs; balanced is prefetch 'auto'; fast adds ack_async and pop_ahead.

Jobs per second of 32 QueenMQ workers by delivery profile. Empty jobs: safe 3,131, balanced 5,681, fast 5,994. 10 ms jobs: safe 1,548, balanced 2,003, fast 2,867. 200 ms jobs: safe 139, balanced 139, fast 147.SafeBalancedFast02,0004,000jobs per secondEmpty jobs10 ms jobs200 ms jobs3,1315,6815,9941,5482,0032,867139139147
Balanced and fast size each pop with prefetch 'auto'. Short jobs gain the most; jobs of 200 ms keep one job per pop and run as fast as safe. Source: benchmark-queen/2026-10-05-laravel-auto-prefetch
32 workers, one burst Safe Balanced Fast Fast, prefetch 4
Empty jobs, jobs/s 3,131 5,681 5,994 6,046
10 ms jobs, jobs/s 1,548 2,003 2,867 2,835
200 ms jobs, jobs/s 139 139 147 148
Pops per job, 10 ms jobs 1.00 0.07 0.09 0.26
Broker CPU per job, 10 ms jobs 0.29 ms 0.19 ms 0.19 ms 0.22 ms
  • With short jobs 'auto' reached batches of about 15 jobs: a worker popped once every 15 jobs, and the broker spent a third less CPU per job.
  • With 200 ms jobs it kept one job per pop.
  • At a steady 500 jobs/s with 16 workers the p50 was 14 to 15 ms in every profile and the p95 23 ms.
  • Fast popped ahead after every job on that quiet queue, about two pops per job: the broker spent 0.81 ms of CPU per job against 0.70 for safe and 0.72 for balanced.

The p99 at 500 jobs/s

Profile p99 of each of five runs
Safe, sleep 1 29, 29, 28, 36, 28 ms
Safe, sleep 0 29, 29, 29, 78, 29 ms
Balanced, sleep 1 30, 37, 29, 29, 338 ms
Balanced, sleep 0 29, 29, 30, 30, 30 ms
Fast, sleep 1 29, 29, 29, 29, 29 ms

The 338 ms run is a burst after a quiet spell. 12 of the 16 workers had come back empty from their long poll and were sleeping for Laravel’s sleep of 1 s, so the 4 awake ones took the burst and grew their batches, and every job over 100 ms came in those first 1.5 s. With sleep 0 an idle worker polls again at once; the broker’s CPU per job stayed within 2%. The 78 ms run of safe is a stall of about one second that hit all 16 workers alike, with one job per pop, so batches play no part in it.

A first version of 'auto' let the batch grow under any pop, and at 500 jobs/s its p99 reached 63 and 145 ms in two runs of three, while a worker held several jobs and others sat idle. The batch now doubles only after two pops in a row came back full, and shrinks to what a short pop brought; the balanced and fast lanes above are that version.

CPU

Lane, CPU per job Horizon, application and backend QueenMQ, application and backend
50,000 jobs, 32 workers 1.83 and 0.37 ms 0.91 and 0.23 ms
500 jobs/s, 16 workers 1.46 and 0.49 ms 1.35 and 0.87 ms
400 jobs/s for 15 minutes 1.56 and 0.54 ms 1.46 and 0.97 ms

Under load the Raft broker spends less CPU per job than Redis, and at a low, steady rate it spends more. A profile of the broker at 500 jobs/s explains it (raw/broker-profile.md in the archive): under a bulk load one Raft entry carries about 12 jobs and shares its fixed costs, while at 500 jobs/s each job is its own entry, and about half of the broker’s CPU goes to handing work between threads (waking them, system calls, scheduling). The fsync itself is about 3%.

The same profile found a cost on the client side. pop_ahead sent a pop with the last job of every batch, and at a low rate the batch is one job, so that pop came back empty. The driver now pops ahead only after a full batch: at 500 jobs/s the workers sent 0.99 pops per job instead of 1.97, which took 12% off the broker’s CPU per job and 8% off the application’s, with the same latency and no change under load.

The worker-side features, one at a time

On the Linux server (2026-10-01), 50,000 jobs and 32 workers, adding one QueenMQ setting at a time:

QueenMQ settings Jobs/s Application memory
prefork, the command-line opcache, a renewal helper per worker 1,890 417 MiB
renewal in the supervisor instead of the helpers 1,879 116 MiB
and ack_async 2,270 120 MiB
and pop_ahead 2,794 120 MiB

Renewal in the supervisor saves memory, because the helper process beside each worker is gone. The asynchronous ACK and the pop ahead save time: a worker that waits for a write waits for an fsync, and these two send the ACK of a job and the pop of the next batch while a job runs, so the worker no longer waits for either. The cURL transport against Guzzle, with 100,000 empty jobs, ran 6,030 against 5,441 jobs/s, at 1.03 against 1.41 ms of application CPU per job.

Failures, replicas, a soak and Laravel features

2026-10-02, the Linux server of 2026-10-01, QueenMQ on one Raft node of 2.0.0-beta.3. The application had 4 CPUs and 1 GiB, the broker and Redis 2 CPUs and 2 GiB each, every lane on a fresh backend. QueenMQ ran its production-like settings: prefetch 1, synchronous ACKs, lease renewal in the supervisor and prefork workers. A fourth column runs the Rust engine with prefetch 4, ack_async and pop_ahead. Every lane had 2 workers unless it says otherwise, a pool timeout of 20 s, retry_after 30 s and shutdown_grace 35 s. Each job writes its attempts to a log on the lane’s volume, so every check reads what the jobs did, not what an engine reports. QueenMQ PHP is the PHP supervisor (php artisan queen:supervise), QueenMQ Rust the native queen-supervisor master.

A lane passes when every job ended as its kind says (completed once, or failed once into failed_jobs, and into the dead-letter queue on QueenMQ) and no two attempts of one job overlapped.

What happens Horizon QueenMQ PHP QueenMQ Rust Rust, prefetch 4
A job throws on every attempt (tries 3, backoff 2) Expected: 3 attempts at least 2 s apart, then one failed-job row pass pass pass pass
A job throws on its first attempt only Expected: It completes on its second attempt, once pass pass pass pass
release() once, or fail() Expected: A released job completes on its second attempt; fail() is final pass pass pass pass
A 15 s job with timeout 3 (tries 2) Expected: The worker is killed and replaced at each attempt; one failed-job row pass pass pass pass
A job takes the worker over --memory (110 of 96 MiB) Expected: The job completes once, then the worker is replaced pass pass pass pass
A job exceeds PHP’s memory_limit (256 of 128 MiB, tries 2) Expected: The worker dies at each attempt; one failed-job row pass pass pass pass1 of 2 jobs failed without running, until the crash hand-back
SIGKILL on a worker during a 4 s job Expected: The killed job runs again; all 8 complete once pass pass pass pass
SIGKILL on the master during 4 s jobs, then a restart Expected: All 8 jobs complete once pass pass pass pass
A deploy (SIGTERM) during 10 s jobs Expected: The jobs finish during the grace and do not run again pass pass pass pass
A deploy during 60 s jobs, longer than the grace Expected: The killed jobs run again; each completes once pass pass pass pass
One worker runs two 60 s jobs of one prefetched batch; a deploy kills it in the second Expected: The first job, done before the deploy, does not run again pass pass pass pass
The backend restarts while 300 jobs of 100 ms run Expected: All 300 complete once fail2 of 300 jobs ran twice pass pass pass
The backend freezes for 5 s during 10 s jobs Expected: Nothing runs twice pass pass pass pass
The backend freezes for 45 s, longer than the 30 s lease Expected: All complete; a job may run twice pass pass pass pass
dispatch() while the backend is stopped Expected: It throws, and works again once the backend is back pass pass pass pass
Two masters on one queue; SIGKILL on one, as a node loss would Expected: All 40 jobs complete once pass pass pass pass
A rolling restart of two masters while 60 jobs run Expected: All 60 complete once pass pass pass pass

Horizon’s two duplicates are worth understanding, because both stacks promise the same thing there. The two jobs finished while Redis restarted, so the worker could not remove them from Redis’s reserved set; their reservation expired after retry_after and they ran again 30 s later. The QueenMQ client tries a failed request up to three times, waiting 1 s and then 2 s, so the ACKs sent during the broker’s restart landed when it came back. Both stacks deliver at least once, and a job must be safe to run twice on either.

What the failure lanes found

The first run of these lanes, with the PHP client of 2026-10-01, found three defects in the client and the supervisors, all fixed in the release the guides describe. With prefetch above 1, a graceful stop handed the unstarted jobs of a batch back with a retry ACK, which the broker counts as a delivery, so jobs reached failed_jobs before running tries times. With ack_async, the ACK of a finished job could stay unwritten until the next job ended, so a worker killed during that job ran the finished job again. And both supervisors counted a worker that Laravel killed after a job timeout as a crash: after five of them the crash circuit cut the pool to one worker for 30 s.

Lane, Rust engine with prefetch 4 First run After the fixes
A 15 s job with timeout 3 3 of 4 jobs reached failed_jobs after one run Pass
A job over --memory 1 of 6 jobs failed instead of completing Pass
A deploy during 60 s jobs 1 of 2 jobs completed twice Pass
A deploy during the second job of a prefetched batch The job done before the deploy ran again Pass
A job over PHP’s memory_limit Both jobs failed after one run 1 of 2 failed without running; passed with the crash hand-back

With the supervisor fix, the four jobs of the timeout lane reached failed_jobs after 44 s on both QueenMQ engines, against 126 to 147 s while the crash circuit held the pool back. The prefetched-batch lane is the proof of the ACK fix: with the client before the fix, the first job, finished before the deploy, ran again; with the fix it ran once, on all three QueenMQ profiles. An audit of the PHP client and both supervisors during the same night fixed ten more defects, each with a test that failed before its fix.

One limit remained after those fixes: a crash charged an attempt to every prefetched job its worker had not started. In the memory_limit lane with tries 2, a job that never ran reached failed_jobs because the job ahead of it crashed the worker twice, and in the worker-kill lane the three jobs the killed worker had not started ran 31 s later as their second attempt. The crash hand-back closes it: each worker journals the transaction its shutdown would send, and whatever renews its lease (the Rust master, or the PHP lease helper) sends it when the worker dies holding the lease. A rerun of five lanes on the same server (raw/vm-hand-back in the archive) passed all twenty, on both engines with prefetch 1 and with prefetch 4. In the worker-kill lane the master handed back the three jobs the killed worker had not started, and the first ran 5 ms later as its first attempt. In the memory_limit lane the master handed the job that never started back 11 ms after the fatal error, it ran 6 ms later, and both jobs then ran twice, as tries 2 says. Three cases still charge the attempt: a lost node, a crash that takes the lease helper with the worker, and a batch popped ahead whose answer the worker had not read yet.

The soak

45 minutes at 5 jobs/s on 8 workers: 85% short jobs, 5% of 5 to 20 s, 5% that fail once and succeed on the retry, 3% that always fail (tries 2) and 2% delayed by 10 to 60 s. A worker was killed every 10 minutes, and a deploy restarted the application after 23 minutes. This table is the second soak, on the release candidate, with the four profiles running at once:

Horizon QueenMQ PHP QueenMQ Rust QueenMQ Rust, prefetch 4
Jobs dispatched 13,501 13,501 13,501 13,501
Jobs that ended as their kind says all all all all
Rows in failed_jobs, one per permanent failure 398 398 398 398
Dead-letter entries (no DLQ) 398 398 398
Master resident memory, first third to last third 49.1 to 49.1 MiB 58.5 to 58.4 MiB 7.0 to 7.0 MiB 7.0 to 7.0 MiB
Median worker, first third to last third 51.0 to 56.6 MiB 40.5 to 44.9 MiB 40.4 to 43.0 MiB 41.1 to 45.0 MiB

The first soak, on the three engines before the last client fixes, gave the same counts and the same flat masters:

Horizon QueenMQ PHP QueenMQ Rust
Master memory, first third to last third 48.9 to 48.9 MiB 58.3 to 57.8 MiB 6.9 to 6.9 MiB
Median worker, first third to last third 50.7 to 54.7 MiB 40.9 to 47.0 MiB 40.5 to 45.0 MiB
The master's resident memory over the 45-minute soak, sampled every minute, with a worker killed every 10 minutes and a deploy after 23 minutes. Every line is flat: QueenMQ PHP at about 58 MiB, Horizon at about 49 MiB and QueenMQ Rust at about 7 MiB, from the first sample to the last.master resident memory, MiB0204060010203040minutesworker killedworker killeddeployworker killedworker killedQueenMQ RustQueenMQ PHPHorizon
The masters do not grow: a worker killed, a deploy and 45 minutes of mixed jobs leave each one where it started. Source: benchmark-queen/2026-10-02-laravel-failure-matrix/raw/soak-memory.csv
The median worker's resident memory over the 45-minute soak, sampled every minute. It grows slowly on every engine, from about 48 to 57 MiB on Horizon and from about 32 to 47 MiB on both QueenMQ engines, and dips by about 2 MiB when the deploy after 23 minutes replaces the workers.median worker resident memory, MiB0204060010203040minutesworker killedworker killeddeployworker killedworker killedQueenMQ RustQueenMQ PHPHorizon
Long-lived PHP workers grow slowly on every engine; the workers forked from one booted Laravel start smaller. Source: benchmark-queen/2026-10-02-laravel-failure-matrix/raw/soak-memory.csv

Every median worker grows slowly over 45 minutes, as long-lived PHP workers do; the masters do not move. Both rows are resident memory, which counts a shared page in every process that maps it, so a forked worker’s row includes the pages it shares with the fork server. The master row is one process: Horizon also runs a horizon:supervisor process the size of its master, and with prefork each QueenMQ engine runs a fork server that holds the booted Laravel. Memory counts them together.

Laravel queue features

Each scenario dispatches ordinary Laravel jobs, listeners, notifications or batches and checks the behaviour the Laravel documentation states. Horizon is the reference: a check that failed on Horizon was a mistake in the test, and was fixed. All 32 passed on the four profiles.

Area Features checked
Delays and retries delay(), a DateTimeInterface delay, backoff as an array, retryUntil(), release() with a delay, maxExceptions, failOnTimeout, ThrottlesExceptions
Uniqueness and limits ShouldBeUnique, ShouldBeUniqueUntilProcessing, WithoutOverlapping, RateLimited, Skip
Chains and batches Bus::chain(), a failing chain’s catch(), Bus::batch() with then(), catch() and finally(), a batch with a failing job, allowFailures(), $batch->cancel(), queue:retry-batch
Other queued work a queued event listener, a queued notification, a queued closure with catch(), ShouldBeEncrypted, $deleteWhenMissingModels, a job that calls pcntl_fork()
Transactions and events afterCommit(), Queue::before(), after() and failing()
Commands queue:retry, queue:forget, queue:flush, queue:prune-failed, queue:monitor, Queue::size()

Laravel’s Redis queue stores a due time in whole seconds, so a Horizon retry can start up to a second early or late: for backoff [1, 3], Horizon’s retries came 1.81 and 4.03 s after the failure, QueenMQ’s 1.02 to 1.04 and 3.05 s. queue:clear is not supported on QueenMQ: the command fails, and the jobs still run.

More partition stripes, 2026-10-02

The driver spreads ordinary jobs over a queue’s partition stripes, and the broker leases each stripe to one worker at a time. The driver allowed at most 64, the partitions one pop checks out, so at most 64 workers could run a queue’s ordinary jobs at once. It now allows up to 1,024 stripes while each pop still asks for at most 64, and this campaign measured 64, 128 and 256 stripes on the Linux server with the Rust supervisor, prefetch 4, ack_async, pop_ahead and the Raft broker. Medians of three runs, 10 ms jobs; every run completed its exact job set with no duplicate:

Lane 64 stripes 128 stripes 256 stripes
50,000 jobs drained by 128 workers 4,722 jobs/s 7,895 jobs/s 9,257 jobs/s
50,000 jobs drained by 64 workers 4,580 jobs/s 5,380 jobs/s 5,519 jobs/s
500 jobs/s, 16 workers, p95 end to end 23.4 ms 17.6 ms 16.7 ms
50 jobs/s, 16 workers, broker CPU per job 1.49 ms 1.53 ms 1.49 ms
Broker memory, 128-worker drain 125 MiB 122 MiB 138 MiB

With 64 stripes, 128 workers drained no faster than 64. At 50 jobs/s the stripe count changed neither the broker’s CPU per job nor the latency beyond run-to-run variation. Not measured: more than 256 stripes, several queues on one connection and the PHP supervisor. The dataset, with every run and the scripts, is benchmark-queen/2026-10-02-laravel-partition-stripes/.

What these runs do not show

  • Every Raft run used one node. A three-node cluster adds a network round trip to every write before it is answered, and none of these runs measured one.
  • One virtual machine’s disk set the fsync cost and with it every absolute number. Another server moves them.
  • The longest soak with a verdict lasted 45 minutes; a 2.5-hour soak of the release candidate ran to its end, but its report failed, so it has none. Forked workers ran for 23 hours, once, with QueenMQ only; longer runs, Horizon over the same time and a broker with retention on over hours were not measured.
  • The benchmark job is tiny, and the job that holds 8 MiB is one way to allocate. Real jobs keep more private memory per worker, which prefork does not share, and pull the throughput ratios towards one as the job itself dominates.
  • The QueenMQ team wrote these lanes and their checks; nobody else has repeated them yet.
Navigation

Type to search…

↑↓ navigate↵ selectEsc close