Testing

test/run.sh is the whole thing. It builds a broker image from the current tree, builds one runner image per language, then runs every selected suite against every applicable topology as its own docker compose project on its own private network, in parallel, and prints a pass/fail matrix.

test/run.sh

Requirements are Docker and Compose v2. Nothing binds a host port and every PostgreSQL runs on tmpfs, so runs are disposable and cannot collide with each other or with a database you are using locally.

What it builds

The broker image is built from server/Dockerfile and tagged queen:test. Because that Dockerfile copies server/webapp into the build context rather than building the dashboard, the dashboard has to exist first. See Building and developing. Runner images come from test/runners/<suite>/Dockerfile. --no-build-broker reuses an existing queen:test, and --no-build reuses everything.

Suites

Environment-variable naming diverges between suites. The JavaScript suite reads QUEEN_SERVER_URL, the Go and Python streams tests read QUEEN_URL, the CLI reads QUEEN_SERVER. Each runner’s entrypoint maps one canonical set (QUEEN_HTTP_URL, QUEEN_PG_*) onto whatever its suite actually reads. Two of those mappings are the difference between running and silently passing: the CLI suite’s TestMain exits 0 having run nothing without QUEEN_E2E=1, and the Go streams tests target localhost and skip if only QUEEN_SERVER_URL is set.

Topologies

The tenanted lane exists to prove a negative. Turning native tenant scoping on must change nothing for a client that sends no tenant header: that is the default-tenant path, and it is what every cloud cell serves. So run.sh runs the unmodified suite on both lanes and compares them, printing a TENANCY PARITY verdict. The exit code is the hard gate; because an exit code is coarse (99 of 112 and 100 of 112 are both rc=1), it also parses each suite’s printed pass tally where one exists and compares that too. A divergence fails the run in either direction. A behaviour change caused by the flag is a regression even when the flag-on side is the greener one.

The ha and ha-tenanted compose files are the same file as single and ha respectively, with QUEEN_TEST_TENANCY exported by run.sh. Duplicating the stack definitions would let the two lanes drift, which is exactly what the parity gate exists to catch.

Isolation

Each pair of suite and topology is its own Compose project, so it gets a private network and its own PostgreSQL and broker. This is not tidiness. The JavaScript, Go and Python suites all wipe the same test-queue name patterns (test-%, edge-%, pattern-%, workflow-%), and the CLI suite flips broker-global maintenance mode. On a shared stack they would corrupt each other’s results. Parallelism is therefore per stack, four at a time by default (-j N, or QUEEN_TEST_MAX_PARALLEL).

How readiness is gated

The runtime broker image is debian:bookworm-slim carrying only the queen binary and none of the tooling a healthcheck would need, in particular no curl. So readiness cannot be a Compose healthcheck on the broker; it lives in the runner, which does have curl. Every runner entrypoint calls test/runners/common/wait-for-broker.sh first, which blocks on curl -fsS against each URL in QUEEN_WAIT_URLS until it succeeds, up to QUEEN_WAIT_TIMEOUT seconds (default 180), and exits 97 on timeout.

That gate is stronger than it looks. The broker applies schema.sql and every stored procedure before it binds its HTTP listener, and /health performs a real database round-trip and answers 503 while PostgreSQL is unreachable. A 200 from /health therefore means broker plus PostgreSQL plus schema, all three, with no init SQL to mount.

Running a subset

--topo filters only the client lanes; mesh and tenancy always supply their own topology. Logs for every cell land in a temporary directory printed at the start of the run, and the tail of every failing cell is echoed at the end.

What the tenancy suite actually asserts

test/runners/tenancy/tenancy-check.sh drives the HA pair directly with the trusted x-queen-tenant header and no proxy in the picture. Every scenario deliberately uses the same queue name, the same partition name and the same consumer-group name for both tenants, with traffic on both brokers, so shared-name collisions are the default rather than the exception. It checks that queue identity and configuration are per-tenant and read back consistently on both brokers; that no message crosses tenants; that a foreign partition id cannot advance another tenant’s cursor; that deduplication keys are per-tenant while within-tenant deduplication still fires; that listings are scoped; that a consumer-group name shared by two tenants keeps independent cursors; that one tenant’s claim-and-ack cannot hide the other’s pending message through the in-process hot-list ring; and that a push by one tenant delivers nothing to another’s parked long poll.

It also refuses to pass vacuously. A probe up front configures one queue name for two tenants with different lease times and aborts if the broker does not keep them apart, so the lane can never “pass” by comparing the default tenant with itself.

Expected-but-harmless crosstalk is printed as note lines rather than asserted, because it is wasted work and not a correctness property: PostgreSQL is the authority on those paths and every stored-procedure call carries the tenant, so the cost is a re-query, not a leak.

What is green on this tree

Verified green: the broker’s in-process unit tests, queen_proxy’s own unit tests, the C++ client suite plus its 14 proxy-contract assertions, the mesh suite, the tenancy suite at 45 assertions and 0 failures, the JavaScript unit tests at 91 of 91, and the Go ./streams/... package at 13 of 13. The core messaging surface (push, pop, ack, transactions, deduplication, consumer groups, dead-letter queue, subscription modes, watermarks and the ack window) is green in five languages.

Two failures are known and deterministic: TestGateTokenBucketBasic in the Go client, and TestLoad_BenchSmoke in the CLI, which is a benchmark deadline rather than a semantic check. Streaming window arithmetic has been the noisy part of the suite historically; this site therefore makes no per-test verification claim about window arithmetic, and the pages that show streams do not claim a specific test proves them.

A defect in the harness

Every stack cell runs:

docker compose -p "$proj" -f "$compose" up --abort-on-container-exit --exit-code-from runner

--exit-code-from runner propagates the runner’s exit code, and --abort-on-container-exit tears the project down when any container exits. The two interact badly: a runner that is merely stopped, rather than having finished and exited non-zero, can yield exit code 0, and run.sh scores 0 as PASS. The matrix cell then says PASS for a suite that never finished.

Two consequences for anyone reading a green matrix:

• A cell that passes suspiciously fast is worth opening. run.sh prints each cell’s duration next to PASS precisely so an anomalously short run stands out.
• The tenancy parity gate compares printed pass tallies as well as exit codes, so for the suites that print a tally, a truncated run shows up as a tally mismatch rather than as a silent pass. Suites that print no tally have no such backstop.

The fix is to assert on the runner’s own printed summary rather than on the Compose exit code. Until then, treat the matrix as a fast screen and the per-cell logs as the evidence.

In CI

.github/workflows/tests.yml is the same harness. It compiles the broker once into an artifact so the matrix cells load an image instead of recompiling Rust six times, then runs one job per suite with test/run.sh --suite <name> --no-build-broker -j 1. The Rust unit tests run natively in their own job with cargo test --manifest-path server/Cargo.toml --locked.

Two other workflows touch tests. cli.yml builds and unit-tests queenctl on Linux and macOS for every change under clients/client-cli or clients/client-go, and runs its heavier end-to-end suite only on pushes and manual dispatch. cpp-server-build.yml still exists, triggers on the release branch, and runs make deps and make build-only in server/, targets that no longer exist. It is dead and cannot pass.

Deeper gates, run by hand

• server/mesh-verify.sh builds its own debug binary and drives two brokers on localhost to check cross-replica wake latency, dead-peer detection and reconnection. The mesh suite only asserts the pair is connected; this is the behavioural gate.
• proxy/scripts/isolation-smoke.sh runs two-tenant isolation through the proxy against the development cell, asserting on response content rather than status codes, and covering blocked operator surfaces, API-key scopes, revocation, storage quota and meters. With QUEEN_PROXY_ENFORCE=true it adds live 429 checks; in shadow mode those are reported as counted skips, never as silent passes.
• proxy/scripts/sdk-smoke/run.sh drives the JavaScript, Python and Go SDKs through the proxy.