Versions and compatibility

The current broker and proxy release is 1.5.3. The compatible JavaScript, Python, Go and Rust clients, queen-protocol and the Go command-line client remain at 1.3.0; the PHP/Laravel source has no tagged Packagist release yet, and the C++ header is consumed at a repository revision. The broker reports its version on GET /health and in its boot log; the value is compiled in from a single file, so the binary, the container tag and the health document can never disagree.

The HTTP API is the compatibility surface

The contract is the HTTP API: the paths under /api/v1 and /streams/v1, their methods, their JSON field names, and the status-code behaviour described in Status codes and error bodies. An SDK is a convenience over that surface, not a separate contract: anything an SDK can do, a plain HTTP call can do.

Everything else is an implementation detail and changes without notice:

• the SQL schema, the table shapes, and all stored-procedure names and signatures;
• the routes under /internal/, which exist for broker-to-broker traffic;
• the multi-broker mesh frame format;
• the embedded dashboard bundle;
• the boot log’s field names.

Two consequences follow from taking the JSON field names (rather than their meaning) as the contract. First, fields are added, not renamed: a client must ignore fields it does not know. Second, some fields are opaque by contract. seq and startOff on a pop response kept their names across the storage-engine rewrite while their meaning changed completely: seq now carries a segment’s base offset and startOff a frame index within it. A client that treats them as tokens to hand back is correct; a client that does arithmetic on them is not.

The embedded Rust API is beta

Since 1.0.0 the broker crate also ships a library target: queen-engine on crates.io, importing as queen, whose engine API runs the broker in-process. That Rust surface is a second contract and it is explicitly beta: the types can change between releases while it hardens, and it does not follow the HTTP surface’s stability promise yet. What does carry over is the wire vocabulary, because the engine’s requests and responses are the queen-protocol types, the same crate the Rust client compiles against. Build against the engine expecting to read release notes; build against HTTP expecting not to.

The schema is applied at every boot

The broker owns its own schema. It embeds schema.sql and 33 stored-procedure files (001–033) into the binary at compile time and applies all of them on every start, in lexical file order, before it accepts a request.

Three properties make that safe to do on every boot:

• Idempotent statements. CREATE TABLE IF NOT EXISTS, CREATE OR REPLACE FUNCTION, ADD COLUMN IF NOT EXISTS. Re-applying is a no-op when nothing changed.
• Serialised across instances. The apply holds a session advisory lock, so a rolling restart of several brokers cannot run two DDL passes concurrently. A replica waiting its turn polls a try-lock rather than blocking, so it holds no snapshot open against the one applying.
• Fail-fast, with bounded patience for locks. Statements run one per transaction, split out of each file by a dollar-quote-aware splitter, and every lock-taking statement runs under a 2 s lock_timeout. The advisory lock serialises appliers against each other but not against a peer that already booted and is running its background cycles, so a statement that cannot get its lock (SQLSTATE 55P03) or deadlocks against live traffic (40P01) is retried with jittered backoff, up to 40 slices of patience (about three minutes); past that, and on any other DDL error, the boot aborts naming the failing file and statement and the sessions holding the conflicting locks. A CREATE INDEX CONCURRENTLY runs with the timeout off and no retry, and a run of DROPs is grouped into one transaction with the statement that follows it, so a live peer never observes the gap of a drop-then-create pair. A broker never serves against a schema it could not bring to the expected shape.

The apply only creates and adds. Every table and every function is defined exactly once, by exactly one file: the CREATE TABLEs declare the final columns for a database created from scratch, and a column added since its table first shipped also rides beside it as an idempotent ADD COLUMN IF NOT EXISTS (a new index on an already-populated hot table as CREATE INDEX CONCURRENTLY IF NOT EXISTS), so a database created by an earlier build of this broker picks the additions up at boot. Nothing is migrated, rewritten or folded beyond that. Order in the file list is load-bearing only where a function body has to resolve a table at creation time. The table files come first, and the partition-counter trigger attachment comes after both the table it attaches to and the functions it binds.

The deployment model is always-virgin

A deployment starts from an empty database (or an empty schema): the boot apply is the schema, not a migration engine. There is no migration step, no version table, and nothing to run by hand, because there is nothing to migrate. Restarting the same version re-applies the same definitions as a no-op, and a rolling restart of identical binaries is safe.

The apply brings an empty database to this version’s shape; an existing one it brings forward only where a file spells the step out (the additive column and index statements above). A CREATE TABLE IF NOT EXISTS against a table that already exists in an older shape leaves the older shape in place, so pointing the broker at a database created by a different implementation, or by a pre-release build, is unsupported: stand up a fresh database and cut over. A downgrade works the same way: an older broker would re-apply its own older definitions over a schema it does not understand. Roll back by restoring a dump into a fresh database, not by restarting an older image against the current one.

One rolling-restart detail survives from the file mechanics: a definition whose return type changed between builds cannot be a CREATE OR REPLACE, so a few are expressed as a drop followed by a create (the pop family is the notable case), and the advisory lock does not stop an older instance from serving while a newer one applies. Roll instances one at a time, and do not point two different broker versions at one database as a steady state.

Running as a low-privilege user

QUEEN_APPLY_SCHEMA=0 skips the apply entirely. A privileged role then owns the DDL and pre-applies it, and the broker’s own database user needs no CREATE rights. The trade is that the ordering guarantee becomes yours: a broker that skips the apply and finds an older schema fails on the first request that needs a missing object, not at boot.

With the apply enabled, the database user needs CREATE SCHEMA and nothing else. The schema requires no extensions: gen_random_uuid() is used as a core built-in, not the pgcrypto function. It does require PostgreSQL 15 or newer, because one unique index is declared NULLS NOT DISTINCT. The test matrix runs against PostgreSQL 16.

Coming from the retired 0.16 broker

The 0.16 broker was a different implementation: C++, and a different storage engine. Replacing it with the current broker is a cutover rather than a data migration, and the cutover runs at the application level.

The move carries no data. The migrate subcommand reports as much: all it does is log migration is not supported on the log engine and exit 1. It never opens a database connection, and there are no migration HTTP endpoints either.

The boot apply never reads what it finds. It creates and adds its own objects, whatever the database already holds; nothing of an earlier engine’s is dropped, read or reconciled. Read that as the always-virgin model rather than a compatibility promise: this broker expects an empty database, and against one carrying an earlier broker’s tables it would create its own objects alongside data it will never read.

The practical upgrade is therefore a cutover, not an in-place migration: stand up the new broker against a clean database, move producers over, let the old consumers drain the old broker, and keep a dump of the old database as the record.

Backup and restore are plain pg_dump and pg_restore, run against the broker’s database like any other PostgreSQL workload.

Mixed versions in a mesh

A multi-broker deployment can be rolled one instance at a time. Where a frame from an older peer omits information a newer instance expects (the tenant on a wake or invalidation frame), the newer instance fans the effect out rather than guessing: it wakes every tenant holding that queue name and drops every tenant’s cache entry for it. Over-waking costs one empty probe; over-invalidating costs one lazy re-fetch. Neither is a correctness problem, so a mixed-version mesh is safe.

Related pages