Upgrades

Queen’s deployment model is always-virgin: a deployment starts from an empty database, and the DDL the broker applies at boot only creates. There are no migration files, no version table, and no upgrade ALTERs: CREATE TABLE declares the final shape, once. The consequence is stated plainly up front because the whole page follows from it:

There is no in-place upgrade path. Upgrading a deployment means standing up the new version against a fresh database and cutting traffic over. Pointing a new build at a database created by an earlier version is unsupported: the boot apply will not reshape existing tables, and nothing else will either.

What is routine and coordination-free is restarting or replacing replicas of the same version. Both halves are below.

What happens at boot

schema::apply runs before anything else touches the database:

1. Take one connection from the pool and hold a session advisory lock, pg_advisory_lock(778120010). This serialises the apply cluster-wide, so replicas restarting at the same moment cannot race each other through the DDL.
2. Execute sql/schema.sql, then each sql/procedures/*.sql file in lexical order (23 procedure files, 24 batches in total). Every statement is idempotent (CREATE OR REPLACE FUNCTION, CREATE TABLE IF NOT EXISTS, CREATE INDEX IF NOT EXISTS), so re-applying on every boot is safe.
3. Release the lock on every exit path, success or failure.
4. Fail the process on any DDL error, naming the file that failed (procedures/005_log_ack.sql: …). A broker never serves a request against a schema it could not bring to the expected shape.

Each function and table is defined exactly once, by exactly one file. The lexical order is still load-bearing, for a narrower reason than redefinition: the table files (001_log_schema, 002_streams_schema) must run before the SQL-language function bodies that resolve those tables at creation time, and the partition-counter trigger attachment (020_log_partition_counters) must run after both the table it attaches to and the functions it binds.

Because every statement only creates, the apply has exactly two outcomes worth naming: on an empty database it builds the full schema; on a database the same version already initialised it verifies and re-asserts (the per-table storage and autovacuum parameters are re-applied on every boot, which is why they cannot drift). What it cannot do is bring a database built by a different version to this version’s shape. That is the always-virgin contract, not an accident.

Restarting and replacing replicas (same version)

Brokers are stateless and every route is served by every replica, so restarting replicas (or replacing their binaries with the same version, as in a node drain or an image re-pull) needs no coordination and no client changes.

Startup drains any leftover spool before the listener binds: crash-left partial files are finalised, then finished files are replayed oldest-first. It stops at the first failure and hands the rest to the background drain loop, and it gives up after an hour of continuous draining, so a large spool delays the port opening but a dead database does not block boot indefinitely.

In-flight leases

Leases are rows, not memory. A pop takes a lease on the span (committed, batch_end] by writing worker_id, batch_end, lease_acquired_at and an absolute lease_expires_at into queen.log_consumers. Restarting a broker does not touch those rows and does not release those leases.

What follows is exactly the at-least-once contract, with no special restart case:

• The leased span stays invisible to other consumers of that group. A pop only claims a (partition, group) whose worker_id is null, or whose lease_expires_at is null or already in the past.
• When the lease expires, the whole batch is redelivered. Not the un-acked tail: the whole span, because the cursor never moved.
• The redelivery is counted. The consumer row remembers the start offset of the previous non-auto-ack delivery; a delivery starting at that same offset is a redelivery and increments the attempt count, rather than resetting it. Once the queue’s retry budget is exhausted, the head message dead-letters.
• A client holding a lease across the restart can still acknowledge it, to any replica, as long as the lease has not expired. The broker’s in-memory ack fast path is lost with the process, and the ack falls back to resolving the transaction identifier through the hash sidecar in SQL. Same result, one extra lookup.

The practical consequence for planning: a consumer whose in-flight batch was interrupted will see those messages again after at most one lease time. Size leaseTime with that in mind: long enough that normal processing finishes, short enough that a redelivery after a restart is not an hour away.

Upgrading a deployment: fresh database + cutover

An upgrade to a new version is an application-level cutover, the same shape whatever versions are involved:

If you cannot drain (a backlog you must not lose), the only reliable path is to read the messages out through the old deployment’s own API and push them into the new one as new messages, giving each one a deterministic transactionId so a retry of your copy loop cannot duplicate it. Nothing in the broker does this for you.

Rollback

The cutover model makes rollback an argument for patience rather than a procedure: the old deployment and its database still exist until you retire them. Keep both running (producers stopped, consumers drained) until you are confident in the new deployment; rolling back is pointing clients back at the old broker. Once you have deleted the old database, there is nothing to roll back to: messages pushed to the new deployment in the meantime would need the same copy-loop treatment in reverse.

Coming from 0.16

The previous Queen broker was a C++ implementation on a different storage engine. Version 1.0 is not an update of it, and there is no data migration, which is the always-virgin model again, not a special case.

• The queen migrate subcommand exists only to say so. It logs that migration is not supported on the log engine and exits 1 without opening a database connection.
• The current engine’s data lives in queen.queues (queue identity and configuration in one table) and the five log tables: log_partitions, log_segments, log_txns, log_consumers and log_dlq. None of the old broker’s tables are created by this schema, and none of its data is read.
• There is also nothing to convert on disk. The old broker’s state was in PostgreSQL too; it is in tables the new engine does not use.

The cut-over is the standard one above: drain the 0.16 deployment, stand up the new version against a fresh database, repoint clients, with the same copy-loop escape hatch if the backlog cannot be drained.

Confirming which version you are running

server/server.json is the single source of truth for the version: build.rs embeds it into the binary at compile time, build.sh uses it as the container image tag, and it is what /health and queenctl version report.

curl -s http://localhost:6632/health

The boot log line listening carries the same version alongside the resolved engine settings, which is the fastest way to confirm that the process you are looking at is the one you deployed.

Related