Open a Queen database with psql today and you find the log engine, the shared management tables,
and nothing else. This page is the record of how it got that way: which engines came before, what
each left behind, and when the leftovers were removed.
One fact frames everything below: deployments are always-virgin. A database is born from
schema.sql plus the 23 procedure files, all of which only create. Nothing is dropped, migrated
or folded at boot, because there is never anything pre-existing to drop. There is no upgrade path;
pointing this build at a database written by an earlier schema is unsupported (fresh database, full
stop). Every removal described here therefore happened in the source tree, not in your database.
The story, in five stages
- The rows engine: one row per message in
queen.messages, coordination inqueen.partition_consumers. The original engine. - The
seg_*engine: the first segment-based engine, addressing positions as(seq, frame_idx)pairs. It folded its cursor columns onto the rows engine’s coordination table rather than owning its own. - The log engine: the current engine, with single per-partition
BIGINToffsets, segments inqueen.log_segments, cursors inqueen.log_consumers. A greenfield replacement for theseg_*engine, with no data migration by design. - Single engine (2026-07-30): the rows engine was removed entirely, first its message plane, then its tables and every surviving reference to them.
- Single queue identity (2026-07-31): the log engine’s own queue table,
queen.log_queues, was merged intoqueen.queues. Queue identity isqueen.queues(id);log_partitions.queue_idreferences it directly, and the(tenant_id, name)bridge joins between config and data are gone, taking with them the cross-tenant name-join hazard class. In the same change the tree dropped its teardown and migration files: the schema apply now only creates.
What was removed, and when
2026-07-30: the rows engine
Its message plane, meaning push, the four generations of pop, ack, transaction, lease renewal,
the pending probe, the two partition_lookup maintainers and streams_cycle_v1. A call-graph
closure over server/src and the surviving SQL found zero callers for all sixteen functions;
the only references left in the tree were comments. Eleven files and 4,662 lines went.
Its tables and its remaining branches. queen.messages, queen.partitions,
queen.partition_consumers, queen.dead_letter_queue, queen.messages_consumed and
queen.partition_lookup were deleted, and every surviving procedure was rewritten to stop
referencing them. That was 49 references across 13 files: 6 unreachable storage = 'rows'
branches, 36 joins over tables that were permanently empty, and 6 genuine defects where the
empty-or-phantom table changed a number somebody read.
2026-07-31: the second queue table, and the teardown files themselves
queen.log_queues held the log engine’s queue rows while queen.queues held the
configuration, and every join between them went through (tenant_id, name). The merge folded
the two into one table: queen.queues absorbed dedup_window_seconds and took lease_time’s
real default (60, the value the pop path had always used while the config display showed 300,
a display lie rather than a behaviour change). The storage column was dropped outright: one engine,
nothing to select, and the wire keeps echoing storage: "segments" as a hard-coded literal.
consumer_watermarks, consumer_groups_metadata and queue_lag_metrics were re-keyed from
queue names to queen.queues(id) with cascading FKs.
The same change deleted the boot-time teardown machinery, because always-virgin deployments made it dead weight:
- the
seg_*teardown file (log_drop_legacy), which had swept away everyseg_%function and table at every boot; - the two rows-engine drop files (
drop_rows_message_planeanddrop_rows_tables), which had removed the retired functions and tables from databases that pre-dated their deletion; - the one-off queue-identity merge migration, which had folded old-shape databases into the merged form.
None of them ever did anything on a fresh database, and a fresh database is now the only kind
there is. The remaining procedure files were renumbered into the contiguous 001–023 roster
documented in the schema catalogue.
The six defects the rows tables were still causing
Five of them traced back to one line: configure_queue_v1 inserted a Default row into
queen.partitions on every /configure, for a partition the engine never used. It was the only
writer that table had.
| Surface | Reported | Truth |
|---|---|---|
GET /resources/namespaces and /resources/tasks, field partitions |
the count of /configure-created rows, so 0 for any queue created by push |
the live log-partition count |
queue_lag_metrics.partitions_created, plotted as “Partitions Created” |
one event per /configure call |
partitions actually allocated by push |
queue_lag_metrics.partitions_deleted |
never moved | partitions reclaimed by the retention cleanup |
/configure response, partitionId |
a uuid resolving to nothing | null, because partitions are created by the first push |
queen_dlq_depth (Prometheus) |
no sample at all, because it read the empty rows DLQ | the live queen.log_dlq depth |
The last one is the one to remember: DLQ alerting could never fire, while GET /api/v1/dlq listed
the same rows correctly the whole time.
Tables the log engine uses
Everything in schema.sql today is either the log engine’s or genuinely shared: queen.queues
(queue identity and configuration, since the merge), consumer_groups_metadata,
consumer_watermarks, stats, stats_history, retention_history, message_traces,
message_trace_names, system_state, system_metrics, the worker-metrics tables and
queue_lag_metrics. Do not assume that anything defined there is retired: nothing is.
Three of them keep a partition_id column that carries a queen.log_partitions id with no
foreign key: stats, retention_history and message_traces. The constraint-less shape is
deliberate, and since the merge it is simply declared that way rather than arrived at by
dropping anything: each of those rows must be able to outlive its partition. A retention audit
row is written by the very step that deletes the partition, and a trace must outlive the segment
it describes. (stats.queue_id, by contrast, does carry a cascading FK to queen.queues(id):
a queue’s stats should die with the queue.)
Why queen.stats reports engine “segments”
Because "segments" is a public compatibility literal, not a description of anything stored.
No table carries a storage or engine column any more; the value is hard-coded at the wire:
POST /api/v1/configureaccepts and returnsstorage: "segments". That is part of the wire contract and did not change when the engines did (012_configureemits the literal).- The queue listings and detail readers echo the same literal wherever the shape expects a
storage value (
010_log_admin,011_log_stats). log_refresh_all_stats_v1labels its refresh summaryengine: "segments", andstats.rsprints that summary verbatim.
So “segments” in a stats row, a summary line, or a /configure response means “the engine”.
Read it as a frozen wire token, not as a version. The tables actually read are log_partitions,
log_consumers, log_segments and log_dlq.
The migrate subcommand
queen migrate ... still exists as a CLI entry point and is a stub. It logs that migration is
not supported on the log engine and exits non-zero without opening a database connection. The
rows-to-segments migration it used to run was built on SQL that was deleted with the seg_*
engine.
There is no rows-to-log and no pre-merge-to-merged migration on this tree. The broker boots its own schema, from empty.
What to do with any of this
Nothing, in normal operation. A deployed database carries no trace of any retired engine: not a dropped column, not a compatibility view, not a teardown file waiting for something to delete, because it never contained one. Every deployment starts virgin on the final shape. Two rules survive as practical guidance:
- When reading the database, start from the
log_*tables. They are the only place the engine keeps data; everything else is shared infrastructure (configuration, stats, metrics, traces). - Never point this build at a database created by an older build. There is no upgrade path and no boot-time cleanup any more; the supported move is a fresh database.