KV internals

queen.kv is one table, one sequence, two small side tables and a set of pure helpers. The product value is not the store. It is that the idempotency marker, the effect and the cursor advance commit in one PostgreSQL transaction: a key/value store standing beside the broker cannot give that at any price, however fast it is. Everything on this page exists so that the shape rules are written in SQL exactly once, and the seven clients, the HTTP routes, the transaction wire and the embedded broker inherit them without a per-language copy.

The table

CREATE TABLE IF NOT EXISTS queen.kv (
    tenant_id  UUID NOT NULL DEFAULT '00000000-0000-0000-0000-000000000001',
    namespace  TEXT COLLATE "C" NOT NULL,
    key        TEXT COLLATE "C" NOT NULL,
    value      JSONB NOT NULL,
    version    BIGINT NOT NULL,
    expires_at TIMESTAMPTZ,
    shard      SMALLINT NOT NULL
               GENERATED ALWAYS AS ((hashtextextended(key, 0) & 63)::smallint) STORED,
    created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
    PRIMARY KEY (tenant_id, namespace, key)
);

The identity is (tenant, namespace, key) and nothing else. There is deliberately no partition_id, unlike queen_streams.state, because a KV key is a business identity and the caller does not know, and must not need to know, which partition it falls on.

That omission has an exact price, and it is the reason two of the seven operations exist. The streams state model gets serialisation for free: the operator holds an exclusive lease on the partition, so nothing else writes those rows. This table has no lease. Two workers can hit the same key at the same instant. putIfAbsent and incr are therefore mandatory primitives rather than conveniences, and a read-modify-write performed by the caller across two round trips is unsound unless the key derives from the partition key the caller already owns.

The omission is also what keeps the table out of the engine’s maintenance costs. Nothing here names a partition, so queen.log_partition_dead_v1 gains no veto leg and its per-partition scan does not get more expensive. That invariant has a free mechanical check, and it is the reason for the file number: log_partition_dead_v1 is LANGUAGE sql, so it resolves its tables at creation time, and it lives in 006 while this table lives in 024. Adding a leg that names queen.kv kills the boot. The file number is the verification.

COLLATE “C” is load-bearing twice, and the first reason is not the obvious one

Both name columns carry an explicit COLLATE "C". Before this file, COLLATE appeared nowhere in the schema, so nothing about it could be assumed from precedent.

1. It is the foundation of the lock order. Several actors take more than one row in this space, so they take them in ascending (namespace, key) order. Under the database default collation, two pods with different lc_collate, or two ICU minor versions, which is the real case during a rolling base-image upgrade, order two keys in opposite directions. Two overlapping bundles then take their rows in inverted order and deadlock with 40P01 on some installations only and for some keys only, which is the worst shape a deadlock can have.
2. Byte order is stable across machines, so a prefix scan is sargable and a keyset cursor keeps its meaning across a libc or ICU upgrade. A cursor whose meaning depends on the host locale is a corruption waiting for a minor version bump.

Two consequences follow, and both are written into the SQL where they can be seen.

Prefix results come back in byte order, so non-ASCII keys are not in locale-alphabetical order. And an extraction from JSONB does not inherit a column’s collation: every ORDER BY over op->>'key' spells COLLATE "C" out. The broker never pre-sorts either. Ordering is a property of the stored procedure, in one place, because one differing libc between two pods is all it takes to reopen the cycle if the Rust starts helping.

The version is a token, not a counter

version is drawn from queen.kv_version_seq, a BIGINT sequence with CACHE 1000, and callers may compare it for equality only. Never arithmetic, never an ordering comparison.

It is not a write count and it is not monotonic. With a cache, each backend draws a block, so backend A can emit 91005 in real time after backend B emitted 92000 for the same key. Gaps are expected.

It comes from a sequence rather than from version + 1 for a reason that only shows up after an expiry: a key that expired, was pruned and was written again must not be able to re-issue a version that some long-lived holder is still carrying. A per-lineage counter can do exactly that, and the compare-and-set that follows succeeds against a lineage that no longer exists.

The large CACHE exists to keep a nextval on the hot path from writing a WAL record per write. It is not a defence against the sequence side channel: with a connection pool, a backend’s block is consumed by interleaved tenants, so the channel becomes coarser rather than closed.

kv_live_v1 is the only definition of existence

CREATE OR REPLACE FUNCTION queen.kv_live_v1(TIMESTAMPTZ, TIMESTAMPTZ)
RETURNS BOOLEAN LANGUAGE sql IMMUTABLE PARALLEL SAFE
AS $$ SELECT $1 IS NULL OR $1 > $2 $$;

Every read, every precondition and every write path routes its notion of “this key exists” through that one predicate. An expired row is invisible from the first read after its instant, whether or not anything has deleted it, so the prune is a disk-space job and never a correctness job.

kv_ver_v1 builds on it and is total: the effective version of an absent row, and of an expired one, is 0. That is what lets expect: 0 mean “must not exist” with no magic flag, and it is why a compare-and-set against a dead lineage cannot succeed.

Both helpers take now as a parameter rather than calling clock_timestamp() inside themselves, so one call to kv_apply_v1 uses one instant for its whole batch. A getMany can never see one key alive and the next dead a few microseconds later. For the same reason none of the pure helpers is declared STRICT: a STRICT declaration turns a documented total function into NULL on its single most important input, an absent row or a forever key, and a NULL inside a WHERE is a row that has silently vanished.

One secondary index, and the ones deliberately not created

CREATE INDEX IF NOT EXISTS idx_kv_shard_expires
    ON queen.kv (shard, expires_at) WHERE expires_at IS NOT NULL;

It has exactly one reader, the sweeper. It is partial, so a key written with forever costs nothing in it, and shard leads because the sweeper filters by shard and then range-scans expires_at.

Not created, with the reason recorded next to each: an index on updated_at, which queen_streams.state carries and no reader queries, and which would make every counter update non-HOT; and an index on value, which has no query to serve because the declared read boundary is keys and prefixes.

Quota and usage are two tables because they have two authors

queen.kv_quota is configuration, written by an operator. queen.kv_usage is measurement, overwritten by the sweeper on its slow sub-cadence. Merging them would leave “who wrote this number?” without an answer. Both are keyed by tenant_id alone: the decided model is that any broker may measure any shard, so a per-shard key would produce write-write contention between brokers and multiply the scan cost, and last-writer-wins on computed_at is the intended semantics.

No write path touches either table. A per-tenant counter row updated by every write would be one row held for the whole duration of a bundle in the outermost of the six lock spaces. That is not a deadlock, it is worse: a throughput ceiling of one over the bundle duration, for every caller of that tenant. Enforcement is an in-process delta held in the broker’s state, and it emits no SQL on the write path, which is why the quota is soft and late by construction.

Numbers on this page that are not measured

The dated blocks in 001_log_schema.sql are the house formula: a storage setting carries the date it was measured, the symptom that produced it, and the number that was observed. The settings below carry no such block, because they were chosen by analogy and the measurement has not been done. They are written here rather than left to be inferred from the DDL.

Each is to be replaced by a measured number after the first soak on a test rig, in the style of those dated blocks, and never by a number taken from a production cluster’s live stack.