KV holds the state a step changes, in the same replicated log as the messages, so a state change can commit in the same entry as the ack that caused it. Keep an entity’s current state there, the markers that make a step idempotent, counters and rate limits. The point is that you no longer need a second database to stay consistent with the broker: there is only one commit.
await queen.kv.put('orders', '9137', { status: 'paid' }, { ttl: '30d' })
const row = await queen.kv.get('orders', '9137') // { found, value, version, expiresAt, updatedAt }
const r = await queen.kv.put('orders', '9137', { status: 'shipped' },
{ ttl: '30d', expect: row.version }) // compare-and-swap
if (!r.applied) console.log(r.reason, r.value) // someone wrote first
await queen.kv.incr('limits', `api:${customerId}:${minute}`, 1, { max: 100, ttl: '2m' })Over HTTP, one call to POST /api/v1/kv carries a list of operations, and the answer is HTTP 200
with one result per operation, in order:
curl -s -X POST http://localhost:6632/api/v1/kv -H 'content-type: application/json' -d '{"operations":[
{"op":"put","ns":"orders","key":"9137","value":{"status":"paid"},"ttlSeconds":2592000},
{"op":"get","ns":"orders","key":"9137"}]}'A single key also has a path of its own, GET, PUT and DELETE /api/v1/kv/:ns/:key, which the
quickstart uses to read back a value. An absent key or a lost race is a field
of the result, never an error status.
Operations
| Op | Does |
|---|---|
get |
One key: found, value, version, expiresAt, updatedAt. null is a value: found: true, value: null is not absence. |
getMany |
Keys of one namespace: rows, in key order, and an explicit missing list. |
getPrefix |
One page of keys under a non-empty prefix: limit (default 100, at most 1,000), after (a keyset cursor), nextAfter. |
put |
Writes a value with a new lifetime. It does not keep the previous TTL. |
putIfAbsent |
put with expect: 0. Of several concurrent callers, one wins; the others get the winner’s value and version. |
delete |
Removes a key, optionally with expect. |
incr |
Adds delta without a compare-and-swap loop. With min or max, an increment past the bound does not apply (reason: "limit"). The TTL is set when the counter is created; an expired counter counts from zero. |
Read applied on every write. The JS SDK returns an object for each write, and an object is
always truthy, so if (await queen.kv.put(...)) tells you nothing.
Every write declares its lifetime
Every put, putIfAbsent and incr names exactly one of ttlSeconds (an integer above 0) or
forever: true. Neither, or both, is a 400 ("error": "kv_bad_request", with
"reason": "kv_expiry_not_specified"). We made the lifetime mandatory because state written by
messages tends to outlive the reason it was written, and without an expiry it would stay on every
node until somebody cleaned it up by hand. The JS SDK also takes ttl: '30d' or until: date. An expired key is never returned and
never counts as existing, even before the sweep removes it.
Versions and expect
Every write gives the key a new version. Versions are unique and never reused, but don’t rely on
their order. expect: 0 means the key must not exist; expect: N means it must be at version N,
and writes nothing when the key is absent. A lost precondition answers applied: false with a
reason (exists, absent or version) and the key’s current value and version, so you can see
who won without a second read.
The other operations of the call still apply. Add required: true to an operation and a lost
precondition aborts the whole call instead: it answers HTTP 200 with ok: false,
reason: "kv_precondition", the failedIndex and the kvReason.
Consistency
- A read of one key is linearizable. A read-only call first waits until this node has applied everything the cluster had committed, so it sees a write that another node answered.
- A call with writes is one log entry, and its reads see its own writes.
- One call over several keys is strict serializable:
putIfAbsenthas exactly one winner, and anincrthe broker commits is counted once. These were Jepsen-tested. - An answer you never got is a different matter. If a write call times out (503, outcome unknown)
and you send it again, a repeated
putonly writes the same value again, but a repeatedincradds twice. The SDKs retry a failed request on their own (three attempts by default), so give a counter that must be exact aputIfAbsentmarker withrequired: truein the same call.
Inside a transaction
queen.transaction().kv.put(...) commits the write with the ack, and when the ack carries its
lease, an expired lease rolls the write back with everything else (see
transactions). That is something a compare-and-swap alone cannot do: an
expect on a version that still matches succeeds even for a worker that lost its message long ago.
A transaction may also read with get and getMany, because you bound their cost, but the reads
are answered after the entry has applied, so they cannot be a condition for the writes beside them:
use expect with required: true for that. getPrefix is refused there
(kv_get_prefix_not_allowed_in_transaction), because its work has no bound inside a commit. A
transaction holds at most 64 KV operations over 256 keys.
What KV is for
KV is the state a step needs next to its ack, and we kept it small on purpose. There are no
queries, secondary indexes or joins: applications read by key, by a list of keys or by prefix, so
records you search belong in your database. A putIfAbsent with a TTL is no lock either. It
expires, nobody revokes it, and a holder that outlived it keeps working, so fence later writes
with expect on the version you won, or keep the work inside a transaction whose ack carries the
lease. Every value travels through the replicated log and is stored on every node, which is the
price of committing it with the ack. And there is no way to watch a key: when a change matters to
someone, push them a message in the same transaction.
Limits
- A namespace matches
^[a-z0-9][a-z0-9._-]{0,63}$and belongs to one tenant. A key is non-empty, has no NUL byte and is at most 512 bytes (less with a long tenant and namespace). - A value is any JSON up to 64 KiB, measured compact (
QUEEN_KV_MAX_VALUE_BYTES). The node that receives a call checks the ceiling, so set it the same on every node or the nodes will disagree about what they accept. Above it, the body and entry limits still apply. - One call to
/api/v1/kvcarries at most 256 operations and 1,024 keys (QUEEN_KV_MAX_KEYS_PER_CALL, at most 4,096; agetPrefixcounts as itslimit), one write per key, and reads at most 4 MiB of values (past it,truncated: true). - Per tenant and per node,
/api/v1/kvtakes 100 write calls a second (burst 200) and 200 read calls a second (burst 400) by default (QUEEN_KV_WRITE_RATE,QUEEN_KV_READ_RATE). A call with any write in it counts as a write, and timer schedules spend from the same bucket. Over it you get 429 withRetry-After. KV inside a transaction is not rate-limited. - Each
getPrefixpage is its own snapshot: good for walking state, not for an exact count.