Tenants, API keys and quotas

The broker on its own has no users: anything that reaches its port can use it. The proxy compiled into the same binary adds them. With it on, every request presents an API key or a console session, and the proxy maps that credential to one tenant’s cluster, checks it against the plan’s limits, meters it, and hands the request to the broker inside the same process. The proxy keeps its own state (tenants, users, API-key hashes, plans and usage) as rows in the broker’s replicated KV, under a reserved system tenant, so every node of a cluster runs the same proxy and there is no database to give it.

Turn it on

Generate the secrets once and keep them in your secret store, because every node needs the same values:

SESSION_SECRET=$(openssl rand -hex 32)   # signs console sessions
CP_TOKEN=$(openssl rand -hex 32)         # opens the control plane, /api/cp/*
QUEEN_KEY="qk_live_$(openssl rand -base64 32 | tr '+/' '-_' | tr -d '=')"

docker run -d --name queen --platform linux/amd64 -p 6711:6711 -p 127.0.0.1:6632:6632 \
  -v queen-data:/var/lib/queen/raft \
  -e QUEEN_PROXY_EMBEDDED=true \
  -e QUEEN_PROXY_PORT=6711 \
  -e QUEEN_PROXY_JWT_SECRET="$SESSION_SECRET" \
  -e QUEEN_PROXY_CP_TOKEN="$CP_TOKEN" \
  -e QUEEN_PROXY_SPOOL_DIR=/var/lib/queen/raft/proxy-spool \
  -e QUEEN_PROXY_BOOTSTRAP_TENANT=acme \
  -e QUEEN_PROXY_BOOTSTRAP_API_KEY="$QUEEN_KEY" \
  -e QUEEN_PROXY_DEFAULT_CLUSTER=acme \
  ghcr.io/queen-mq/queen:latest

The boot log says where each part went:

INFO boot: single binary: proxy in-process on its own port; the broker router serves PORT (internal) proxy=0.0.0.0:6711 broker=0.0.0.0:6632
INFO proxy: bootstrap tenant ready tenant=acme created=true

A push through the proxy’s port now needs the key:

curl -s -X POST localhost:6711/api/v1/push -H "authorization: Bearer $QUEEN_KEY" \
  -H 'content-type: application/json' \
  -d '{"items":[{"queue":"orders","partition":"customer-123","payload":{"orderId":8891}}]}'

[{"index":0,"message_id":"01a0fcca-5436-7000-ab29-ef2bea95314f","transaction_id":"01a0fcca-5436-7000-ab29-ef2bea95314f","queueName":"orders","status":"queued","offset":0}]

Without it the answer is 401 {"code":"unauthorized","error":"missing bearer credential"}. Clients send the key the same way:

const queen = new Queen({ url: 'https://acme.queen.example.com', bearerToken: process.env.QUEEN_KEY })

The bootstrap runs at every boot on every node and changes nothing once its rows exist, so the same environment can stay on all nodes for good. It creates the tenant acme, a cluster of the same name on the dev plan (QUEEN_PROXY_BOOTSTRAP_PLAN), an admin user (QUEEN_PROXY_BOOTSTRAP_EMAIL, default admin@localhost, who can sign in with a password only if you set QUEEN_PROXY_BOOTSTRAP_PASSWORD), and a key with every scope on that cluster from QUEEN_PROXY_BOOTSTRAP_API_KEY. A key has to start with qk_, which is how the proxy tells an API key from a session token. Keep QUEEN_PROXY_SPOOL_DIR on the data volume; its default is relative to the working directory, which is read-only in a hardened container.

Two ports

QUEEN_PROXY_PORT decides what the broker’s own port, PORT (6632), still serves:

Run a cluster with the proxy on its own port. The nodes talk to each other over PORT (ephemeral partitions are forwarded and handed over there), and the membership and kill-switch routes under /api/v1/system/* are blocked through the proxy for every credential, so a cluster needs a broker port the proxy is not in front of. That port is for trusted clients inside the network, probes and scrapers, and it never faces the internet. TLS for the proxy’s port is QUEEN_TLS_CERT and QUEEN_TLS_KEY (security).

Tenants and clusters

A tenant is an organisation, and its users belong to it. A tenant owns clusters, and a cluster is what clients talk to: a slug, a plan, and a broker tenant of its own (a random UUID), so the queues, KV and timers of one cluster are invisible to every other.

The proxy finds a request’s cluster from the first DNS label of its Host, so acme.queen.example.com is cluster acme. A Host that names no cluster, like localhost, falls back to QUEEN_PROXY_DEFAULT_CLUSTER, and a host listed in QUEEN_PROXY_SHARED_HOSTS fronts many clusters and takes the cluster from the credential instead. Whatever x-queen-tenant header a client sends is dropped. A key is bound to its cluster for life, so presenting it on another cluster’s host fails:

{"code":"forbidden","error":"key/cluster mismatch"}

QUEEN_PROXY_TENANT_HEADER=false stops the proxy from scoping requests at all: every cluster lands in the broker’s default tenant, the one the internal port sees. That suits a single team that wants keys and a console in front of one shared broker, and nothing else, because the clusters then share every queue. The control-plane routes that act on a cluster’s broker data refuse to run in that mode (409 system_tenant).

A cluster’s status is active, push_blocked, suspended or deleting, a tenant’s is active, grace, suspended or deleting, and the worse of the two applies. push_blocked and grace refuse pushes and keep consumption open, so a customer can drain what is there; suspended and deleting answer the whole data plane 403 cluster_suspended.

API keys and roles

A key is qk_<env>_ followed by 43 base64url characters, and only its sha256 is stored. Each key carries a non-empty set of scopes, while humans hold one role per cluster:

A produce-only key that tries to list queues gets 403 {"code":"forbidden","error":"operation not permitted for this credential"}. A revoked key stops working on every node within about a second, because each node polls the replicated invalidation feed every QUEEN_PROXY_INVAL_POLL_MS (1000). API keys are refused on the console, which is for humans only. The complete classification of every broker route is at the end of this page.

Plans and quotas

Four plans are seeded at the first boot. A cluster takes its plan when it is created (free by default through the control plane, dev through the bootstrap), and per-cluster overrides change single limits:

curl -s -X PUT localhost:6711/api/cp/clusters/$CLUSTER_ID/overrides \
  -H "x-queen-cp-token: $CP_TOKEN" -H 'content-type: application/json' \
  -d '{"max_queues": 500, "max_retained_bytes": null}'

A null limit is unlimited, and a body of null clears every override.

The proxy starts in shadow mode (QUEEN_PROXY_ENFORCE=false): rate limits, queue and partition caps and parked-pop caps are computed and logged with target: limits and would_block=true, and the request goes through. That lets you watch a plan against real traffic before it bites. With QUEEN_PROXY_ENFORCE=true the same decisions answer 429 with Retry-After, or 403. Three limits apply in shadow mode too, because they bound what the proxy itself has to hold or store: the size caps (413 payload_too_large), the storage quota (max_retained_bytes, after which pushes answer 403 storage_quota_exceeded) and the monthly message quota (monthly_msgs_quota, unset on every seeded plan).

Metering

Every proxied request is metered per cluster, minute and class (push, delivery, txn, configure, read): requests, messages, and bytes in and out. Items the broker refused as error are not counted, and a duplicate is counted once. Each node writes its own rows every 15 seconds (QUEEN_PROXY_METER_FLUSH_MS), keeps the minutes for 90 days (QUEEN_PROXY_USAGE_KEEP_DAYS) and rolls them into days every hour. Usage a node cannot write during a long outage, or at shutdown, goes to QUEEN_PROXY_SPOOL_DIR and is written back at the next start. The control plane reads the last hour, summed over the nodes:

curl -s localhost:6711/api/cp/clusters/$CLUSTER_ID/usage -H "x-queen-cp-token: $CP_TOKEN"

{"minutes":[{"bytes_in":70,"bytes_out":173,"minute":"2026-10-02T13:25:00Z","msgs":1,"op":"push","reqs":1}]}

The console and sign-in

/console on the proxy’s port is for people: the cluster’s plan, limits and month-to-date usage, usage graphs, a read-only queue list, its API keys and, for admins, its members. The dashboard is served at / on the same port behind the same login. Sign-in is at /auth/login, with a local password (the bootstrap admin’s), Google (GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET, GOOGLE_ALLOWED_DOMAINS) or GitHub (GITHUB_CLIENT_ID, GITHUB_CLIENT_SECRET), and QUEEN_PROXY_PUBLIC_URL set so the callbacks have an address. An identity the proxy has never seen gets no account unless you turn on QUEEN_PROXY_AUTOPROVISION=true and name the tenant it joins with QUEEN_PROXY_AUTOPROVISION_TENANT; it then gets QUEEN_PROXY_DEFAULT_ROLE (viewer) on every cluster of that tenant.

Operators are the people who run the deployment itself:

QUEEN_PROXY_OPERATOR_ENABLED=true
QUEEN_PROXY_OPERATORS=ops@example.com,alice@example.com

An operator is admin on every cluster and can open the cell-wide pages: node status, host metrics, and the raft status and members behind the dashboard’s System page. At each boot the listed accounts are granted and every other operator is revoked, so the list in your deployment is the whole truth; an empty list revokes everyone, and leaving the variable unset leaves the flags as they are. No API route grants the flag, and with QUEEN_PROXY_OPERATOR_ENABLED off (the default) the operator routes answer 404 to everyone.

The control plane

Everything after the bootstrap can be driven over /api/cp/* on the proxy’s port, with x-queen-cp-token: $CP_TOKEN on every call. The token is checked before any body is read, bodies are capped at 1 MiB, and with QUEEN_PROXY_CP_TOKEN unset the whole surface answers 404. We designed it for a provisioning system that delivers its commands at least once, so every route is safe to repeat: a retry of a call that already took effect answers success with the same ids.

provision is the call to build on. You mint the key yourself and send only its sha256, so the plaintext never reaches the proxy, and you choose the user’s id, which is the subject of the sessions your own sign-in issues:

KEY="qk_live_$(openssl rand -base64 32 | tr '+/' '-_' | tr -d '=')"
HASH=$(printf %s "$KEY" | sha256sum | cut -d' ' -f1)
curl -s -X POST localhost:6711/api/cp/provision -H "x-queen-cp-token: $CP_TOKEN" \
  -H 'content-type: application/json' -d '{
    "tenant_slug": "globex", "cluster_slug": "globex", "plan": "pro",
    "user_id": "308d3f3d-2306-41de-95f1-e32b12ec7470", "email": "ops@globex.example",
    "role": "admin", "key_name": "default", "key_hash": "'"$HASH"'",
    "scopes": ["produce", "consume", "read"], "overrides": {"max_queues": 500}}'

{
  "broker_tenant_uuid": "ff2ab01d-c30e-45e3-960b-5bd7deecd6f5",
  "cluster_id": "0acf78a8-567a-4a9b-b01d-89be075fe56a",
  "created": { "cluster": true, "key": true, "tenant": true, "user": true },
  "key_id": "76648399-b1b0-469e-ba7a-a2ebfb2466da",
  "overrides": { "max_queues": 500 },
  "plan_code": "pro",
  "tenant_id": "b52683ee-7a9b-4e5d-b831-c4806931d866",
  "user_id": "308d3f3d-2306-41de-95f1-e32b12ec7470"
}

Sent again, the same call answers the same ids with every created flag false. Taking a tenant down is three steps, and the proxy refuses them out of order. The delete answers 409 not_deleting while the tenant is active, and 409 not_purged while its broker still lists a queue; the purge is done only on a pass that finds nothing left to delete, which is why the first pass below answers done: false:

curl -s -X PUT localhost:6711/api/cp/tenants/globex/status -H "x-queen-cp-token: $CP_TOKEN" \
  -H 'content-type: application/json' -d '{"status":"deleting"}'
curl -s -X POST localhost:6711/api/cp/tenants/globex/purge -H "x-queen-cp-token: $CP_TOKEN"
curl -s -X POST localhost:6711/api/cp/tenants/globex/purge -H "x-queen-cp-token: $CP_TOKEN"
curl -s -X DELETE localhost:6711/api/cp/tenants/globex -H "x-queen-cp-token: $CP_TOKEN"

{"clusters":[{"broker_tenant_uuid":"ff2ab01d-c30e-45e3-960b-5bd7deecd6f5","id":"0acf78a8-567a-4a9b-b01d-89be075fe56a","slug":"globex"}],"ok":true,"tenant_id":"b52683ee-7a9b-4e5d-b831-c4806931d866"}
{"clusters":[{"broker_tenant_uuid":"ff2ab01d-c30e-45e3-960b-5bd7deecd6f5","done":false,"partitions_deleted":1,"slug":"globex"}],"done":false,"existed":true}
{"clusters":[{"broker_tenant_uuid":"ff2ab01d-c30e-45e3-960b-5bd7deecd6f5","done":true,"partitions_deleted":0,"slug":"globex"}],"done":true,"existed":true}
{"clusters":[{"broker_tenant_uuid":"ff2ab01d-c30e-45e3-960b-5bd7deecd6f5","cluster_id":"0acf78a8-567a-4a9b-b01d-89be075fe56a","slug":"globex"}],"existed":true}

From the moment the status is deleting, the tenant’s own traffic answers 403 cluster_suspended. The control plane never acts on the broker’s default tenant or on the proxy’s own system tenant, whatever a request names.

Limits

Rate buckets and parked-pop gauges live in each node’s memory, so behind a load balancer over three nodes a cluster can reach three times its plan’s rate. The storage quota is read from the broker’s own measurement, which every node shares.

Ephemeral queues behind the proxy are safe on a single node only, for now. In a cluster, an ephemeral request that the receiving node forwards to the partition’s owner does not keep its tenant and lands in the broker’s default tenant; durable queues are isolated as described above. Separately, an ephemeral hand-over to a node that is still restarting can drop that partition’s messages, which we have seen during rolling restarts (ephemeral queues).

The proxy does not isolate tenants from the people who run it. A live operator, and anyone who holds QUEEN_PROXY_CP_TOKEN, can act on every cluster, and the broker port behind the proxy answers whatever reaches it under the broker’s own JWT setting (security).

No route edits a plan or moves a cluster to another deployment; overrides are how one cluster’s limits change. When the replicated KV cannot answer (a cluster without a majority), a key the proxy has recently checked keeps authenticating for up to 10 more minutes (QUEEN_PROXY_STALE_GRACE_MS), while a key it has not seen is refused.

Route classes

The proxy classifies every broker-bound request into exactly one class, and anything API-shaped that matches no rule is blocked. The table below applies that classifier to the broker’s own route list.

produce

consume

queue admin

read

gated (streams)

gated (traces)

gated (kv)

gated (timers)

gated (ephemeral)

operator

blocked

81 of the broker’s 109 method + path pairs are reachable with a tenant credential; the rest are operator or blocked surfaces.