Kafka facade settings, topic configs and API versions, where it differs from Apache Kafka, and what Java, librdkafka, franz-go, sarama, kafkajs and aiokafka clients need.

This is the contract a Kafka client gets from Queen, in the form you check things against: the settings the facade reads, what it does with topic configs, the APIs and versions it answers, the places where it knowingly behaves differently from Apache Kafka, and what each client library needs. How it all fits together, with a working example, is on Kafka clients.

QUEEN_KAFKA_EMBEDDED=true
QUEEN_KAFKA_ADVERTISED_ADDR=kafka-1.example.internal:9092   # required: what clients dial for this node

Settings

Read once, when the node starts. An empty value counts as unset, and a value that does not parse or is out of range stops the node with the variable’s name, except for QUEEN_KAFKA_THREADS and QUEEN_KAFKA_SHUTDOWN_GRACE_MS, which fall back to their defaults.

Variable	Default	What it does
`QUEEN_KAFKA_EMBEDDED`	`false`	`true` runs the facade inside the broker. Nothing below is read otherwise.
`QUEEN_KAFKA_ADDR`	`0.0.0.0:9092`	The address the Kafka listener binds.
`QUEEN_KAFKA_ADVERTISED_ADDR`	none, required	The `host:port` clients dial for this node. A value without a port is refused at boot.
`QUEEN_KAFKA_PEERS`	unset	The address the other nodes are advertised at, `1=host:port,2=host:port`, by raft node id. Unset, a node is advertised at the host of its raft address with this node’s Kafka port, which is the shape of a StatefulSet (one headless name per pod, one port everywhere).
`QUEEN_KAFKA_THREADS`	`min(4, cores / 2)`, at least 1	Threads of the facade’s own runtime.
`QUEEN_KAFKA_DEFAULT_PARTITIONS`	`1024`	The width of a topic created without one, 1 to 100,000.
`QUEEN_KAFKA_AUTO_CREATE`	`true`	Metadata creates a topic it does not know when the client allows it.
`QUEEN_KAFKA_DEDUP_WINDOW_SECONDS`	`900`	The dedup window of a topic the facade creates, and of a topic without one when an idempotent producer first writes to it, 0 to 604,800. A producer retries a batch for at most `delivery.timeout.ms` (2 minutes by default), so a longer window catches no retry. `0` leaves every topic as it is.
`QUEEN_KAFKA_GROUP_JOIN_DELAY_MS`	`3000`	How long the first join of an empty group waits for more members (Kafka’s `group.initial.rebalance.delay.ms`), 0 to 300,000.
`QUEEN_KAFKA_SASL`	unset	`plain` (or `required`) makes every connection log in with SASL/PLAIN before it may send anything but ApiVersions and the SASL handshake (security). `off`, or unset: no login. Any other value stops the node.
`QUEEN_KAFKA_MAX_CONNECTIONS`	`10000`	Connections served at once, 1 to 1,000,000. Past it a new connection is closed, with a line in the node’s log, and the client tries again.
`QUEEN_KAFKA_MAX_REQUEST_BYTES`	`104857600` (100 MiB)	The largest request frame, 1 MiB to 1 GiB (Kafka’s `socket.request.max.bytes`). A larger one closes the connection.
`QUEEN_FETCH_TOPIC_WAKE_MS`	`5`	How long a parked fetch that has read nothing waits at least before it answers, empty, because another partition of its topic was written on the node, 0 to 30,000. `0` parks it on its own partitions only. Read by the broker, so `POST /api/v1/fetch` behaves the same.
`QUEEN_KAFKA_SHUTDOWN_GRACE_MS`	`5000`	How long a stopping node waits for the facade to close.

Topic configs

A topic is a Queen queue. Of the configs a client sends with CreateTopics, two become the queue’s settings, as they did before 2.6; the others are accepted and not applied yet.

Config	Accepted	What it does
`retention.ms`	`-1` (default), or 1000 and more	The queue’s retention (`retentionSeconds`), rounded down to whole seconds. `-1` keeps every record. Less than a second is refused with `INVALID_CONFIG`, because it would round to nothing.
`cleanup.policy`	`delete` (default), `compact`, `compact,delete`	`delete` lets `retention.ms` delete by age. `compact` keeps every record whatever `retention.ms` says, because Queen does not compact and the last value of every key has to stay. `compact,delete` deletes by age and compacts nothing. Anything else is refused with `INVALID_CONFIG`.
Anything else		Accepted, not applied.

A topic created on first use, by a metadata request, has no configs and keeps every record. The facade also sets the queue’s dedup window to QUEEN_KAFKA_DEDUP_WINDOW_SECONDS. The configs are read at creation only: AlterConfigs and IncrementalAlterConfigs are not offered in 2.6, so change a topic’s retention afterwards through Queen (queue options), and DescribeConfigs answers no entries.

APIs and versions

Every Kafka client opens a connection by asking which APIs and versions the broker speaks, and then speaks only those. This is that answer, with the reason each window stops where it does.

The facade advertises 26 Kafka APIs. Every row is read out of protocols/queen-kafka/src/versions.rs when the site is built: the table the ApiVersions answer is made from, and the one every incoming request is checked against.

API	Versions	Where the window ends, and why
`AddOffsetsToTxn`	v0–v3	v3 answers everything v4 does. v4 exists for KIP-890’s transaction protocol 2, in which clients stop sending this request.
`AddPartitionsToTxn`	v0–v3	Ceiling: v4 is a different request, KIP-890’s broker-to-broker verification, which no client sends.
`ApiVersions`	v0–v3	Ceiling: one below the schema, on purpose. v3 is what clients negotiate with a Kafka 3.x broker, and a client that opens at v4 (Java 4.x does) is answered with this window and retries at v3, as the protocol specifies.
`CreatePartitions`	v0–v3	The whole window. An increase raises the width of a topic, and the next Metadata reports it; a decrease or an equal count is refused with Kafka’s own sentences, and so is a manual replica assignment.
`CreateTopics`	v2–v6	Ceiling: v7 returns a topic id, which the facade does not mint. A partition count of -1 (v4) takes the default width, and v5 answers the width and the replication factor (the number of raft voters) with no configs. `retention.ms` and `cleanup.policy` become the queue’s retention; the other configs are accepted and not applied yet.
`DeleteGroups`	v0–v2	The whole schema; v2 only changes the encoding. Deleting a group with no members deletes the Queen consumer group of that name, with its position on every queue.
`DeleteTopics`	v1–v5	Ceiling: v6 accepts a topic id in place of a name. v5 adds `error_message`, which says in words that there is no such queue.
`DescribeConfigs`	v1–v4	Up to the schema’s v4. It answers no entries, the retention a topic was created with included (that is read in Queen), rather than values nothing enforces; an unknown topic answers UNKNOWN_TOPIC_OR_PARTITION.
`DescribeGroups`	v0–v3	Ceiling: v4 carries `group_instance_id`. v3’s authorized operations are answered with Kafka’s own ‘omitted’ value, because there is no ACL model.
`EndTxn`	v0–v3	Ceiling: v5 returns a new producer epoch from inside EndTxn (KIP-890’s transaction protocol 2), which the facade does not do. A commit writes the transaction’s records and offsets in one raft entry; an abort drops them, so no uncommitted record ever enters the log.
`Fetch`	v4–v6	Ceiling: v7 introduces fetch sessions (KIP-227), state a broker keeps per connection. The facade keeps none, so a request names every partition it wants every time. The raft leader serves it, because a follower can be behind offsets a consumer already read. One side effect: librdkafka compresses zstd only for brokers at Fetch v10 or later, so librdkafka-based producers send their zstd batches uncompressed here.
`FindCoordinator`	v0–v3	Ceiling: v4 is the batched form, for clusters where groups live on different brokers. Here the raft leader coordinates every group and every transactional id, and every answer names it.
`Heartbeat`	v0–v2	Ceiling: v3 carries `group_instance_id`.
`InitProducerId`	v0–v4	Ceiling: v5 exists for KIP-890’s transaction protocol 2, which the facade does not run. v3 matters: it is KIP-360’s epoch bump, which lets a Java producer recover from a gap in its sequence instead of failing. A transactional id keeps its producer id across restarts, at the next epoch, from the raft leader.
`JoinGroup`	v0–v4	Ceiling: v5 carries `group_instance_id`, and static membership is not supported. A member that joins without an id is given one in the same answer, with no MEMBER_ID_REQUIRED round trip.
`LeaveGroup`	v0–v2	Ceiling: v3 carries `group_instance_id` and lets one request remove several members.
`ListGroups`	v0–v4	Ceiling: v5 adds `group_type`, which tells classic groups from KIP-848 ones, and KIP-848 is not supported. v4’s state filter (`--state`) is honoured.
`ListOffsets`	v1–v5	Ceiling: v7 adds MAX_TIMESTAMP, a question about the records’ own timestamps, which the broker does not read. Up to v5 there are three questions (earliest, latest, a concrete time) and the raft leader answers all three; a time is looked up on the broker’s append clock.
`Metadata`	v0–v9	Ceiling: v10 adds topic ids, and the facade has no registry to resolve a topic by id. v9 is already the flexible encoding and carries every field a client reads. Every raft voter is listed as a broker and as an in-sync replica of every partition, and the raft leader as the leader of them all.
`OffsetCommit`	v2–v6	Ceiling: v7 carries `group_instance_id`, and static membership is not supported. Floor: v0 and v1 are the ZooKeeper-era offset store.
`OffsetFetch`	v1–v7	Ceiling: v8 asks about several groups in one request and changes the response shape. v7’s `require_stable` costs nothing: a transaction’s offsets are written at its commit, in the same raft entry as its records, so every offset returned is already stable.
`Produce`	v3–v9	Floor: v3 is the first version that carries RecordBatch v2, the format every client of the last decade sends. Ceiling: v10 adds a leader-change hint, and v13 names topics by id. Records go to the raft leader, which leads every partition; another node answers NOT_LEADER_OR_FOLLOWER and the client refreshes its metadata.
`SaslAuthenticate`	v0–v1	Ceiling: v2 only changes the encoding. v1’s `session_lifetime_ms` is answered 0, so no client re-authenticates on a timer.
`SaslHandshake`	v1	v1 only: the handshake is followed by SaslAuthenticate requests, the framed flow current clients use. PLAIN is the one mechanism, and its password is a Queen token.
`SyncGroup`	v0–v2	Ceiling: v3 carries `group_instance_id`.
`TxnOffsetCommit`	v0–v3	v3 has to be in the window: kafka-clients refuses to build this request below v3 when group metadata is set, which every consume-transform-produce loop does. Ceiling: KIP-890’s transaction protocol 2.

Not offered

A client that sends one of these gets no answer: the connection closes, which is what Apache Kafka does with a request it cannot parse (the node logs the key at debug level). A client that read the ApiVersions answer never sends one.

API	What a client wants it for	Why it is not here
`ConsumerGroupHeartbeat`	The KIP-848 consumer group protocol (`group.protocol=consumer`).	Not supported. Groups use the classic protocol, which every Kafka 4.x client still uses by default; a client set to `consumer` fails at once with an error that names `group.protocol=classic`.
`DeleteRecords`	Truncating a partition below an offset: `kafka-delete-records.sh`, the clear-messages buttons of kafka-ui and AKHQ, and Kafka Streams purging its repartition topics.	Queen has no trim to an offset yet: a partition’s start moves by retention and when the queue is deleted. Answering would report a low watermark that did not move. Delete and recreate the topic instead.
`OffsetForLeaderEpoch`	Detecting log truncation after a leader change.	Clients check ApiVersions first and skip that validation for a broker without the key. Raft never truncates an acknowledged entry.
`DescribeAcls`	Listing ACLs (`kafka-acls.sh --list`).	There is no ACL model: authorization is Queen’s, on the token.
`CreateAcls`	Creating ACLs.	Same reason.
`DeleteAcls`	Deleting ACLs.	Same reason.
`AlterConfigs`	Replacing a topic’s configuration (the deprecated full-replacement form).	A topic’s retention is set when it is created (`retention.ms`, `cleanup.policy`) and changed through Queen afterwards; its other configs are not applied, so a change would report a setting that nothing enforces.
`IncrementalAlterConfigs`	Changing a topic’s configuration (`kafka-configs.sh --alter`).	Same reason.
`DescribeLogDirs`	Log directories and the bytes each partition holds (`kafka-log-dirs.sh`). Some topic creators also wait on it until every replica has its log.	Not offered yet. A tool that needs it reports the broker as too old; to wait for a new topic, read Metadata (leaders and in-sync replicas) instead.
`OffsetDelete`	Deleting a group’s committed offsets for some partitions (`kafka-consumer-groups.sh --delete-offsets`).	Not offered yet: a group’s positions go with the group (DeleteGroups).
`CreateDelegationToken`	A broker-signed token derived from a SCRAM login.	The facade mints no credentials: the password of a SASL/PLAIN login is a Queen token, checked by Queen.
`RenewDelegationToken`	Extending a delegation token.	Same reason.
`ExpireDelegationToken`	Revoking a delegation token.	Same reason.
`DescribeDelegationToken`	Listing a principal’s delegation tokens.	Same reason.
`ElectLeaders`	Moving a partition’s leadership to its preferred replica (`kafka-leader-election.sh`).	Every partition is led by the raft leader, which raft elects on its own.
`AlterPartitionReassignments`	Moving replicas between brokers (`kafka-reassign-partitions.sh`, Cruise Control).	Every raft voter holds every partition, so there is nothing to move.
`ListPartitionReassignments`	Reassignments in flight.	Same reason.
`DescribeClientQuotas`	Produce and fetch quotas per user and client id.	Queen’s limits are per tenant; Kafka’s quota entities have nothing to map onto.
`AlterClientQuotas`	Changing those quotas.	The same mapping problem, and it would let a client raise its own tenant’s limit.
`DescribeUserScramCredentials`	Listing SCRAM users.	SASL here is PLAIN, checked against Queen, and the facade keeps no user store.
`AlterUserScramCredentials`	Creating or rotating a SCRAM credential.	Supporting SCRAM would make the facade a credential store, with secrets of its own at rest.
`DescribeQuorum`	KRaft’s metadata quorum (`kafka-metadata-quorum.sh`).	Queen’s raft group is not a KRaft quorum. Its state is in `GET /health` and `GET /api/v1/raft/status`.
`DescribeCluster`	Cluster id, controller and brokers in one call.	Answerable, and left out because the Java and librdkafka admin clients build `describeCluster()` from a Metadata request, which works.
`DescribeProducers`	Producer state per partition (`kafka-transactions.sh find-hanging`).	An idempotent producer keeps no state here: each record’s producer id, epoch and sequence become its Queen transaction id, and Queen’s dedup window drops a retry.
`DescribeTransactions`	One transaction’s state.	An open transaction’s records wait in the raft leader’s memory until its commit; there is no transaction log to describe.
`ListTransactions`	Transactions in flight.	Same reason.

Where it differs from Apache Kafka

The cluster a client sees is the raft cluster: every voter is a broker and an in-sync replica of every partition, and the raft leader leads every partition and coordinates every group and every transactional id. Produce, Fetch, ListOffsets and the group and transaction APIs reach the leader; another node answers NOT_LEADER_OR_FOLLOWER or NOT_COORDINATOR and the client asks again.
Inside a transaction, a produce answers offset -1, because no offset exists until the commit. The transaction’s records wait in the leader’s memory and its commit writes them, with its offsets, as one raft entry. There are no commit markers, so a committed transaction of N records advances the log by N and an aborted one by 0, and read_committed and read_uncommitted read the same records. A leader change aborts the transactions that were open on the old leader.
An idempotent producer’s record is deduplicated by Queen’s dedup window: its producer id, epoch and sequence are its Queen transaction id, so a retry within the window writes nothing. There is no per-partition sequence check, so a gap in a producer’s sequence is not refused.
acks=1 is acks=all. acks=0 gets no answer; an error closes the connection instead.
CreateTopics refuses names that begin with __.
Every leader epoch is 0. OffsetForLeaderEpoch is not offered, and clients skip truncation detection against a broker without it.
A timestamp lookup (ListOffsets with a time) is answered on the broker’s append clock, and the records carry the producer’s timestamps (details).
Committed offsets are the Queen consumer group’s positions and never expire. DeleteGroups removes a group with no members and its position on every queue.
Group membership is held in the leader’s memory. When the leader changes, members are told NOT_COORDINATOR or UNKNOWN_MEMBER_ID, rejoin at the new leader and resume from their committed offsets, the same sequence as a Kafka coordinator failover.
A group with no members leaves the leader’s memory ten minutes after its last member left. ListGroups stops listing it and DescribeGroups answers Dead; its committed offsets stay, and a member that joins later starts it again at generation 1. Kafka lists an empty group as Empty until its offsets expire, seven days by default.

Known issues in 2.6

Right after a topic is created through another node, the leader can answer UNKNOWN_TOPIC_OR_PARTITION to ListOffsets and DescribeConfigs for up to a second.
Metadata auto-creation does not refuse names that begin with __, as CreateTopics does, so a client that asks for __consumer_offsets with auto-creation on creates a queue of that name.

Clients

Each row was run with its suite in protocols/queen-kafka/compat/ against the 2.6 facade on 2026-10-20, on one node and on three, with clients bootstrapping at a follower. What failed was what is not offered (above), checks written for the first facade, and the two known issues.

Client	Versions run	Set	Watch for
Java kafka-clients, Spring Kafka	3.6.2, 3.9.1, 4.3.1; Spring Boot 3.5.16 (kafka-clients 3.9.2)	nothing	Keep `group.protocol=classic` on 4.x.
librdkafka: kcat, confluent-kafka-python, Confluent.Kafka (.NET), @confluentinc/kafka-javascript, node-rdkafka, rust-rdkafka, rdkafka-ruby, php-rdkafka	librdkafka 2.6.1 to 2.15	`allow.auto.create.topics=true` on a consumer that has to create its topic	zstd batches go out uncompressed. librdkafka 2.11.0 and older send gzip, snappy and lz4 uncompressed too. node-rdkafka cannot send binary header values.
franz-go	1.21.6	nothing
segmentio/kafka-go	0.4.51	`RequiredAcks` on a `&kafka.Writer{}` literal, which otherwise writes with `acks=0`	Its `Conn` API sends OffsetCommit v2 and OffsetFetch v1 without asking, exactly the floors advertised here.
IBM/sarama	1.60.2	below 1.46.0, `Config.Version = sarama.V1_0_0_0`	Below 1.46.0 without it, producing works and the consumer reconnects forever on EOF.
kafkajs	2.2.4	nothing	Unmaintained upstream since 2024.
@platformatic/kafka	2.11.0	Node 22.22, or 24.6 and later; `mode: 'earliest'` to read from the start
kafka-python	2.3.2	do not pin `api_version` above `(0, 11, 0)`
aiokafka	0.14	0.13 or later for consumer groups	0.12 hangs forever on JoinGroup.
brod (Erlang, Elixir)	4.6.3	nothing	It pins Metadata v2, so it cannot decline auto-creation.

A client running in a container dials the advertised address too, so the node has to advertise one the container can reach, such as host.docker.internal:9092; localhost there is the container itself. A consumer configured for the KIP-848 group protocol (group.protocol=consumer) or for static membership (group.instance.id) fails when it joins, with an error from the client.

Security

There is no TLS listener in 2.6: keep the Kafka port on a private network, or put a TLS-terminating TCP proxy in front of it. With QUEEN_KAFKA_SASL=plain, a client logs in with SASL/PLAIN: the password is a Queen token, the one an SDK sends as Authorization: Bearer, and the username is a label for the logs. Until a connection has logged in it may only negotiate versions and authenticate; anything else closes it. Without TLS, SASL puts tokens on the wire in clear.

bootstrap.servers=kafka-1.example.internal:9092
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required \
  username="orders-service" password="<a Queen token>";

Every call a connection makes into the broker carries the token it logged in with, and the broker authorizes it as it authorizes the HTTP route of the same operation. A connection that did not log in carries none: on a broker that checks tokens (JWT_ENABLED) its calls are refused, so turn SASL on there, and on a broker that does not, every password is accepted.

Ceilings

Compiled in, and listed because each one explains an answer you might otherwise have to guess at.

Ceiling	Value	What happens past it
A request frame	`QUEEN_KAFKA_MAX_REQUEST_BYTES` (100 MiB)	The connection closes.
One produce request, decompressed	256 MiB or 2,000,000 records	The partition that crosses it is refused.
Topic width	100,000 partitions	CreateTopics and CreatePartitions answer `INVALID_PARTITIONS`.
Session timeout a consumer asks for	1 second to 1 hour	`INVALID_SESSION_TIMEOUT`.
One transaction	8 MiB, 50,000 records, 1,000 partitions	A produce past the bytes or the records answers `MESSAGE_TOO_LARGE`; a partition past the thousandth answers `INVALID_TXN_STATE`.
Every open transaction on the leader, together	256 MiB	A produce into a transaction answers `MESSAGE_TOO_LARGE` until others end.
`transaction.timeout.ms`	1 hour	`INVALID_TRANSACTION_TIMEOUT`. Kafka’s `transaction.max.timeout.ms` is 15 minutes; one hour here because that is what Flink’s sink asks for.
Topic name	249 characters of `[a-zA-Z0-9._-]`	`INVALID_TOPIC_EXCEPTION`.
Group id	255 bytes	`INVALID_GROUP_ID`.

Kafka API compatibility reference