Consume Queen messages in Laravel with Artisan

php artisan queen:consume reads a Queen queue and passes each message to a class of yours. Use it for messages that other services push to Queen. Laravel jobs keep going through queue:work (Laravel).

The command connects with the settings of config/queen.php, as the queue connection does (QUEEN_URL, QUEEN_BEARER_TOKEN). The service provider registers it for the console only. This page describes PHP client 2.3.0 and later; before 2.3.0 lists what differs in earlier releases.

Try it

Start the command before the first push. A new group starts with the messages pushed after its first pop, unless you pass --subscription-mode=all (where a new group starts).

The handler

• Built by the container. The command resolves the class with app(), so constructor injection works. One instance handles every message of the run.
• One message or a list. handle() gets one message array. With --batch above 1 it gets a list of up to that many messages.
• The message. $message['data'] is the payload. transactionId, partitionId, leaseId, deliveryAttempt and the other keys are in the PHP client reference.
• Exceptions. The command prints the exception’s message as an error line, nacks the message (or the list) with --auto-ack only, then pops again. Laravel’s exception handler does not see it.

Ack and nack

• A nack spends a retry. With --auto-ack, a message that always throws reaches the dead-letter queue once the queue’s retries are spent (retry budget). The nack carries the exception’s message as its error.
• Without --auto-ack, the handler settles. A throw is printed, then Not nacked without --auto-ack. The message comes back when its lease expires, which spends no retry. Nack it in the handler when a failure must count, as the SDK consumers expect.
• Client-side. The command acks after handle() returns, so delivery stays at-least-once.
• One call per pop. With --batch, one ack or nack covers the whole list.
• Refusals are printed. A refused call prints one warning, for example Ack refused for 1 of 1 message: invalid or expired lease.

Without --auto-ack, ack in the handler, with the group you passed:

<?php

namespace App\Queen;

use Queen\Laravel\QueenFacade as Queen;

final class RecordPayment
{
    public function handle(array $message): void
    {
        // ... the work ...
        Queen::ack($message, true, ['group' => 'ledger']);
    }
}

Without --group, leave group out of the context. With --auto-ack, leave the ack to the command.

Lease

Every pop asks for a lease of --lease seconds, by default retry_after of config/queen.php (90). Keep handle() well inside it, or turn on renewal.

• Renewal. With lease_renewal true in config/queen.php and --auto-ack, the command renews the lease while handle() runs, with the renewer of queue:work (lease renewal and fencing).
• Timing. The lease_renewal_* keys apply, measured against --lease. A timing that does not fit stops the command at start, with exit code 1 (lease renewal keys).
• Fencing. When a renewal cannot finish in time, the renewer sends the command SIGTERM, then SIGKILL after the kill grace, before the lease ends.
• Before each ack or nack. The command checks the lease and stops its renewal. A lease that is no longer safe is neither acked nor nacked: the command prints Not acked: (or Not nacked:) and the reason, and the broker hands the message out again.
• Without --auto-ack. Renewal stays off, and the command prints one line to say so. A handler that acks by itself releases the lease, and renewing a released lease stops the process.
• Long polls. The command counts the lease from the start of each pop. Keep --timeout well below --lease: a message that arrives late in a long poll can leave too little lease to track, and the command then stops with exit code 1.

Options

• --partitions. The partitions of one pop share the --batch budget.
• --no-autopilot. The broker stops choosing the partitions. QUEEN_SDK_POP_AUTOPILOT=off does the same for the whole process.
• --lease. An integer from 1 to 2147483647. Another value stops the command with exit code 1.
• --conflation. It needs --group and broker 1.1.0 or later. It works with --auto-ack.
• --limit. It counts every message handed to handle(), failed ones too. A pop asks for at most the messages the limit leaves, so the command never handles more than N.
• --idle-timeout. Counted from the start, then from the end of the last handle(). A long poll never waits past it. The command prints No message for N ms (--idle-timeout): stopping. and exits with code 0.

Before production

• --group has no default. The command does not read QUEEN_CONSUMER_GROUP, which is the group of the queue connection.
• No restart on deploy. The command does not watch queue:restart. Restart it with your process manager.

Stop and exit codes

SIGINT (Ctrl+C) and SIGTERM stop the command after the pop or the message in progress. It then prints Consumer stopped. Processed N messages. This needs pcntl, and the handlers the command installs replace yours for those two signals.

A pop that times out, or fails with a connection error, counts as an empty pop. After a connection error the command waits 1 s before the next pop. It also prints Queen broker unreachable: and the error, again at most every 30 s while it lasts, and Queen broker reachable again after N s. once a pop gets an answer.

Errors that leave the loop include:

• ConflationUnsupportedException. --conflation against a broker older than 1.1.0.
• ConflationPolicyMismatchException. The group conflates, and the command has no --conflation.
• HttpException. The broker refused the pop, for example with a 403.

Before PHP client 2.3.0

• The nack carried no error. With --auto-ack, a throw was nacked without the exception’s message.
• No --lease and no renewal. Every pop got the queue’s own lease time.
• --idle-timeout had no effect. The command ran until a signal or --limit.
• --limit counted only successes. With --batch it could handle up to --batch minus 1 messages more than N.
• Nothing was printed for a refused ack or for a broker the pops could not reach.

The command is a loop around the client’s KafkaConsumer-style consumer. Write your own loop with Queen::queue('payments')->getConsumer() when you need another ack policy.