Skip to main content

Configure Workers

The workers block in your config.cap.yaml defines background processes that run alongside your main application, for processing queues or handling long-running tasks. Each worker needs a unique name of at most 32 characters, and either a command or a subcommand to run.

workers:
  my-queue-processor:
    command: php artisan queue:work --queue=my_queue
    replicas: 1

Options

Property Type Description Default
command string or array The full command line to run, including a binary such as php. See Command or subcommand. None
subcommand string or array The command to run with your framework's binary. See Command or subcommand. The name
replicas integer The number of instances of this worker to run. Negative values are rejected. 1
resources.cpu integer or string CPU allocated to each instance. A number of cores, or thousandths such as 250m. 125m
resources.memory integer or string Memory allocated to each instance. A number of bytes, or a value with a unit such as 512Mi. 256Mi
stateful boolean Whether instances get stable hostnames and are replaced one at a time. See Stateful workers. false
terminationTimeout integer or string How long the worker has to finish its work and exit before it is forcibly stopped. See Graceful shutdown. 30
restartInterval integer or string How often the worker is stopped and started again. Values below 1 second are rejected. See Restarting workers. None
scale boolean or object Whether to match the number of instances to the length of a queue. See Autoscaling on a queue. false
scale.enable boolean Whether to match the number of instances to the length of a queue. false
scale.connection string The name of the RabbitMQ connection the queue lives on. default
scale.queue string The name of the queue that is watched. The name
scale.vector integer The number of queued messages one instance is expected to handle. 20
scale.maxReplicas integer The highest number of instances the queue can grow to. Values below replicas are rejected. 10
variables object Environment variables for this worker only, on top of the global ones. See Environment Variables. None

command and subcommand cannot both be set on the same worker.

Command or subcommand

Every worker runs one of two things. command is the full command line, exactly as you would type it. subcommand is just the part after your framework's binary, and CAP fills in the rest from your application type.

workers:
  emails:
    subcommand: queue:work --queue=emails   # php artisan queue:work --queue=emails
  legacy:
    command: php /app/bin/legacy-worker.php # run verbatim

Which binary a subcommand gets depends on the type: php artisan on Laravel, php bin/console on Symfony and Shopware, php bin/cake on CakePHP, npm run on Node. See Application Types for the full list.

Omitting both runs the worker's own name as the subcommand, so a worker named import runs php artisan import on Laravel.

For workers, subcommand also wraps the process in a supervisor that restarts it when it exits, which is what you want for a queue consumer that is meant to run forever. A command is run as given and is not restarted, so use command only when your process manages its own lifecycle.

Custom base image required for subcommands. The subcommand option relies on a binary that only exists in our base container. Use command if you build your own image.

Graceful shutdown

By default, when a rollout happens or a shutdown is requested for any other reason, a worker has 30 seconds to finish its current work and exit cleanly before being forcibly stopped. You can raise or lower this limit with terminationTimeout.

The value can be an integer (seconds) or a human-readable duration string:

workers:
  my-queue-processor:
    command: php artisan queue:work
    terminationTimeout: 300   # 5 minutes, as seconds

  my-slow-worker:
    command: php artisan process:batch
    terminationTimeout: 10m   # the same limit, as a duration string

Accepted duration string units include s (seconds), m (minutes), h (hours), and d (days), and they can be combined (e.g., '1h 30m').

Set terminationTimeout high enough to cover the longest job your worker may be processing at the time of a deployment. When the timeout expires before the process exits, the worker is stopped anyway and any in-progress work is lost.

Restarting workers

Long-running workers can become stale, for example because they hold on to memory or to a connection that has gone away. With restartInterval the worker is stopped and started again at a fixed interval, so it always runs on a fresh process.

workers:
  my-queue-processor:
    subcommand: queue:work
    restartInterval: 1h      # restart every hour

  my-slow-worker:
    subcommand: process:batch
    restartInterval: 21600   # 6 hours, as seconds

The value accepts the same formats as terminationTimeout. Without it the worker keeps running until something else stops it.

A restart follows the same steps as a normal shutdown: the worker is asked to stop, finishes the job it is working on, and is only forcibly stopped when that takes longer than terminationTimeout.

Stateful workers

By default, worker instances are interchangeable, and during an update an old and a new instance can briefly run at the same time. Setting stateful: true changes that:

  • Each instance gets a stable, predictable hostname (my-worker-0, my-worker-1).
  • Instances are replaced in order: the existing one is stopped before the new one starts.
  • The number of running instances never exceeds the configured replicas.

This is useful for workers that must not run concurrently, such as singleton background processors, or for workloads that benefit from a stable hostname.

workers:
  my-singleton-worker:
    stateful: true
    command: php artisan my-command

With multiple replicas, each instance gets a dedicated hostname:

workers:
  my-worker:
    stateful: true
    replicas: 3
    # Instances: my-worker-0, my-worker-1, my-worker-2

Stateful workers can be combined with scale. Instances are still replaced in order.

Autoscaling on a queue

CAP can match the number of worker instances to the length of a RabbitMQ queue, adding instances when a backlog builds up and removing them again when the queue drains.

workers:
  runner:
    scale: true

That watches the queue named after the worker on the default connection. Write scale as a block to tune it:

workers:
  runner:
    replicas: 2
    scale:
      enable: true
      connection: default
      queue: my-custom-queue
      vector: 20
      maxReplicas: 10

replicas is the lowest number of instances, held even when the queue is empty. The target number of instances is the queue length divided by vector, so a lower vector adds instances sooner.

Back to top