Daemons
A daemon is several workers of the same logic under supervision. A crashed one is restarted with a growing back-off, a stuck one is killed and replaced, and the number of working hands can change on the fly — by hand or by load.
What a daemon is and why
A daemon is a component of two parts: a supervising master and the fleet of identical workers it manages.
The problem. A process solves “keep working continuously” but remains one. The moment there is more work than one worker can handle, questions appear that a lone process never had:
- run five copies — how? Five twin classes, or five
systemdunits? - one crashed — who notices and brings it back? And if it crashes in a loop, how do you avoid taking the system down with endless restarts?
- one hung on a network call forever — formally it is alive,
psshows it; - at night there is half the work — how do you remove the extras without cutting off the busy ones?
Each answer alone is not hard, but together they are a supervisor you would rather not write.
The solution. Winter provides one. You describe the body of one worker and say how many are needed; the master watches the fleet: forks them, counts crashes, waits before restarting, kills stuck ones, and adds or removes workers on your signal.
Quick start
Extend Daemon, describe the work in workerRun() and state the replica count:
<?php
namespace Main\Process;
use Flytachi\Winter\DI\Attribute\Autowired;
use Flytachi\Winter\Kernel\Process\Stereotype\Daemon;
class Emails extends Daemon
{
#[Autowired] private OutboxRepository $outbox;
#[Autowired] private Mailer $mailer;
protected int $replicas = 4; // four workers
protected function workerRun(): void
{
while ($this->isRunning()) {
$letter = $this->outbox->takeNext();
if ($letter === null) {
$this->sleep(1);
continue;
}
$this->markBusy();
try {
$this->mailer->send($letter);
$this->outbox->markSent($letter->id);
} finally {
$this->markIdle();
}
}
}
}Declare it in the manifest:
#[EnableDaemon(\Main\Process\Emails::class)]
final class Application extends WinterApplication { /* ... */ }Or start it on its own:
php call daemon main.process.Emails start -d
php call daemon main.process.Emails status | [================================= Daemon =================================]
| [ Daemon Status ]
| Main.Process.Emails ......................................... [Daemon ● RUNNING]
| - - - - - - - - - - - - - - - - - - - - - - -
| PID 51204
| State RUNNING
| Activity BUSY
| Started 2026-08-20 16:18:40 +00:00
| Uptime 6h 12m
| Workers 4
| Restarts 1
| - - - - - - - - - - - - - - - - - - - - - - -
| [ Workers (4) ]
| SLOT PID STATE ACT UPTIME RESTARTS
| #0 51210 running busy 6h 12m 0
| #1 51211 running idle 6h 12m 0
| #2 51212 running busy 6h 12m 0
| #3 52890 running idle 41m 1
| [ Daemon Status ]
| [================================= Daemon =================================]The worker body is `workerRun()`, not `run()`
A daemon’s run() is declared final: it belongs to the supervisor itself.
Describe the work in workerRun() — that is the body of one worker, and it
executes in a child process, not in the master.
Everything a process has is available inside: isRunning(), sleep(),
markBusy(), markIdle(), spawn(), activity(), touch(), the ready
$this->logger. A daemon inherits Process in full — the
complete list is in
What else is available inside.
The worker as a separate class
If the worker body is already described by a standalone process, workerRun() can
be omitted — name the class instead:
class Emails extends Daemon
{
protected int $replicas = 4;
protected ?string $workerClass = \Main\Process\SendWorker::class;
}That is convenient when the same worker must be runnable both alone and as a fleet.
The body must be given by one of the two means: with neither workerRun() nor
$workerClass the daemon goes into an endless crash loop.
Settings
| Property | Default | What it sets |
|---|---|---|
$replicas |
1 |
The base number of workers |
$workerClass |
null |
The worker class, when the body is not in workerRun() |
$grace |
30.0 |
How many seconds to wait for a worker to finish before SIGKILL |
$livenessTimeout |
0.0 |
Watchdog: kill a worker silent longer than this; 0 — off |
$concurrency |
0 |
The limit on concurrent spawn() tasks inside a worker |
`$grace` applies to each worker
The default is the same as for an ordinary process — 30 seconds, like Kubernetes’
terminationGracePeriodSeconds: one stuck worker should not hold up stopping the
whole fleet. Zero means “wait forever” — set it only when the work must never be
interrupted.
The minimum replica count is one: $replicas = 0 is raised to 1. Collapsing the
fleet to zero is only possible dynamically, through desiredReplicas().
Restarting crashed workers
The daemon restarts its workers; nobody restarts the daemon
Everything below is about the workers inside a daemon. The application does not restart
the daemon itself: when it gives up (spends maxRestarts) or crashes, the critical
parameter decides — stop the application, or leave the daemon down. A daemon whose
workers never came up within startTimeout has not started — see
Supervision.
What to do with a dead worker is decided by the restart policy.
use Flytachi\Winter\Kernel\Process\Daemon\{RestartPolicy, RestartMode};
protected function restart(): RestartPolicy
{
return new RestartPolicy(
mode: RestartMode::ON_FAILURE,
maxRestarts: 0,
backoff: 1.0,
);
}| Parameter | Default | What it sets |
|---|---|---|
mode |
ON_FAILURE |
When to restart |
maxRestarts |
0 |
The restart limit for the whole fleet; 0 — no limit |
backoff |
1.0 |
The base pause before a restart, in seconds |
Three modes, named by the Kubernetes and systemd convention — if you have written a restart policy there, the semantics are the same:
| Mode | Behaviour |
|---|---|
ALWAYS |
Restart on any exit — the worker must live until the daemon is stopped |
ON_FAILURE |
Restart only after a crash; a clean exit is treated as final |
NEVER |
Do not restart: one attempt and that is it |
Every worker crash is logged with its cause and what was decided:
worker#1 (PID 38) crashed: killed by signal 9; restarting in 1s.
worker#3 (PID 52) crashed: exit code 1; not restarted (restart policy).
worker#2 (PID 61) crashed: exit code 255; restart limit reached.not restarted (restart policy) means the policy does not replace the worker (NEVER);
restart limit reached means maxRestarts ran out, and the daemon stops its fleet. The
number in worker#N is the one ps shows: slot #0 is worker#1.
The growing pause
A restart is not immediate: the pause doubles with each crash of that worker and caps at 30 seconds. With a one-second base that gives
1 → 2 → 4 → 8 → 16 → 30 → 30 → …
The point is that a worker crashing because the database is unreachable does not become an endless fork loop and does not stop the system from recovering.
`maxRestarts` counts across the fleet and stops the daemon entirely
It is not “how many times to restart each worker” but how many restarts are allowed
in total. On reaching the limit the supervisor does not merely stop refilling one
slot — it stops the whole daemon, moving it to the FAILED state with a
critical log entry.
The default 0 (no limit) is right for most cases: a daemon should survive a
temporary outage of its dependencies. Set a limit when failing visibly beats working
forever to no effect.
The liveness watchdog
A crashed worker is visible; a stuck one is not: the process is alive, ps shows
it, the work has stopped. There is a separate mechanism for that:
protected float $livenessTimeout = 120.0;Every worker regularly signals that it is alive. When no signal arrives for longer than the given time, the master kills the worker outright, and from there it follows the ordinary crash path — a pause and a restart.
Do not kill a healthy one
The liveness signal is given when a worker yields — on a database wait, a network call or a pause. Long work without I/O (a heavy computation) looks exactly like a hang to the watchdog.
Set $livenessTimeout comfortably above your longest uninterrupted operation, and
inside such an operation signal yourself:
$this->touch();
Scaling
The worker count can change on the fly. Override desiredReplicas() — it is polled
regularly and its answer becomes the target:
protected function desiredReplicas(): int
{
$depth = $this->outbox->pendingCount();
return match (true) {
$depth > 10_000 => 20,
$depth > 1_000 => 10,
$depth > 0 => 4,
default => 1,
};
}Zero is allowed too — the fleet collapses entirely and unfolds when work appears.
Why the fleet does not change at once
The answer from desiredReplicas() is a signal, not a command. The supervisor
damps it so a random spike does not jerk the fleet back and forth:
use Flytachi\Winter\Kernel\Process\Daemon\ScalingPolicy;
protected function scaling(): ScalingPolicy
{
return new ScalingPolicy(
scaleInterval: 1.0,
scaleUpDelay: 0.0,
scaleDownStabilization: 60.0,
cooldown: 3.0,
scaleStep: 0,
);
}| Parameter | Default | What it sets |
|---|---|---|
scaleInterval |
1.0 |
How often desiredReplicas() is polled |
scaleUpDelay |
0.0 |
How long demand must hold before the fleet grows; 0 — at once |
scaleDownStabilization |
60.0 |
How long low demand must hold before shrinking |
cooldown |
3.0 |
The minimum between two scaling actions |
scaleStep |
0 |
The most workers per action; 0 — unlimited |
The model is deliberately asymmetric: up quickly, down cautiously. A load spike has to be served immediately, whereas removing workers over a minute of quiet is a bad idea — a minute later they would have to come back.
Shrinking respects business: all else being equal the idle workers go first, and a
busy one is given until $grace to finish.
Sharing work across workers
The daily difference between a daemon and a process is that there are several workers, and they reach for work at the same time. The framework starts and watches them but does not coordinate them: how to divide the work is your decision.
A naive select hands one task to everybody
Code like this in a worker body looks right and breaks immediately:
$letter = $this->db->query('SELECT * FROM outbox WHERE sent = 0 LIMIT 1');
Four workers run it almost simultaneously, receive the same row — and the letter
goes out four times. The bug does not show at replicas: 1, which is why it usually
arrives in production together with scaling.
A task must be claimed atomically: marked as yours and returned in one action, so nothing slips in between “selected” and “took”.
The most portable technique is to mark the row with a conditional update and then read back exactly what you marked:
-- 1. try to claim: mark exactly one free row with our marker
UPDATE outbox
SET claimed_by = :worker, claimed_at = NOW()
WHERE id = (SELECT id FROM outbox WHERE claimed_by IS NULL ORDER BY id LIMIT 1);
-- 2. take what we, specifically, claimed
SELECT * FROM outbox WHERE claimed_by = :worker AND claimed_at IS NOT NULL LIMIT 1;In PostgreSQL and MySQL 8 the same thing is shorter — a select that skips locked rows:
SELECT * FROM outbox
WHERE claimed_by IS NULL
ORDER BY id LIMIT 1
FOR UPDATE SKIP LOCKED;The slot number or the worker’s PID makes a convenient marker: then stuck tasks show who claimed them.
You write the queue
Winter provides no queue — neither a table nor a broker. The OutboxRepository in
the examples is yours; usually it is a table with state and claim columns, over
which you write “take”, “complete” and “return”. Talking to the database is covered
in Repository.
A task a worker took must be able to come back: if the process is killed mid-work, the claim marker is left hanging. That is usually cured by a scheduler task that frees records claimed too long ago.
A worker’s number is not for dividing work
A worker can learn its slot number — Process::current()->slot. It is useful in logs and
metrics, to tell workers apart:
$this->logger->info('batch done', ['slot' => Process::current()->slot]);But dividing work by it up front — “worker #0 takes even records, #1 odd ones” — is a bad idea. Numbers live exactly until the first crash: a dead worker is replaced, the fleet may grow or shrink, and a partition tied to the replica count becomes wrong after every such event — part of the work goes to two workers, part to none.
So work must be divided through shared state, not by position: an atomic claim from a queue as above, or a claim over a whole chunk when the unit of work is large.
-- the worker claims not a task but a whole partition of work
UPDATE shards
SET owner = :pid, taken_at = NOW()
WHERE id = (SELECT id FROM shards
WHERE owner IS NULL OR taken_at < NOW() - INTERVAL '5 minutes'
ORDER BY id LIMIT 1);The second condition — about expiry — is mandatory here: if a worker dies, its partition has to return to the pool rather than stay claimed forever.
Lifecycle hooks
All four are optional and are called in the master.
| Hook | When |
|---|---|
onWorkerStart(int $slot, int $pid) |
A worker started in a slot |
onWorkerExit(int $slot, int $pid, bool $crashed) |
A worker finished; $crashed says whether it was a failure |
onScale(int $from, int $to) |
The fleet size changed |
tick() |
Periodically, roughly every scaleInterval — handy for metrics |
protected function onWorkerExit(int $slot, int $pid, bool $crashed): void
{
if ($crashed) {
$this->metrics->increment('emails.worker_crashed');
}
}
protected function tick(): void
{
$this->metrics->gauge('emails.queue_depth', $this->outbox->pendingCount());
}An exception inside a hook is caught and logged — a broken hook will not bring down the supervisor.
Hooks run in the supervisor, not in a worker. Daemon::supervising() tells it apart: in the
supervisor it returns the daemon class, everywhere else null. Process::current() is the
other way round — null in the supervisor: it has no body of its own, the workers do the
work.
Daemon::supervising(); // 'Main\Process\Emails' in the supervisor, null in workers and on the webQuerying the database from a hook is fine — as in the tick() example above. Before every
fork of a worker the supervisor closes its database and Redis connections, so the worker
neither inherits nor tears them down; the supervisor reconnects on its next query. The one
thing not to do is leave a transaction open in a hook: a connection inside a transaction
cannot be closed, so the worker is not started until the transaction is closed, and every
attempt is logged as an error.
Reloading on SIGHUP
SIGHUP to a daemon is a reload, not a stop. The master runs its own onReload() and
passes SIGHUP to every live worker, where the worker body’s onReload() runs — of the
same daemon class, or of the $workerClass class. The same Daemon::supervising() tells
where the hook is running.
protected function onReload(): void
{
if (Daemon::supervising() !== null) {
$this->logger->info('master: re-reading the fleet settings');
return;
}
$this->logger->info('worker: dropping my caches');
}A daemon under php call run gets SIGHUP from the application:
kill -HUP <application PID> reloads every component — see
Signals to the application. A worker
that is still starting ignores such signals until it sets its own handlers: by default
SIGHUP would end it.
Worker states
In the status output every slot has its own state. It helps you see what the fleet
is doing right now.
| State | What it means |
|---|---|
starting |
Forked, waiting for the first liveness signal |
running |
Working |
retiring |
Told to finish and is completing its work |
killing |
Missed $grace, SIGKILL sent |
restarting |
Died unexpectedly, waiting out the pause before a restart |
retired |
Died, and the policy decided not to replace it |
Slot numbers are stable: a restart returns the worker to the same slot, so #2
in the logs always means the same member of the fleet.
What the fleet looks like in ps
The master and the workers name themselves so that the whole tree is found by one
grep:
ps -eo pid,command | grep winter-daemon12825 winter-daemon: StableDaemon master
12826 winter-daemon: StableDaemon worker#1
12827 winter-daemon: StableDaemon worker#2The number in the name is one plus the slot number: slot #0 in the status table
shows up in ps as worker#1. The name comes from $processTitle, or from the class’s
short name when it is not set — the same as for a plain process.
Control from code
A daemon inherits the same static control surface as a process, so the fleet can be started, stopped and inspected straight from the application.
| Method | What it does |
|---|---|
start(): void |
Raises the supervisor in the current process, never returning |
dispatch(?string $output = '/dev/null'): int |
Raises the supervisor in the background, returns its PID |
status(bool $usage = false): ?DaemonStatus |
The fleet’s state, or null when the daemon is not running |
stop(): bool |
Gracefully stops the whole fleet |
relaunch(): bool |
Restarts it: stop, then start again; false when that could not be done |
While the application runs, stop(), dispatch() and relaunch() go through it:
stop() leaves the daemon stopped, dispatch() starts it again inside the application,
relaunch() asks the application to restart it — not counted as a crash. The details are
in Controlling a running component.
use Main\Process\Emails;
$pid = Emails::dispatch(); // the supervisor went into the background
Emails::stop(); // stop the whole fleetAs with a process, start() blocks its caller — from the web use dispatch().
Fleet status
status() returns not merely the master’s state but a snapshot of the whole fleet:
two fields are added to the usual process ones.
| Field | Type | What it holds |
|---|---|---|
$restarts |
int |
How many restarts have happened since startup |
$workers |
WorkerStatus[] |
One record per occupied slot |
For each worker:
| Field | Type | What it holds |
|---|---|---|
$slot |
int |
The slot number — stable across restarts |
$pid |
int |
The worker process id |
$state |
SlotState |
running, starting, restarting and the rest |
$activity |
Activity |
IDLE or BUSY |
$startedAt |
int |
When this worker was started |
$restarts |
int |
How many times this particular slot restarted |
That lets you build your own checks and dashboards without parsing console output:
use Flytachi\Winter\Kernel\Process\Activity;
$status = Emails::status();
if ($status === null) {
Emails::dispatch(); // the fleet is down — raise it
return;
}
$busy = array_filter(
$status->workers,
fn($w) => $w->activity === Activity::BUSY,
);
// every worker busy for over a minute — worth a look
if (count($busy) === count($status->workers)) {
$this->alerts->fire('Emails: the whole fleet is busy');
}The snapshot can be returned as it is
DaemonStatus and WorkerStatus serialise to JSON themselves, so the status object
works as a controller response with no assembly by hand.
Control from the console
php call daemon list # every daemon and its fleet size
php call daemon main.process.Emails # the supervisor in the current terminal
php call daemon main.process.Emails start -d # in the background
php call daemon main.process.Emails stop # graceful stop of the whole fleet
php call daemon main.process.Emails restart # stop and start again
php call daemon main.process.Emails status # state + the worker table
php call daemon main.process.Emails status -v # plus the master's resource usageStopping happens in two steps
The first signal stops the fleet gracefully: no new tasks are taken, busy workers
finish what they started within $grace. A second signal terminates everyone
immediately.
Examples
A fixed-size fleet
The commonest case: there is steadily plenty of work and autoscaling is not needed.
<?php
namespace Main\Process;
use Flytachi\Winter\DI\Attribute\Autowired;
use Flytachi\Winter\Kernel\Process\Stereotype\Daemon;
class Webhooks extends Daemon
{
#[Autowired] private WebhookRepository $hooks;
#[Autowired] private HttpClient $http;
protected int $replicas = 8;
protected float $grace = 20.0;
protected float $livenessTimeout = 90.0;
protected function workerRun(): void
{
while ($this->isRunning()) {
$hook = $this->hooks->takeNext();
if ($hook === null) {
$this->sleep(1);
continue;
}
$this->markBusy();
try {
$this->http->post($hook->url, $hook->payload);
$this->hooks->markDelivered($hook->id);
} catch (\Throwable $e) {
$this->hooks->retryLater($hook->id);
$this->logger->warning('webhook failed', ['id' => $hook->id]);
} finally {
$this->markIdle();
}
}
}
}Three things without which the fleet works worse: markBusy() stops a worker being
retired mid-delivery and resets the state between tasks; retryLater() in the
catch returns the task to the queue; $livenessTimeout catches hung HTTP calls.
Autoscaling by queue depth
<?php
namespace Main\Process;
use Flytachi\Winter\DI\Attribute\Autowired;
use Flytachi\Winter\Kernel\Process\Daemon\ScalingPolicy;
use Flytachi\Winter\Kernel\Process\Stereotype\Daemon;
class Imports extends Daemon
{
#[Autowired] private ImportQueue $queue;
protected int $replicas = 2; // the base size
protected float $grace = 120.0; // an import must not be cut in half
protected function desiredReplicas(): int
{
return min(16, max(2, (int) ceil($this->queue->pendingCount() / 50)));
}
protected function scaling(): ScalingPolicy
{
return new ScalingPolicy(
scaleUpDelay: 5.0, // a spike must hold for 5 s
scaleDownStabilization: 300.0, // shrink only after 5 minutes of quiet
scaleStep: 4, // no more than 4 workers at a time
);
}
}The settings here suit the particular job: an import is long, so shrinking should be unhurried, and growth is capped by a step so the database is not handed sixteen connections at once.
Note that there is no workerRun() here — so the body comes from $workerClass. If
that is not set either, the daemon crashes on every fork.
One worker, but supervised
A daemon makes sense with a single replica too: you get restarts after crashes and the liveness watchdog, neither of which an ordinary process has.
class SnmpPoller extends Daemon
{
protected int $replicas = 1;
protected float $livenessTimeout = 60.0;
protected function restart(): RestartPolicy
{
return new RestartPolicy(mode: RestartMode::ALWAYS, backoff: 5.0);
}
protected function workerRun(): void { /* ... */ }
}ALWAYS fits better than the default here: a poller should never exit cleanly at
all, so if it did, something is wrong and it must come back regardless.
Next
- Processes — the worker body primitives: loop, pauses, units of work
- Components — where
#[EnableDaemon]is declared - Scheduler — when the work is tied to the clock
- Logging — where the supervisor writes
- Actuator / Health — watching the application from outside