Connection pool
Application code never sees the pool: it takes a store and works. This page is about what happens underneath, and about the three decisions a person still makes — how many connections, what to do when they run out, and what to wire in at startup.
What the pool is and why
The problem. A Redis connection is one socket with a sequential protocol: a command, a
reply, the next command. While one request is using it, a second cannot go through. In
classic PHP that did not matter — the process served one request and died. A resident worker
serves many requests at once, each in its own coroutine, and one shared connection there
means two coroutines writing into it interleaved: a reply arrives for somebody else’s
request, packets out of order shows up in the log, and under load the whole worker goes
down.
Opening a connection per request is the other extreme: a handshake, authentication and a
database select for every little thing, and a thousand simultaneous requests turn into a
thousand connections and max number of clients reached on the server.
The solution. A set of ready connections a coroutine borrows one of for the duration of its work and returns when it ends. Connections are reused, their number has a ceiling, dead ones are replaced. Application code does not change at all: it takes a store and works.
How connections are handed out
Under Swoole
Every config class gets its own pool. The first call inside a coroutine borrows a
connection from it and puts it in the coroutine context; a defer returns it when the
coroutine ends — on a normal exit and on an exception alike.
// one coroutine = one connection for its whole life
$store->get('a'); // the connection is borrowed here
$store->set('b', 1); // the same connection
// end of request — it went back on its ownTwo consequences are worth keeping in mind. First: two coroutines never share a socket, so commands never interleave. Second: the connection is held for the whole life of the coroutine, not for the duration of a command — if a request talks to Redis at the start and again at the end, the pool counts it as busy in between.
Without Swoole
A process serves one unit of work at a time and there is nothing to distribute: each config gets one self-maintaining connection for the life of the process. Its liveness checks and lifetime are the same, so a long-running CLI worker does not wake up holding a socket the server closed hours ago.
Application code is identical in both cases — there is nothing to branch on.
What the pool does with connections
- Probes before handing over. A connection idle for more than half a second is
checked with
PING; a dead one is retired and replaced. Hot connections skip the probe, so a busy service does not pay for it. - Watches their age. A connection older than half an hour is reopened — that is how the pool keeps up with infrastructure moving underneath it: proxies, load balancers, failover addresses.
- Caps their number. A thousand concurrent requests do not become a thousand connections: the extra ones wait their turn.
- Cleans up after the request on return. A connection goes back to the pool in the
state it was handed out in: an abandoned
MULTIor an unfinishedpipeline()is discarded (DISCARD), and if the request switched the database withSELECT, the configured database is selected back.
All of this is the machinery of CPool, shared with PPA.
What the pool cleans up on return
MULTI, pipeline() and the selected database live on the connection, and the connection
outlives the request. Without a cleanup the next request would inherit them. An abandoned
MULTI is the worst — a request began a transaction and failed before exec():
- the next request’s reads return not values but a
\Redisobject — the command was only queued; - its writes never reach the server;
- the first
exec()on that connection publishes the failed request’s writes, the ones it meant to abandon.
So on return the pool looks at the mode and the database number — phpredis keeps both on
its side, nobody goes to the server to ask — and, if there is something to clean up, sends
DISCARD and SELECT back. If that fails, the connection is evicted. Every such cleanup is
logged at ERROR:
App\Config\CacheRedis: connection returned to the pool with MULTI (cid=42) — reset; queued commands discarded. Close every multi()/pipeline() with exec()/discard(); use a separate config class per database instead of select().The cleanup is a safety net, not a way of working
Why DISCARD and not EXEC: the request did not finish the sequence, and executing it on
the request’s behalf means writing what it never meant to write. A line in the log means
there is a path in the code on which MULTI is not closed. Sequences are best run inside
transaction(): on an exception it cleans up after
itself, including a WATCH the pool cannot see on return.
What happens during one request
$store->get('a'); // 1. the pool hands over a connection, the defer is armed
$store->set('b', 1); // 2. the same connection, from the coroutine context
$store->hash('c')->all(); // 3. and again the same one
// 4. the coroutine ended — the defer returned itStep by step on that first call: the pool looks for a free connection; if there is one
and it has been idle longer than aliveBypassWindow, it is probed with PING and
replaced if dead; if none is free and the ceiling is not reached, a new one is opened; if
the ceiling is reached, the borrower waits up to poolWaitTimeout. The connection
obtained goes into the coroutine context, and a defer is registered.
Every later call in that coroutine takes the connection from the context — no further
probes, no further borrowing. The defer fires on a normal exit, on an exception and on
exit alike, so a connection cannot be lost even by a request that crashed.
Pool size
The defaults are 10 connections per config and a 3 second wait. Change them through a pool-aware config.
| Setting | Default | What it does |
|---|---|---|
$poolMaxConnections |
10 |
ceiling on the number of connections |
$poolWaitTimeout |
3.0 |
deadline for a whole borrow: waiting, discarding dead ones, reopening |
$keepaliveTime |
120.0 |
background ping of idle connections; 0 is off |
$idleTimeout |
600.0 |
close idle connections; 0 never closes |
$minimumIdle |
0 |
warm connection floor |
How to size the ceiling: count across all workers, not one. The number the server sees
is worker_num × poolMaxConnections × instances, and it should be compared with your
Redis maxclients (10 000 by default — considerably more generous than a database’s
max_connections, which is why the default here is higher than PPA’s).
Housekeeping is on by default, and for Redis that matters more than for the database: the
pool here is ten per worker, so a pause during which the server closed the sockets left
ten corpses for the next request to find. keepaliveTime pings idle connections every two
minutes — exactly what Redis’s own timeout directive, which many managed offerings use to
close inactive clients, would otherwise do to them. idleTimeout hands the connections
back after ten idle minutes instead of holding worker_num × 10 of them overnight.
You switch them off in the opposite cases: keepaliveTime when Redis is next door and
connections are cheap, idleTimeout when the pool should stay warm between bursts — and
that is also when you set minimumIdle, so the next burst does not start from zero.
The timer is armed by the first borrow
Until a connection has been borrowed there is no timer and the pool costs nothing. After
the first borrow it lives until RedisPool::shutdown() — see the warning below.
Exhaustion
When every connection is busy, the borrower waits. If nothing frees up within
poolWaitTimeout:
RedisPoolException: RedisPool: no connection for [Main\Configurations\MainRedisConfig] —
ConnectionPool: no free connection within 3s — raise maximumPoolSize or connectionTimeout.That is a guard rail, not a failure: a queue inside the application beats max number of clients reached on the server, which takes the neighbouring applications down with it.
Before raising the ceiling, look for slow requests
A connection is held for the whole life of the coroutine. A request that talks to Redis and then waits half a second on an external API holds its connection all that time — and a pool of ten serves twenty requests a second instead of thousands. Raising the ceiling hides that rather than fixing it.
Observing it
use Flytachi\Winter\Redis\RedisPool;
RedisPool::stats();
// [
// 'Main\Configurations\MainRedisConfig' => [
// 'total' => 4, 'idle' => 3, 'active' => 1, 'maximum' => 10,
// ],
// ]The numbers are per worker: each worker keeps its own pool in its own memory. From the
outside they show at /actuator/pools —
marked "source": "redis", for the worker that served the request; the endpoint has to be
opened with #[EnableActuator(expose: [...])]. /actuator/health does not show pools: it
checks reachability by connecting with fresh config instances. Redis pools never appear in
call db pool — that command shows database pools only. The non-coroutine path has no pool
and is not reported.
active sitting at maximum while borrowers wait is the signal to raise the ceiling —
or to find the code that borrows and never lets go.
A broken connection
Usually there is nothing to do: ext-redis reconnects transparently — verified both on
an explicit close() and on a server-side CLIENT KILL, with the selected database and
the authentication restored on its own.
When a command fails and the connection is in doubt, it can be reported:
use Flytachi\Winter\Redis\RedisPool;
try {
$store->set('key', 'value');
} catch (\RedisException $e) {
RedisPool::reportFailure(MainRedisConfig::class, $e);
throw $e;
}The verdict comes from a probe, not from reading the error text: the pool sends one
PING. If it answers, the connection is alive, the fault was in the command, and the
connection stays in the pool. If it does not, the connection is evicted and the next
command gets a new one. The failed command is not retried: the pool cannot know
whether the server had already applied it, and a retry could double a write.
A connection outside the pool
Some commands occupy their connection for as long as they run: BLPOP, BRPOP,
SUBSCRIBE, MONITOR. Issuing one on a pooled connection holds a pool slot for exactly
that long, and a few consumers will empty the pool while Redis itself sits idle.
use Flytachi\Winter\Redis\RedisPool;
$config = RedisPool::dedicated(MainRedisConfig::class);
$redis = $config->connection();
$redis->setOption(\Redis::OPT_READ_TIMEOUT, -1); // let the blocking read wait as long as it needs
$redis->subscribe(['events'], $handler);
$config->disconnect();dedicated() opens a connection that is never shared and never returned: the caller
owns it. Such a connection does not appear in stats() — the pool knows nothing about it
and its ceiling does not cover it, so counting them is on you.
Which error means what
| Exception | When | What to do |
|---|---|---|
RedisPoolException |
no live connection was obtained: the pool is full, the server is unreachable, or it accepts the socket without serving on it | read the message — it names which; then stats() and getPrevious() |
RedisCommandException |
the server refused the command: wrong key type, a counter that is not a number | a bug in the code, not in the infrastructure |
RedisFeatureException |
the command is newer than the server (hash field lifetimes) | upgrade the server, or make do with expireKey() |
RedisException (driver) |
a dropped connection, a read timeout | the connection is suspect — reportFailure() |
The first three are ours; the last comes from ext-redis unchanged.
What lands in the log
With a logger set, the pool narrates itself at debug level and reports losses at
warning:
DEBUG config registered: Main\Configurations\MainRedisConfig redis://localhost:6379/0
DEBUG pool created: Main\Configurations\MainRedisConfig maxConnections=10
DEBUG slot opened: Main\Configurations\MainRedisConfig redis://localhost:6379/0
DEBUG cid=7 borrow: Main\Configurations\MainRedisConfig
DEBUG cid=7 release: Main\Configurations\MainRedisConfig
WARN cid=9 evict: Main\Configurations\MainRedisConfigSeveral slot opened in a row mean the pool is growing under load; frequent evict
means connections are dying between requests: check whether keepaliveTime has been
switched off, and whether something is cutting them faster than once every two minutes.
Wiring it into the application lifecycle
There is nothing to do. The kernel wires the pool up itself, at startup, when the package is installed:
Kernel::init()
└── DepSupport::has(Dep::Redis)
├── RedisPool::setLogger(...) pool logs go to the 'Redis' channel
├── BeforeFork::register(RedisPool::closeBeforeFork()) the parent closes its connections before a fork
├── ForkReset::register(RedisPool::reset(...)) the child forgets the inherited ones
└── RuntimeShutdown::register(RedisPool::shutdown()) closing at process exit
workerExit
└── RedisPool::shutdown() closing at worker exitYou can confirm it took effect from the log: at debug level the pool starts talking
straight away — config registered, pool created, slot opened.
Why forking is a concern of its own
fork() copies file descriptors, so a connection opened before the fork is physically
the same one in parent and child. Two commands sent into one socket from two processes
break the protocol for both, and the symptom — packets out of order, or a reply to
somebody else’s request — points nowhere near the fork.
So the parent closes its own connections before the fork
(closeBeforeFork()), and the child calls reset(): it forgets the
inherited connections without closing them — closing would tear down the parent’s
connection. The child opens its own on first use.
Winter’s daemons and processes fork, so this registration matters — and the kernel does it for you.
If the application boots without the kernel
The package is standalone: it reaches for none of the framework’s globals, which is exactly what lets it be used without Winter. In such an application the same is written by hand:
RedisPool::setLogger(LoggerFactory::getLogger(RedisPool::class));
// before every pcntl_fork() — in the parent
RedisPool::closeBeforeFork();
$pid = pcntl_fork();
if ($pid === 0) {
RedisPool::reset(); // in the child
}And RedisPool::shutdown() at process exit. That one is always mandatory: housekeeping is
on by default, so a pool that has served a single borrow holds a timer, and a worker cannot
exit while its reactor still holds a repeating one.
RedisPool reference
RedisPool is a static facade over every pool in the process. Application code rarely needs
it: the store hands out the client, and the application’s bootstrap handles closing and
resetting. The methods are described here because diagnostics go through them, and because
two of them do almost the same thing — and mixing those two up is expensive.
Access is static, with no dependency injection: the pool holds process-wide state, and there cannot be a second instance of it.
store()
Returns the client of the current unit of work.
Under Swoole it borrows a connection from the pool on the first call within a coroutine,
caches it in the coroutine’s context and registers a defer that returns it when the
coroutine ends. Without Swoole it hands out the process’s single connection. This is the
method RedisStore::raw() calls.
Syntax
public static function store(string $configClass): \RedisParameters
$configClass — the configuration class, MainRedisConfig::class.
Returns
A \Redis client already bound to the current coroutine. There is no need to release it by
hand.
Errors
RedisPoolException — if the borrow never reached a live connection within
poolWaitTimeout: everything busy, a connection that could not be opened, or a window
spent on dead connections (the server accepts the socket but does not serve on it). The
real cause is in getPrevious().
config()
Returns the registration instance of a configuration.
For reading settings or checking reachability without occupying a pooled connection: the
address, the database index, ping().
Syntax
public static function config(string $configClass): RedisConfigInterfaceParameters
$configClass — the configuration class.
Returns
The configuration instance; created once per class and reused.
The instance from `config()` is not the one in the pool
It exists so settings can be read. The connections are held by other instances — one per pool slot, and those are the ones the pool probes and closes.
dedicated()
Opens a new connection outside the pool and hands ownership to the caller.
For blocking commands: SUBSCRIBE, BLPOP, XREAD BLOCK occupy a connection for the whole
wait, and holding a pooled one for that would take it away from ordinary requests.
Syntax
public static function dedicated(string $configClass): RedisConfigInterfaceParameters
$configClass — the configuration class.
Returns
A new configuration instance with its connection already open. Closing it is the caller’s job.
For queues this is already done: $list->consume() takes
such a connection itself and releases it on close().
stats()
Returns the utilisation of every pool.
Counted for the current worker: each worker has its own pool, and a request sees the one that served it.
Syntax
public static function stats(): arrayParameters
None.
Returns
An array keyed by configuration class, each value four numbers: total (open in all), idle
(free), active (handed out) and maximum (the ceiling).
Without Swoole the array is empty: there are no pools there.
showConfigs()
Returns every configuration registered in the process.
A diagnostic tool: the list shows the connection points this process has already used,
and each one can be polled with
pingDetail(). The built-in
/actuator/health does not rely on this list: it finds every configuration of the project by
scanning and checks fresh instances.
Syntax
public static function showConfigs(): arrayParameters
None.
Returns
An array of configuration instances keyed by class name. Only the ones already used appear in it: registration is lazy.
reportFailure()
Reports to the pool a failure that happened on a borrowed connection.
The method decides whether the connection is to blame: a lost connection is evicted and the next request gets a new one; a refused command means the connection is healthy and nobody touches it.
Syntax
public static function reportFailure(string $configClass, \Throwable $error): boolParameters
$configClass — the configuration whose connection failed.
$error — the exception exactly as the driver threw it.
Returns
true if the error was classified as a connection loss and the connection was evicted;
false if the server is healthy.
setLogger()
Sets where the pool writes its events.
Syntax
public static function setLogger(LoggerInterface $logger): voidParameters
$logger — any PSR-3 logger. By default the pool stays silent.
shutdown()
Closes every pool and connection the process owns.
Called when a worker stops. Besides the sockets it releases the housekeeping timer: a live timer keeps the worker’s reactor from draining.
Syntax
public static function shutdown(): voidcloseBeforeFork()
Closes the process’s connections right before fork(), so the child has nothing to
inherit.
Called in the parent. A fork copies the client together with its socket, and a child that
goes on to use Redis would send its commands into the parent’s session — CLIENT ID is the
same on both sides — and two processes writing into one socket corrupt the protocol for both.
Closing the connections first leaves the child nothing to share; both reconnect on their next
command.
It covers connections outside coroutines — the only kind a forking process has.
Syntax
public static function closeBeforeFork(): voidErrors
RedisPoolException when the client is in MULTI or pipeline(): closing it would throw away
the commands its owner queued, so the fork is refused.
The kernel calls this itself
Before every fork() the framework makes — spawn() without Swoole, starting a daemon
worker, the application (call run) starting each of its components, and its fork of
winter-sentinel — closeBeforeFork() is called automatically.
reset()
Forgets every pool and connection without closing them.
Called in a child process straight after fork(). The difference from shutdown() is
fundamental: a fork copies file descriptors, so a socket inherited from the parent is
physically the same one — close it in the child and you tear down the parent’s connection.
Usually there is nothing to forget: the parent calls closeBeforeFork()
first.
Syntax
public static function reset(): void