Redis · Lists

Lists

A list is an ordered sequence of strings under a single key. In applications it is almost always a job queue or a log of recent events. Below: what the structure is, where it fits, and every method in detail with arguments and examples.

Package flytachi/winter-redisClass RedisListMethods 18

What a list is and why

The problem. Work that cannot be done right now has to wait somewhere. An email, a report, an export — anything longer than a request is set aside for somebody else to pick up. Keeping the queue in the process’s memory will not do: there are several workers, and a restart takes everything accumulated with it. Keeping it in the database means polling it in a loop and working out at every step which worker already took which row.

The solution. A Redis list is a sequence shared by everyone, with fast operations at both ends. A producer adds at one end, a consumer takes from the other, and Redis is single-threaded: the removal is atomic, so one job never goes to two workers.

php

$jobs = $store->list('jobs');

$jobs->push(json_encode(['type' => 'email', 'to' => $email]));   // the producer

$job = $jobs->consume(timeout: 5);                               // the consumer

Properties of the structure

A Redis list is a doubly linked list of strings, not an array. Everything else follows from that:

Adding and removing at either end is O(1), regardless of length. A million elements do not make push slower.
Access by index is O(N): to reach the middle Redis walks the elements. at(500000) really does walk half a million nodes.
Order comes from insertion, not from value. A list is not sorted and allows duplicates.
Length goes up to 4,294,967,295 elements.

The key exists for exactly as long as it holds at least one element: removing the last one removes the key too. The reverse holds as well — a push into a key that does not exist creates it. There is no separate “create a list” step and nothing to do it with.

Elements are always strings

A list stores bytes. Arrays and objects go into it only if the configuration has a serializer; otherwise an array turns into the string "Array", and only a PHP warning mentions it.

Where it is normally used

A job queue. The most common case by far. The producer adds at the tail, the consumer takes from the head — that is FIFO.

A log of the last N events. A list plus a length cap: an audit trail of a user’s actions, recent errors, an activity feed. Old entries fall off by themselves.

A buffer between the fast and the slow. A request handler drops a record in, a background worker takes them in batches. A spike does not take the slow part down, because the list accepts faster than the worker drains.

A stack. Adding and taking at the same end — LIFO. Less common than a queue.

And where a list is the wrong choice:

Task	Why not a list	What instead
Check whether an element is present	O(N) over the whole length	a set (`SET`)
Store unique values	a list allows duplicates	a set
Keep things ordered by value	order comes from insertion only	a sorted set (`ZSET`)
Several consumers, each getting every message	an element goes to one of them	a stream or pub/sub
History and replay are needed	a removed element is gone	a stream (`XADD`)
Frequent reads from the middle	O(N) on every access	a hash or another structure

Reference

`RedisList`

RedisList is a handle on one list: the object through which commands against that key are run. It is taken from the store:

php

$jobs = $store->list('jobs');    // the server will see 'queue:jobs'

list() executes nothing — it is a view, not a request. The prefix is applied once, here, so there is nowhere left to forget it. The handle holds no connection either: every call asks the store for the client — except consume(), which has a connection of its own.

The handle can be created per call, but for a consumer loop it is better kept: then the blocking reads reuse one connection instead of opening a new one.

Adding

`push()`

Adds an element at the tail of the list.

That is the end pop() does not take from — which is why push() + pop() gives a queue: first in, first out.

Syntax

php

public function push(mixed $value, ?int $cap = null): int

Parameters

$value — what to put in. A string or a number; arrays and objects need a serializer.

$cap — the maximum length of the list. null by default — no cap. With a value set, the list trims itself, keeping the last $cap elements.

Returns

The length of the list after the push. With a $cap — the length after trimming, so never more than $cap.

Example

php

$jobs->push(json_encode(['type' => 'email', 'to' => $email]));   // 1
$jobs->push(json_encode(['type' => 'sms']));                     // 2

A log of the last N

php

foreach ($lines as $line) {
  $audit->push($line, cap: 1000);      // only the last thousand are kept
}

$audit->count();   // 1000, however many lines arrived

The cap is applied in one transaction (MULTI: push, trim), so the list is never for a moment longer than $cap — and does not stay longer if something breaks between the two commands.

`pushFront()`

Adds an element at the head of the list — the end pop() takes from.

Needed in two cases: building a stack, and putting a job at the front of the queue.

Syntax

php

public function pushFront(mixed $value, ?int $cap = null): int

Parameters

$value — what to put in.

$cap — the maximum length. null by default. Trimming keeps the first $cap elements — which, since you are pushing at the head, means the newest ones.

Returns

The length of the list after the push and the trim.

Example

php

// a stack: push and take at the same end
$undo->pushFront($action);
$undo->pop();            // the last action

// a priority job — to the front of the queue
$jobs->pushFront($urgentJob);

Removing

`pop()`

Takes an element from the head and removes it from the list.

The operation is atomic: however many workers call pop() at once, each element goes to exactly one of them.

Syntax

php

public function pop(): mixed

Parameters

None.

Returns

The element, or null if the list is empty or does not exist. The driver answers false there; null was chosen so “empty” does not get confused with a stored false.

Example

php

while (($job = $jobs->pop()) !== null) {
  $this->handle($job);      // drain what has piled up, then leave
}

A removed element exists only in the worker's memory

If the worker dies between pop() and the end of processing, the job is gone — and there is nowhere to learn that from. When that is unacceptable, move instead of removing: moveTo().

`popBack()`

Takes an element from the tail and removes it from the list.

Syntax

php

public function popBack(): mixed

Parameters

None.

Returns

The element, or null if the list is empty or does not exist.

Example

php

$jobs->push('a');
$jobs->push('b');

$jobs->popBack();    // 'b' — the last one added
$jobs->pop();        // 'a' — the first one added

Waiting

`consume()`

Waits for an element and takes it from the head.

If an element is already there it returns straight away — the wait only begins on an empty list. This is the consumer’s main tool: it lets a worker sleep until there is work instead of polling the list in a loop.

Syntax

php

public function consume(float $timeout = 0.0): mixed

Parameters

$timeout — how many seconds to wait. 0.0 by default — wait indefinitely.

Returns

The element, or null if the time ran out and the list stayed empty.

Example

main/Daemons/JobWorker.php

$jobs = $store->list('jobs');

while (!$this->stopping) {
  $job = $jobs->consume(timeout: 5);

  if ($job !== null) {
      $this->handle($job);
  }
  // null — the timeout simply expired; check the stop flag and wait again
}

$jobs->close();

The timeout is not there to avoid “waiting too long” but so that the loop periodically regains control — otherwise the daemon never notices a stop signal.

`consume()` does not take a connection from the pool

A blocking command occupies its connection for the whole wait. Ten consumers with timeout: 30 on a pool of ten connections would hold the entire pool for half a minute, and every other request would fail with “no free connection” — while Redis sat idle, and the whole thing looked like “Redis is slow”.

So the handle opens a connection of its own on the first consume() and reuses it on the next ones. That is exactly why the handle is worth keeping for the whole loop: a new handle means a new connection.

The read timeout is raised automatically

The connection is set up to throw read error on connection if the server stays silent longer than readTimeout (2 seconds by default), and a blocking read is precisely “the server is deliberately silent”. So consume() raises the read timeout for the duration of the wait. Without that, consume(timeout: 5) would die on the second second — and not on a timeout but with a connection error.

`close()`

Closes the connection consume() opened.

Always safe to call: if consume() was never used the method does nothing. Losing the last reference to the handle closes the connection without an explicit call, but in a long-running daemon that is not worth relying on.

Syntax

php

public function close(): void

Parameters

None.

Returns

Nothing.

Moving

`moveTo()`

Atomically moves an element from the head of this list to the tail of another.

That is, it takes from where pop() takes and puts where push() puts. This is the basis of a queue that does not lose jobs when a worker dies: at any moment the element is in exactly one of the two lists.

Syntax

php

public function moveTo(self $target): mixed

Parameters

$target — the list to move into. It must belong to the same configuration.

Returns

The moved element, or null if the source list is empty.

Errors

LogicException — if the lists belong to different configurations.

Example

php

$jobs       = $store->list('jobs');
$processing = $store->list('jobs:processing');

$job = $jobs->moveTo($processing);

if ($job !== null) {
  $this->handle($job);
  $processing->remove($job);      // acknowledge: handled
}

What to do with the jobs left in processing after a worker died — put them back into jobs, raise an alert, park them as dead — is the application’s decision. The package provides the tool and imposes no policy: only the job’s author knows the deadlines and the cost of a retry.

Within one configuration only

LMOVE runs on a single connection, so both keys have to live on one connection point. Moving into a list of another configuration is refused with an exception — without that check the element would be written into our database under someone else’s name, and everything would look like it worked.

Moving into itself

The element travels from the head to the tail, so the list rotates. That is how a queue is walked in a circle without losing anything:

php

$workers->moveTo($workers);    // 'a','b','c' → 'b','c','a'

Reading

Nothing in this section removes anything.

`count()`

Reports the length of the list.

Syntax

php

public function count(): int

Parameters

None.

Returns

The number of elements; 0 for a key that does not exist. An O(1) operation: Redis stores the length rather than counting it.

Example

php

if ($jobs->count() > 10_000) {
  $this->alert('the queue is growing faster than it drains');
}

`range()`

Returns a slice of the list without changing it.

Syntax

php

public function range(int $start = 0, int $stop = -1): array

Parameters

$start — the index of the slice’s first element. 0 by default.

$stop — the index of the last element, inclusive. -1 by default.

Both indexes may be negative — counted from the tail: -1 is the last, -2 the one before. The defaults 0, -1 mean “the whole list”. Bounds beyond the length are not an error: the slice simply comes out shorter or empty.

Returns

An array of elements; an empty array for a key that does not exist.

Example

php

$events->range();          // everything
$events->range(0, 9);      // the first ten
$events->range(-5, -1);    // the last five
$events->range(100, 200);  // [] — if there are fewer elements

`$stop` is included in the result

This differs from array_slice(): range(0, 9) returns ten elements, not nine. That is how the LRANGE command itself works.

`at()`

Returns the element at an index without removing it.

Syntax

php

public function at(int $index): mixed

Parameters

$index — the position. A negative one is counted from the tail: -1 is the last.

Returns

The element, or null if there is no such index or no such key.

Example

php

$jobs->at(0);     // the next job, without taking it
$jobs->at(-1);    // the last one added
$jobs->at(999);   // null

Remember the O(N): cheap at the ends, expensive in the middle of a long list.

Changing

`remove()`

Deletes elements equal to the given value.

The comparison is byte for byte, as the server sees them. The main use is acknowledging work in tandem with moveTo().

Syntax

php

public function remove(mixed $value, int $count = 1): int

Parameters

$value — what to delete.

$count — how many occurrences to delete and which end to search from. A positive number means from the head, a negative one from the tail, 0 means every occurrence. 1 by default.

Returns

The number of elements actually deleted.

Example

php

// the list: a, b, a, c, a
$list->remove('a');            // 1 → b, a, c, a   (the first from the head)
$list->remove('a', -1);        // 1 → b, a, c      (the last from the tail)
$list->remove('a', 0);         // 1 → b, c         (everything left)

Acknowledging a job

php

$processing->remove($job);     // exactly one copy of this job

`trim()`

Keeps only the given range, deleting everything else.

For “the last N” a separate trim() is usually unnecessary — push() has a cap argument that does the same thing atomically along with the push.

Syntax

php

public function trim(int $start, int $stop): bool

Parameters

$start — the first index to keep.

$stop — the last index to keep, inclusive.

The indexes work as in range(): negative ones are allowed and counted from the tail.

Returns

true if the command went through.

Example

php

$events->trim(0, 999);      // keep the first thousand
$events->trim(-100, -1);    // keep the last hundred

A range beyond the length empties the list

trim(5, 10) on a list of two elements deletes both, and the key stops existing — Redis does not keep empty containers. This is not an error and nothing reports it, so the bounds are worth computing rather than guessing.

`set()`

Overwrites the element at a given position. Does not change the length.

Syntax

php

public function set(int $index, mixed $value): bool

Parameters

$index — the position. A negative one is counted from the tail.

$value — the new value.

Returns

true on success and false if the index does not exist. There is deliberately no exception: the list knows its own length, and going past it is an ordinary answer, not a failure.

Example

php

$jobs->set(0, $patchedJob);    // true
$jobs->set(999, $x);           // false, the list is unchanged

The key as a whole

List elements have no lifetimes of their own — unlike hash fields. The lifetime belongs to the key.

`expireKey()`

Gives the whole list a lifetime.

Syntax

php

public function expireKey(int $seconds): bool

Parameters

$seconds — in how many seconds to delete the key entirely. Counted from now, not up to a moment: expireKey(time() + 3600) asks for fifty-six years and says nothing about it.

Returns

true if the lifetime was set; false if there is no key.

Example

php

$draft->push($chunk);
$draft->expireKey(3600);     // the draft lives for an hour

`keyTtl()`

Reports how long the list has left to live.

Syntax

php

public function keyTtl(): ?int

Parameters

None.

Returns

The remaining seconds, or null — both when no lifetime is set and when there is no key.

`deleteKey()`

Deletes the whole list.

Syntax

php

public function deleteKey(): bool

Parameters

None.

Returns

true if the key existed.

Housekeeping

`name()`

Returns the key’s name with the prefix — the one the server sees.

Needed when working with the list through raw(): the prefix is not applied there.

Syntax

php

public function name(): string

Parameters

None.

Returns

The full key name.

Example

php

$store->raw()->lPos($jobs->name(), $needle);     // a command the handle does not wrap

`configClass()`

Returns the configuration class the list lives on.

moveTo() uses it too, to refuse moving an element between different connection points.

Syntax

php

public function configClass(): string

Parameters

None.

Returns

The fully qualified name of the configuration class.

Mapping to Redis commands

The method names describe intent; here is what goes to the server.

Method	Command
`push(value)` / `push(value, cap)`	`RPUSH` / `MULTI(RPUSH, LTRIM)`
`pushFront(value)` / `pushFront(value, cap)`	`LPUSH` / `MULTI(LPUSH, LTRIM)`
`pop()` / `popBack()`	`LPOP` / `RPOP`
`consume(timeout)`	`BLPOP`
`moveTo(list)`	`LMOVE`
`count()` / `range()` / `at()`	`LLEN` / `LRANGE` / `LINDEX`
`remove()` / `trim()` / `set()`	`LREM` / `LTRIM` / `LSET`
`expireKey()` / `keyTtl()` / `deleteKey()`	`EXPIRE` / `TTL` / `DEL`

Commands not here — LPOS, LINSERT, RPOPLPUSH, BLMPOP, LMPOP — are reachable through raw(), remembering name().

Hashes — fields and their own lifetimes
Stores — strings, counters, raw() and transactions
Streams — when history and delivery tracking are needed
Connection pool — why consume() stays out of the pool

Lists

What a list is and why

Properties of the structure

Where it is normally used

Reference

RedisList

Adding

push()

pushFront()

Removing

pop()

popBack()

Waiting

consume()

close()

Moving

moveTo()

Reading

count()

range()

at()

Changing

remove()

trim()

set()

The key as a whole

expireKey()

keyTtl()

deleteKey()

Housekeeping

name()

configClass()

Mapping to Redis commands

Next

`RedisList`

`push()`

`pushFront()`

`pop()`

`popBack()`

`consume()`

`close()`

`moveTo()`

`count()`

`range()`

`at()`

`remove()`

`trim()`

`set()`

`expireKey()`

`keyTtl()`

`deleteKey()`

`name()`

`configClass()`