Decision records

ADR-0037: Listener generations — listener changes without restarting the controller

Status: accepted (design, E2) — 2026-10-07

Context

With listen: inherited (ADR-0033 addendum) the controller opens the listening sockets once. A configuration whose listeners differ was refused (“restart scalws-controller”), and a restarted controller opened fresh sockets next to the ones its adopted instances still held — two accept queues on one address until the old instances left.

Decision

Listening sockets become part of the configuration generation.

• Socket identity = transport + bound address (tcp 0.0.0.0:443, later udp 0.0.0.0:443, ADR-0039). Listener name, TLS, PROXY protocol, HTTP versions are instance configuration, not socket properties: changing them reuses the socket.
• The controller keeps a socket table identity → fd, generations using it. A generation’s listener set maps listener names to identities.

Reload (new generation), transactional:

1. parse and validate the new configuration (as today);
2. for every identity not in the table, pre-bind a socket (bind + listen) while all current sockets keep serving;
3. if any bind fails (port in use, address not local, permission), close what step 2 opened and reject the reload — the active generation and its sockets are untouched;
4. publish the generation with its listener set; the rolling reload starts instances of the new generation, which receive their generation’s sockets (the handout request names the generation);
5. old-generation instances drain as today; sockets they share with the new generation keep their single queue — no gap, no reset;
6. a socket that no live generation uses any more is closed when the last instance of the generations that used it has exited (it is draining until then);
7. the generation is committed when the rolling reload completed; a failed rolling reload (new instances never ready) leaves the old generation serving, and the new-only sockets are closed.

The old production socket is never closed before the replacement generation is serving.

Controller restart: the new controller rebuilds the socket table from its adopted instances: each instance answers on <runtime_dir>/<id>/listen-handback.sock (0600, peer must be the instance’s own uid) with duplicates of the sockets it holds, by identity. Only identities that no live instance holds are bound fresh. One queue per address survives controller restarts.

Status: status shows per identity: fd state (active / draining), the generations using it, and the last listener-transition error.

Consequences

• Add/remove ports, change bind addresses, add HTTPS or HTTP/3 (with ADR-0039: UDP identities) — all by reload.
• A removed address keeps accepting until its last instance left (bounded by drain_timeout); connections still in its queue when it closes are reset — the cost of removing an address, the same as for one process.
• Changing only TLS/names/options never touches a socket.