Decision records

ADR-0041: HTTP/3 graceful lifecycle (single and multiple instances)

Status: accepted (design, D7) — 2026-10-07

Context

The HTTP/3 accept loop closed the whole QUIC endpoint (Endpoint::close) as soon as the shutdown signal arrived. That sends CONNECTION_CLOSE on every connection at once: the per-connection GOAWAY and the wait for in-flight requests never got a chance, so every restart and every drain cut H3 requests — also with a single instance. Multi-instance operation (ADR-0039) adds scale-out/in, demux registration and Alt-Svc timing.

Decision

Drain of an instance (SIGTERM, scale-in, rolling reload):

1. stop placement of new connections on it: the instance tells the demux draining (multi-instance) or stops accepting incoming connections on its endpoint (single);
2. keep receiving for its existing connections (the demux keeps routing its server id);
3. send HTTP/3 GOAWAY on every connection (no new requests; in-flight ones continue);
4. finish accepted requests;
5. when a connection has no request left, close it with H3_NO_ERROR;
6. after h3_drain_timeout (default: the server’s shutdown_timeout) close what is left with H3_NO_ERROR and count it as forced;
7. wait until the endpoint is idle (close frames delivered, bounded) and exit.

An established connection is never redirected to another instance.

Scale-out: start → endpoint with the controller’s sockets and keys → register with the demux (not yet placeable) → readiness → activation makes it placeable. Existing connections stay where they are.

Alt-Svc: an instance advertises h3 only while its QUIC endpoint can take new connections: single instance — endpoint open; multi-instance — registered with the demux and not draining. When HTTP/3 is removed from the configuration, advertising stops first, existing QUIC connections drain, the UDP socket closes last (ADR-0037).

Crash of an instance: its server id disappears from the demux; datagrams for it get a stateless reset from another instance; clients reconnect (new placement).

Crash of the demux: the controller restarts it with the same UDP sockets; instances reconnect and register again (with their draining state); datagrams queued in the kernel socket meanwhile are delivered afterwards, the rest are retransmitted by QUIC.

Controller restart: adopts demux and instances (pid + start time + executable, like instances, ADR-0035); asks the demux for its live registrations to rebuild the placement view.

Metrics

Per instance: active QUIC connections, H3 streams in flight, GOAWAYs sent, drain duration, graceful and forced closes, handshakes; per demux: datagrams in/out per instance, placements, unroutable datagrams, resets requested, drops (full instance socket).