Decision records

ADR-0038: Connection distribution — measure pressure, not connection counts

Status: accepted (design, E3) — 2026-10-07

Context

With two instances the busiest one holds 53–62 % of the connections, with the kernel hash (reuseport) and with the shared queue (inherited) alike; long-lived connections stay where they were accepted. Equal connection counts are not a goal: a keep-alive connection that sends one request a minute costs nothing, an HTTP/2 connection with 100 streams costs a core.

Decision

1. Observe work, per instance: accepted connections per second, open HTTP/1, HTTP/2, WebSocket (and later QUIC) connections, requests per second, bytes per second, CPU (from /proc/<pid>/stat), event-loop lag, in-flight requests, p95/p99. The controller already samples most of this for PressureScore (ADR-0034); it adds the per-instance view and an imbalance score:

   imbalance = max(pressure_i) − mean(pressure_i) over serving instances,

   with pressure_i the instance’s PressureScore. Connection-count imbalance is reported next to it, but drives nothing.

2. Harm threshold: imbalance matters only when the busiest instance is saturated (pressure ≥ scale_out_above) and the mean is clearly lower (imbalance ≥ 0.25) for a scale-out window — then adding instances would not help, moving new work would.

3. Mitigation, only if step 2 is seen under realistic load and hurts p99 or throughput, in this order: (a) pressure-biased accept — an instance whose loop lag is high yields new accepts for a moment (inherited queue: it simply does not call accept while saturated, so the idle peer takes the connection); (b) HTTP/2 / WebSocket specific placement. Established connections are never migrated. No TCP eBPF without evidence.

Consequences

• Imbalance is visible in status and metrics; whether it harms anything is decided by measurement (E3 benchmark: skewed long-lived H2/WebSocket mixes).
• Nothing changes on the request path unless the measurement says so.