Adaptive Runtime

Connection distribution — what is measured and what it means

Design: ADR-0038. Applies to several instances under scalws-controller.

Facts

With the shared accept queue (listen: inherited) and with the kernel hash (listen: reuseport) alike, the busiest of two instances holds 53–62 % of the open connections (bench/listen-modes.sh). Long-lived connections (HTTP/2, WebSocket, keep-alive HTTP/1.1) stay on the instance that accepted them.
With the shared queue an idle instance takes the next connection: connection-heavy traffic gains 8–17 % throughput and 20–28 % lower p99 against the kernel hash.
HTTP/3 connections are placed by the QUIC demux (rendezvous hash over the instances accepting new connections, ADR-0040); they also stay where they were placed.

Equal connection counts are not a goal: a keep-alive connection that sends one request a minute costs nothing; one HTTP/2 connection with 100 active streams can fill a core.

What the controller reports

scalws-controller status → distribution (every second, from each instance’s /metrics and /proc):

Field	Meaning
`instances[]`	per instance: `pid`, `pressure` (PressureScore, ADR-0034), `cpu` (CPUs used), `lag_ms` (event-loop lag), `in_flight`, `rps`, `accepts_per_s`, `connections` (open)
`pressure_imbalance`	max(pressure) − mean(pressure)
`busiest_connection_share`, `busiest_request_share`	share of the busiest instance (0–1)
`harmful`	the busiest instance is saturated (pressure ≥ `scale_out_above`) and the imbalance is ≥ 0.25

When harmful holds for a scale-out window the controller logs “work imbalance: the busiest instance is saturated while the others are not (more instances would not help)”. It does not act on it.

Measured

bench/imbalance.sh (2026-10-07: 8 HTTP/2 connections × 32 streams opened in a burst plus HTTP/1.1 traffic, two instances, 3 runs per mode): busiest connection share 0.58–0.67, pressure imbalance ≤ 0.22, no harmful second in either listen mode. No mitigation is built.

Mitigation policy

Only if harmful is observed under realistic load and costs p99 or throughput (ADR-0038):

pressure-biased accepting — a saturated instance stops taking new connections from the shared queue for a moment, so idle peers take them (no new mechanism, no eBPF);
HTTP/2 / WebSocket-aware placement.

Established connections are never migrated.