ScalWSScalWS

    Get started
    ScalWSScalWS
    Get started

    Internals: decision records

    ADR-0049: Upstream pools and run-time members

    Status: accepted, 2026-10-14

    Context

    A proxy runtime took one upstream address. There were no pools of members, no load balancing, no session affinity, no passive health tracking, no retries to another member and no weighted split between backends; a plain load balancer in front of several application servers needed another product. Kubernetes phase 2 makes ScalWS an ingress controller (phase 2b: Ingress, phase 2c: Gateway API). In a cluster, pod addresses change all the time, and the only way to change an address was a full configuration reload, which rebuilds every changed proxy and used to reset per-application rate-limit state. Phase 2a adds the data-plane features both phases need, without any Kubernetes code.

    Decision

    • Pools by name. A top-level upstreams map defines pools (names follow the tenant and application name rule). A proxy runtime uses one with upstream: {pool: name}, or several with backends: [{pool, weight}] (weights 0 to 1000, at most 64, at least one above 0, chosen per request by smooth weighted round robin). upstream and backends are mutually exclusive. The single-address form (http://, https://, unix:) keeps the existing ProxyHandler path: no retries, no ejection, no new allocations per request. preserve_host, the timeouts and websocket stay on the runtime; client is ignored on pool routes with a warning (members are shared between routes, so they use server.upstream_client), and tls_ca_file is refused there (pools have tls).
    • Members. ip:port, [ipv6]:port, host:port (resolved at connect time) or unix:/path (only as the single member of a plain-HTTP pool); weight 1 to 1000; no duplicates; at most 10,000 per pool. Each member has its own connection pool (keepalive idle connections, default 32, at most 1024).
    • members_from. file (default): the configuration’s members. api: the member list is replaced at run time through the admin API; the file’s members only seed a new pool.
    • Selection. round_robin is smooth weighted round robin (nginx). least_conn picks the fewest in-flight requests divided by weight, ties round robin. hash is weighted rendezvous hashing (weight / -ln(u), u from a stable 64-bit hash of key and member), so removing a member moves only its keys; the key is an affinity cookie, a request header or the client address after trusted_proxies. A missing key falls back to round robin.
    • Affinity cookie. The value is the first 16 bytes of HMAC-SHA256 of the member’s canonical address under a random key generated per process, in lower-case hex. It reveals nothing and cannot name an arbitrary member. Attributes: Path=/; HttpOnly; SameSite=Lax, Max-Age from cookie_max_age (0: session cookie), Secure on HTTPS requests. Set only when a member answered and the request did not already hold it; appended next to the member’s own cookies.
    • Passive health. A failure is a connect error or timeout, no response head within response_timeout, a connection broken after the request was written, or a 502, 503 or 504. Failures caused by the client’s request body (over the body limit, a body read timeout, an aborted upload, or an upload still waiting for client data when the response timeout passes) are neutral: no health change, no retry. After failures consecutive failures (default 3, 0 turns it off, at most 100) the member is ejected for cooldown (default 10 s, 1 ms to 1 h), then it is on probation: one success restores it, one failure ejects it again. Results of requests that were in flight when the member was ejected are ignored, so an ejection is never extended or ended by stale results. When every member is ejected, the one ejected longest ago is used: passive health alone never refuses traffic. No active checks.
    • Retries. At most retries (default 1, 0 to 5) further attempts, each on a member not tried yet, and only when the pool has more than one member, nothing was sent to the client, and either the connection failed before the request was written (any method), or the method is idempotent (GET, HEAD, OPTIONS, PUT, DELETE) and the member answered 502, 503 or 504 or timed out. The body must be replayable: bodies with a Content-Length of at most 64 KiB are kept in memory; larger bodies and bodies without a length (also short chunked ones) are sent once. A connection broken after the request was written counts for health but is not retried. WebSocket upgrades go to one member, without retry.
    • Run-time members. PUT /v1/upstreams/{name} with {"members": [...]} on the admin socket or the mTLS listener replaces the list of an api pool: 404 unknown pool, 409 file pool, 400 invalid list (nothing changes), 413 body over 2 MiB, 200 with the pool’s state. It takes the same lock as reloads (Control.lock), so a reload never plans member changes against a set a PUT is replacing. A new member set is built beside the current one and swapped in atomically; members whose canonical address stays are the same objects, with their connections, load and health. Planning (new members’ connectors) and pruning metric series run on the blocking pool while the lock is held; https members of a pool share one TLS client configuration, so the system roots are read once per pool, not per member. GET /v1/upstreams and GET /v1/upstreams/{name} show pools, members, weights and state; these views are not served on the unauthenticated metrics listener.
    • Reload. Pools are matched by name. A pool whose settings (everything except members) are unchanged is kept; a kept file pool gets its new member list planned during prepare and installed after the generation swap, under the reload lock. A pool whose settings changed is rebuilt; an api pool keeps its run-time members in either case (also when it switches from file to api). Per-application rate-limit and quota state is kept when the limit’s definition is unchanged. After a process restart an api pool starts with the file’s members, or empty.
    • Metrics. scalws_upstream_requests_total{pool, member, class}, scalws_upstream_request_duration_seconds{pool}, scalws_upstream_member_ejected{pool, member}, scalws_upstream_pool_retries_total{pool} and scalws_upstream_members{pool}. The retries family is not named scalws_upstream_retries_total: that name already counts stale-connection retries by tenant, application and protocol (ADR-0043). Series of removed members and pools, and of pools a refused reload built, are dropped; member series are bounded by the 10,000 member limit. The access log gains upstream_member and upstream_attempts.
    • Tools. scalwsctl upstreams, upstreams show <pool> and upstreams set <pool> addr[=weight]...; a read-only Upstreams view in the console.

    Consequences

    • The affinity secret is per process: a restart moves every cookie client once, and several instances behind SO_REUSEPORT or scalws-controller do not share affinity, balancing or health state. by: header and by: client_ip give the same member on every instance.
    • Under scalws-controller (listener_mode: inherited or reuseport) the PUT answers 501, on the instances and on the controller’s control socket, because the instances of a generation would disagree about the members; file pools work. The ingress pods of phase 2b run scalwsd alone. Sharing run-time members across instances is left for a later phase.
    • Members of api pools are lost on restart; whatever maintains them must set them again.
    • A request body of up to 64 KiB is buffered when a retry is possible; nothing is buffered for single-member pools or retries: 0.
    • A configuration that keeps the single-address form behaves and performs as before.