Decision records
ADR-0036: State with several scalwsd instances on one host
Status: accepted (design, D0) — 2026-10-07
Every piece of mutable state, classified, with the rule that keeps behaviour correct when N instances serve the same configuration.
| State | Class | Rule |
|---|---|---|
| Response cache (memory) | per-instance | Acceptable: each instance fills its own; correctness unchanged (lower hit ratio at first). Purge/invalidation via admin API must be fanned out to every instance by the controller. |
| Response cache (disk tier) | coordinated | Each instance gets its own subdirectory (<dir>/<instance-id>); sharing one directory is not supported. |
Open file cache, .htaccess cache | per-instance | Already identity/TTL checked against the filesystem; no change. |
| TLS sessions / tickets | per-instance | Resumption may miss across instances; ticket keys shared later only if measured worthwhile. |
| Certificates, ACME | coordinated | Only the controller (or one designated instance) orders/renews; instances load from the shared state directory and reload on change. |
| Per-IP rate and connection limits | coordinated | Must not multiply with N. MVP: the controller passes each instance its share (limit / active instances, rounded up) and updates it on every scale action; documented as approximate under uneven hashing. |
| Tenant quotas (concurrency, rate) | coordinated | Same share rule as per-IP limits. |
| Route rate limits | coordinated | Same share rule. |
| Metrics | per-instance | Each instance exports its own (instance label added by the controller’s aggregation); counters are not summed inside instances. |
| Upstream connection pools | per-instance | Upstream keep-alive connection counts scale with N; bounded by per-instance pool limits. |
| Managed PHP-FPM / Node / Python | coordinated | Not duplicated per instance: one application pool per application, owned by the controller (or the first instance in the MVP); other instances reach it as an external FastCGI/HTTP endpoint. Duplicating application pools is an application-tier scaling decision (D6+), never a side effect of web-tier scaling. |
| Admin socket, metrics address, runtime dir, logs | per-instance | Derived from the instance id; no two instances bind the same path or port (except the reuseport listeners). |
| Temp files | per-instance | Under the instance runtime directory. |
No global shared-memory locks are introduced to make caches identical.
D1 validates the rules that exist today (separate dirs/ports) and measures what the per-instance caches cost; the share rules for limits and quotas are implemented before any automatic scaling (D4) — scaling must never bypass a limit.