Benchmarks
2026-10-08 — Security fixes vs RC1 vs nginx 1.24
Host: development VM (8 vCPU EPYC 7642, Ubuntu 24.04, 31 GiB). Server on CPUs 0-3,
upstreams and applications on 4-5, oha on 6-7. Raw results (every run’s oha JSON,
summary.csv, environment.txt) are in
results/2026-10-08-audit-vs-nginx124/.
Builds:
- fixed —
scalwsdat the end of the security-audit fixes (release,--features http3) - rc1 —
scalwsd 1.0.0-rc.1(before the fixes) - nginx — Ubuntu package 1.24.0, same CPUs, workers = CPUs, access log off
A console demo instance (scalwsd, one worker thread, idle) was running on the host for
all variants alike.
Saturated throughput (MODE=release bench/multi.sh, 5 × 30 s, order rotating)
| Scenario | fixed req/s | rc1 req/s | nginx req/s | fixed / nginx | fixed p99 ms | nginx p99 ms | fixed CPU µs/req | nginx CPU µs/req | RSS MiB fixed / nginx |
|---|---|---|---|---|---|---|---|---|---|
| static-small | 98 035 | 97 842 | 101 317 | 0.97× | 4.28 | 9.55 | 35.8 | 29.8 | 39 / 74 |
| static-1m | 8 784 | 8 861 | 9 323 | 0.94× | 14.57 | 14.46 | 320.8 | 309.7 | 39 / 69 |
| respond | 104 480 | 102 955 | 104 826 | 1.00× | 9.20 | 10.31 | 22.2 | 22.5 | 39 / 69 |
| proxy | 80 390 | 81 937 | 95 167 | 0.84× | 1.60 | 1.42 | 49.4 | 37.6 | 44 / 72 |
| cache-hit | 103 561 | 102 091 | 101 488 | 1.02× | 9.70 | 9.32 | 26.6 | 32.9 | 44 / 72 |
| TLS + HTTP/2 static | 64 201 | 65 682 | 64 289 | 1.00× | 2.41 | 10.24 | 43.5 | 43.4 | 48 / 82 |
| PHP-FPM | 31 979 | 32 804 | 29 476 | 1.08× | 3.22 | 3.27 | 84.7 | 83.6 | 49 / 82 |
| Node.js | 21 380 | 22 053 | 26 407 | 0.81× (see below) | 5.72 | 4.63 | 71.6 | 67.0 | 52 / 82 |
| Python (uvicorn) | 12 051 | 11 932 | 12 133 | 0.99× | 9.65 | 8.37 | 97.0 | 89.5 | 52 / 82 |
Cost of the security fixes (fixed vs rc1): within the run-to-run spread (CV 1–4 %) everywhere; proxy −1.9 %, static-1m −0.9 %, PHP −2.5 % (CV 9.5 %). Two avoidable costs were found by profiling and removed afterwards (see “Follow-up”): the cache stage copied the whole request header map for its variant, and the proxy wrapped bodyless request bodies.
The Node.js gap is a measurement artefact
The application process uses the same CPU with either server (30.4 s), yet nginx got
24 % more requests out of it — which pointed at the requests scalws sends. They are the
same (captured byte-for-byte: same headers, keep-alive, 64 upstream connections, no
connection churn: zero accept calls during load). The difference comes from the state
of the long-lived Node.js process: in run.sh without FRESH_SERVER the managed Node.js
process has been idle through six scenarios and the PHP scenario when it is measured.
With a fresh server per scenario (FRESH_SERVER=1, raw results in
results/2026-10-08-node-fresh-check/):
| scalws req/s | nginx req/s | |
|---|---|---|
| after the PHP scenario (as above) | 21 057 | 27 696 |
| fresh server | 30 139 | 26 696 |
Comparisons from now on use FRESH_SERVER=1.
The proxy gap is real CPU per request
49.4 vs 37.6 µs per request. Profile (perf record -g, proxy load): 63 % kernel (TCP),
33 % scalwsd. In scalwsd the largest single item was CancellationToken::is_cancelled
(1.2 %): every request cloned, checked and dropped the server’s shutdown token, each a
mutex shared by all connections — plus the two copies above.
Follow-up
Fixes being measured: shutdown state read from a per-connection atomic flag, no header
copy in the cache stage, no body wrapper for bodyless proxied requests. Then: nginx 1.30.5
(stable) and 1.31.6 (mainline) built from source, and real applications (WordPress,
Laravel, Express, Django; bench/real/setup.sh).