Benchmarks

Release-mode comparison with nginx and OpenLiteSpeed, and soak — 2026-10-11

Machine and pinning as in 2026-10-10-xeon-bare-metal-pgo.md: 2× Xeon E5-2650 v2, server on four physical cores (CPUs 0-3), upstream and applications on 4-7, oha on socket 1; performance governor, turbo off. MODE=release: 10 s warm-up, 30 s measurement, 5 runs, medians. scalws: the PGO + BOLT build of tools/pgo-build.sh (ADR-0047, commit 9eaebe7); final with both experimental settings, final-hyper with defaults. OpenLiteSpeed 1.9.3 (official release tarball, SHA-256 checked) with bench/configs/openlitespeed; it has no fixed-response directive, so respond is not measured for it. Every measurement of every server answered 2xx only.

Server CPU per request (µs, cpu_us): static-small 36.6 / nginx 1.31 41.3 / OLS 40.8; proxy 44.5 / 43.5 / 57.6; respond 20.3 / 18.7 / —.

Versions: nginx 1.28.3 (Ubuntu 26.04 package) and 1.31.6 (built from the signed source tarball); load: oha with 64 connections.

Peak resident memory of the server processes (MiB, median of 5 runs, peak_rss_mib):

Open points:

• respond (0.92×) and proxy (0.98× in release mode; 1.09× in the quick runs of 2026-10-10) remain behind nginx 1.31.
• php under scalws varies between runs (17.2k–36.8k): in the slow run the PHP-FPM workers used as much CPU as in the fast ones for half the requests; the externally started pool behind nginx is steady (CV 0.8 %).
• python (application-bound, two uvicorn workers): 7 % behind nginx with the same application CPU.
• node: OpenLiteSpeed is 8 % ahead of both scalws and nginx.

Follow-up on the open points (same day)

• proxy / respond. With PGO + BOLT, instruction-cache misses per respond request equal nginx’s (1 956 vs 1 960); L1 data-cache misses remained higher (1 111 vs 677), 10 % of them in memmove copying the ~3 KiB pipeline future into a fresh allocation per request. The own HTTP/1 server now keeps that future in the connection’s state (RequestService::call_inline, commit 989f486). A/B, 3 quick runs: proxy 88 617 → 95 093 req/s (CPU 45.0 → 41.9 µs/request; nginx 1.31 above: 91 672), static-small +3 %, respond unchanged (~0.91× nginx 1.31; the remaining misses are spread over the pipeline, no single cause).
• php variance — the machine, not scalws. In slow runs instructions and cycles per request are unchanged for scalws and for PHP-FPM, but both need twice the time: the cores of socket 0 ran at 1 197 MHz instead of 2 593 MHz (/proc/cpuinfo during the run; 1 795 MHz in a medium run). The firmware (HP ProLiant power regulator, PCC) lowers the clock under partial load despite the performance governor, also with intel_pstate in active mode and min_perf_pct=100. Scenarios that saturate the server cores are not affected; application-bound ones (php, node, python) are. Fix: BIOS power profile “Maximum Performance” / Power Regulator “Static High Performance”. Until then the php, node and python numbers above carry this noise.
• php, scalws’s own cost. fastcgi-client 0.12.0 zeroed a 64 KiB buffer for the params and again for stdin of every request (~15 % of scalws’s cycles on the PHP path). The crate is vendored with that buffer starting at 4 KiB (third_party/fastcgi-client): 197k → 167k cycles per PHP request.
• python. uvicorn receives the same request headers from scalws and nginx and makes the same syscalls per request (3 sendto, 1 recvfrom). scalws sets FORWARDED_ALLOW_IPS=* for managed uvicorn (applications see the client address); an A/B behind nginx showed no measurable cost. The same nginx + uvicorn setup measured ~5 950 req/s in the follow-up versus 6 534 in the release run, so the 7 % difference is within the frequency noise above; to be re-measured after the BIOS change.

Soak (gate 6, part 1)

bench/soak.sh with the same build and both experimental settings, all ten scenarios at once (16 connections each, including HTTP/3).

• 8 h overnight (2026-10-10/11): PASS — 789 M requests in nine scenarios without a single error or non-2xx answer; RSS 63.7 MiB at the baseline and at the end (slope -0.23 MiB/h); 312 descriptors throughout. The HTTP/3 load generator was OOM-killed at the end (oha keeps every result in memory; it had grown to 35 GB), so HTTP/3 had no result. soak.sh now runs the load in 30-minute segments.
• 2 h (2026-10-11), segmented: PASS — 285 M requests, 0 errors, 0 non-2xx, HTTP/3 76.7 M requests (10.6k/s, p99 5.5 ms); RSS 59.7 → 62.4 MiB, descriptors 321 → 337.

Results: xeon:[private host path],xeon-release-ols,soak-20261010T202747Z,soak-h3-20261011T071756Z}.

Re-measurement after the BIOS change (same day, evening)

BIOS power profile set to maximum performance; the clock log (socket-0 cores every 5 s during the whole run) shows 2 593–2 600 MHz under load (28 of 7 680 samples below 2.4 GHz, all between measurements). scalws build: commits 989f486 and 59832b5 (handler future in the connection’s state, FastCGI buffer fix), PGO + BOLT; final with both experimental settings, final-hyper with defaults. MODE=release, 5 runs, medians; every response 2xx. Results: xeon:[private host path].

With a steady clock the PHP-FPM pool behind nginx doubled (16.7k → 37.0k req/s): the earlier php, node and python numbers of every server were measured at lowered clocks and are superseded by this table. scalws’s php run-to-run spread is 4.1 % CV (was 25 %). Server CPU per request: proxy 41.5 µs (nginx 1.31 43.3), php 61.0 µs (78.4).

Still behind: respond 0.90× nginx 1.31 (= nginx 1.28), python 0.95× nginx 1.31 and 0.92× nginx 1.28 (the gap persists with a steady clock, so it is real), node 0.92× of OpenLiteSpeed (equal to nginx).