Internals: benchmark reports

ScalWS 1.0.0 on bare metal: the installed package with three warm instances (2026-11-12)

The first measurement of a release as it is installed: the 1.0.0 .deb on the Xeon host, scalws-controller running three scalwsd instances of two CPUs each, started before the measurement (no autoscaling), with the service sandbox on (what a new install gets). nginx and OpenLiteSpeed run on the same six CPUs. A second part measures scaling of the instances themselves against rc.1-11.

Comparison with nginx and OpenLiteSpeed

Host: 2x Xeon E5-2650 v2 (no AVX2), 121 GB, Ubuntu 26.04, kernel 7.0.0-38, performance governor, turbo off, no VM or container running. Server on CPUs 0-5 (socket 0), the upstream, PHP-FPM, Node.js and uvicorn on 6, 7, 22, 23, oha 1.16.0 on 8-15 and 24-31 (socket 1), 64 connections. bench/multi.sh MODE=release: 10 s warm-up, 30 s measurement, 5 runs with rotating order; medians. Every measurement of every server answered 2xx only; no run was excluded.

• ScalWS: scalws_1.0.0-1_amd64.deb (the published package) under its systemd unit (SERVERS=scalws-service of bench/run.sh): instances: {min: 3, max: 3, cpus_per_instance: 2, cpus: "0-5"}, sandbox: on, the bench sites of bench/configs/scalws.yaml with the applications as the external processes the other servers use (the controller requires external endpoints). “own HTTP/1” adds http1_server: experimental and upstream_client: experimental (ADR-0046, ADR-0045); “defaults” is the package as installed.
• nginx 1.31.6 (built from the signed source tarball; every ratio is against it) and 1.28.3 (Ubuntu package), worker_processes 6; OpenLiteSpeed 1.9.3, 6 workers.
• LiteSpeed Web Server Enterprise is not in this run: its trial licence has run out.
• The “defaults” and the nginx and OpenLiteSpeed columns are one session (09:27 to 11:32 UTC); “own HTTP/1” was measured afterwards with the same harness (13:05 to 13:38 UTC). nginx 1.31 measured again in between (diagnosis below) was within 2 % of this table.

Requests per second:

OpenLiteSpeed has no fixed-response directive: respond is not measured for it.

p99 latency (ms, median of 5 runs):

Server CPU per request (µs, median of 5 runs; ScalWS: the controller and its instances, the unit’s whole cgroup):

Peak RSS: ScalWS 109 to 120 MiB (controller and three instances together), nginx 103 to 131 MiB, OpenLiteSpeed 55 to 70 MiB.

Findings:

• ScalWS leads on the cached response path (1.54 to 1.70x nginx 1.31), HTTPS/2 (1.27 to 1.29x) and PHP (1.06x), with the lowest CPU per request on the first two; on PHP it uses more CPU per request than nginx (116 against 93 µs) while serving more requests per second.
• With the package defaults, respond (0.86) and proxy (0.83) are behind nginx; with ScalWS’s own HTTP/1 server and client they are at 0.93 and 0.98, and static-small at 1.03.
• With the own client, Python (0.90) and Node.js (0.96) are slower than with the default client (0.97, 0.98): to be understood before the own paths become the defaults (1.1, P10).

Where the default gap comes from

Same host and harness, SCENARIOS="respond proxy static-small", 3 rotating rounds of 10 s

• 30 s, medians (results/diag-100, diag-100b):

The controller, the number of instances and the sandbox move these by 1 to 3 %; scalwsd alone with the default paths has the same gap. The gap is the default HTTP/1 server and client (hyper’s); the own implementations close most of it.

Instances: three against one (bench/instances.sh)

Instance i on CPU i, one event loop each, PHP-FPM and the upstream on CPU 3, oha on socket 1; 3 runs of 10 s warm-up + 20 s, medians; scalwsd 1.0.0 against rc.1-11 (the 1.0.0 binary of the package; rc.1-11 the published 1.0.0~rc.1-11 package). Zero non-2xx answers.

• Three instances serve three times one instance (efficiency 0.98 to 1.04), and slightly more than one process with three event loops on the same CPUs.
• PHP is bound by the single CPU of PHP-FPM in this topology: equal for every count and build. On one instance 1.0.0 is 1.17x rc.1-11 on PHP.
• 1.0.0 needs 25 to 30 % less CPU per request than rc.1-11 on the web paths (the profile-guided build of ADR-0047 and the 1.0.0 fixes).

Raw results, in the bench host’s results directory: results/xeon-1.0.0-cmp, xeon-1.0.0-exp, xeon-inst-1.0.0, xeon-inst-rc1-11. (scalwsd --version in the environment records shows an older build first on the host’s PATH; the service runs the package’s /usr/bin/scalwsd 1.0.0.)