Users' guide

Sandbox

ScalWS can confine its web server process, scalwsd, with two Linux kernel features: Landlock (which files it may read, write or execute) and seccomp (which system calls it may make). If an attacker ever found a way to run code inside scalwsd through a bug in request handling, the sandbox keeps that code from installing anything, changing ScalWS’s configuration, keys or other sites, starting programs or reaching the kernel’s rarely used interfaces.

Turning it on

server:
  sandbox: on          # off (default) | audit | on

The first install of the package adds sandbox: on to /etc/scalws/scalws.yaml; the packaged file itself has no such line. An upgrade or a reinstall never adds it, and an installation that has no key keeps running without the sandbox until you add it.

The longer form adds directories:

server:
  sandbox:
    mode: on
    read_paths: [/srv/shared]        # extra directories to read (under scalws-controller)
    write_paths: [/var/spool/mysite] # extra directories to write

The setting is read when scalwsd starts; a reload never changes it. Under scalws-controller a configuration change starts new instances, so a changed setting takes effect with the next scalwsctl reload.

When it applies

The sandbox is applied when all of these hold, and otherwise scalwsd logs sandbox on, not applied: <reason> and runs exactly as with off:

• the configuration starts no processes: no managed PHP-FPM, Node.js, Python or container runtimes and no tenant users (tenants[].user). Applications reached as external endpoints (php: {fastcgi: ...}, proxy) are fine. Under scalws-controller, which refuses managed runtimes, and in panel mode the sandbox therefore applies unless the configuration has tenant users, and that case is refused, not skipped (below). Sandboxing configurations with managed runtimes is planned for a later release;
• scalwsd does not run as root. A root scalwsd (a standalone one started by hand or by a custom unit, including setups for eBPF accounting) runs without the sandbox and says not applied: running as root; run as the scalws user (the packaged services do). The packaged scalwsd.service and scalws-controller.service, the container image and the Kubernetes manifests run it as an unprivileged user already. To move a root setup: run it as scalws with CAP_NET_BIND_SERVICE (as packaging/systemd/scalwsd.service does: User=scalws, AmbientCapabilities=CAP_NET_BIND_SERVICE), make the configuration, certificates and document roots readable by scalws, and give it the state, cache and runtime directories (chown -R scalws: /var/lib/scalws /run/scalws); eBPF accounting additionally needs CAP_BPF and CAP_NET_ADMIN in the unit. --service-sandbox still applies to a root scalwsd;
• the kernel has Landlock (Linux 5.13 and later with Landlock enabled: Ubuntu 22.04 and later, Debian 12 and later, EL9 if its kernel enables it, EL10, SUSE 16, Alpine). EL8’s kernel 4.18 has none: there on reports “not available” and changes nothing;
• the machine is x86_64.

Tenant users under scalws-controller need sandbox: off. A standalone scalwsd with tenant users runs without the sandbox and says so. scalws-controller does not start such a configuration with on or audit: the reload is refused, and the running instances keep serving:

server.sandbox is on and this configuration starts processes (tenant web1: tenant user), which a sandboxed instance refuses; tenant users under scalws-controller need `server.sandbox: off` (set it in the configuration file, then reload)

A new install has sandbox: on. To use tenant users there, change that line in /etc/scalws/scalws.yaml to sandbox: off when you add tenants[].user, then run sudo scalwsctl reload.

What it allows

Document roots with root_symlinks: refuse (panel mode sets it) get their read rule without following symbolic links: a root that is, or passes through, a link gets no rule, so the link never widens what the instance reads (that application answers 503, as the server refuses the root itself). No read rule lies in or over scalws-controller’s state directory: a path whose symbolic link leads there, or to a file beside it (which would open the file’s directory), gets no rule, and the check is made again on what the sandbox opens when it applies the rules, so a root swapped for a link after the plan gets none either. A read-only path the kernel refuses as a rule (a link to a namespace file) is skipped with a note like one that cannot be opened: only that application answers 503.

Applications that ScalWS starts never inherit the sandbox; the systemd units (or, on OpenRC, the service sandbox below) still apply to them, and ReadWritePaths= drop-ins for application directories keep working.

Controller restarts

With systemd 254 and later, the tenant cgroups are removed when scalws-controller stops, and a running sandboxed instance cannot use the ones created after that. So a restarted controller replaces sandboxed instances one at a time, with a copy of their configuration, instead of keeping them as it keeps other instances: connections are not dropped, and with every slot up to instances.max taken capacity dips by one instance for the moment. With instances.roll_on_upgrade: false it only logs a warning; run scalws-controller reload then, or tenant limits fail in those instances. On OpenRC the instances stay as they are.

Reloads of a standalone scalwsd

New document roots, sites and certificates reload as before. A reload whose configuration needs a new writable directory outside the ones above is refused, and the old configuration keeps serving:

reload rejected: the sandbox fixes its writable directories at start: /data/cache is outside them; restart scalwsd or add a parent to server.sandbox.write_paths

A reload that would add a managed runtime is refused the same way. Restart scalwsd (systemctl restart scalwsd) to apply such a change.

Audit mode

sandbox: audit applies the same checks but refuses nothing: every system call outside the allowlist is logged by the kernel instead. Landlock has no logging-only mode, so files are not fenced in audit. Use it to try the sandbox on an unusual setup, then switch to on.

Finding a refusal

A refused file access fails with EACCES (“Permission denied”) or EPERM, a refused system call with EPERM or ENOSYS, and scalwsd logs the error with the file name where there is one. The kernel also records refused system calls (seccomp, type 1326):

• with auditd (RHEL family, SUSE): ausearch -m SECCOMP -ts recent;
• without auditd (Ubuntu, Debian, Alpine): dmesg | grep 'type=1326' or the kernel log in the journal (journalctl -k);
• in a container: the host’s kernel log.

Kernels 6.15 and later also log Landlock refusals to the audit log (ausearch -m 1423). With SELinux enforcing, an AVC record (ausearch -m avc) means SELinux refused it; no AVC and EACCES means the sandbox or ordinary file permissions.

Checking it

$ sudo -u scalws scalwsd --config /etc/scalws/scalws.yaml --sandbox-selftest
ok    apply Landlock           ABI 6
ok    apply seccomp
ok    read the configuration   /etc/scalws/scalws.yaml
ok    load the system roots    140 certificates (140 without the sandbox)
ok    read outside the rules   reads not limited (standalone)
ok    write inside the rules   /tmp/.scalws-selftest-4242
ok    ftruncate                refused (EPERM)
ok    open with O_TRUNC        refused (EACCES)
ok    write outside the rules  /etc/scalws: refused (EACCES)
ok    run a program            refused (EPERM)
ok    fork                     refused (EPERM)
ok    io_uring                 refused (ENOSYS)
ok    packet socket            refused (EPERM)
ok    UDP socket               allowed
sandbox self-test passed

scalwsctl status shows one line, for example sandbox: on: Landlock ABI 6 (writes, no exec, reads limited), seccomp filter, and the admin API’s /status has a sandbox object.

Service sandbox on systems without systemd

On systemd hosts the units restrict ScalWS (ProtectSystem=strict, ReadWritePaths=, ProtectKernelModules=, LockPersonality=, and for the console RestrictSUIDSGID=; the server units pass --restrict-suid-sgid, the same rules without refusing openat2). Where there is no systemd (Alpine with OpenRC), scalwsd, scalws-controller run and scalws-console run take --service-sandbox to do the same themselves: everything but the service’s own directories becomes read-only (with Landlock), and module loading, persona changes, set-user-ID modes and foreign-architecture system calls are refused. Everything the service starts inherits it, as it inherits a unit’s settings. --service-rw <dir> (repeatable) is the twin of a ReadWritePaths= drop-in, for directories applications write.

The console keeps uid 0 and CAP_DAC_OVERRIDE, so its fence is as narrow as its unit’s write view (PrivateDevices=, ProtectKernelTunables=): no /proc and only /dev/null, /dev/zero, /dev/full and /dev/shm of /dev. Of OpenRC’s state it may write only the control file of scalws-controller’s supervisor (/run/openrc/supervise-scalws-controller.ctl, for the controller restart), as the file that existed when the console started: rc-service scalws-controller start and restart restart a running console to follow a new supervisor’s file.

A write fence alone does not stop a root process from changing the mode, owner, ACLs, extended attributes or times of files outside it (Landlock has no right for those). So the console also makes the read-only mount view of its unit’s ProtectSystem=strict itself: it starts with CAP_SYS_ADMIN and CAP_SETPCAP (the OpenRC service keeps them only with service_sandbox=yes), moves into a mount namespace of its own in which every mount is read-only except the paths of its fence, and drops both capabilities from every set, the bounding set included, before anything else runs. Such changes outside the fence then fail with EROFS, by path or through a descriptor opened for reading. Its log says read-only view (N writable); without CAP_SYS_ADMIN (a container that withholds it) it says no read-only view (no CAP_SYS_ADMIN), and where the mount calls are refused (a container’s seccomp profile or an LSM such as AppArmor’s docker-default) it says no read-only view (mounts refused: ...); in both cases the console starts and only the fence applies. Mounts that were read-only stay so. Mounts the host makes after the console started (a disk, a container’s file system, a bind) appear in the view with their own flags wherever they lie, as with systemd’s ProtectSystem=strict: the fence still refuses writes there, but not mode, owner or time changes. Restart the console after such a mount to make it read-only in the view.

Without Landlock (a kernel without it, or a container that refuses it) only the system call deny-list applies. scalwsctl status then says OpenRC: system call deny-list only, no write fence (no Landlock), and the admin API’s sandbox object has "service_fence": false.

Downgrading

Versions before the sandbox refuse a configuration with server.sandbox (unknown keys are errors). Remove the key before installing an older version.