Reference

scalwsctl

scalwsctl is the operator CLI: it checks configurations, manages site files, and talks to a running server through its admin API.

scalwsctl [GLOBAL OPTIONS] <COMMAND> [ARGS]

Every command accepts --help; scalwsctl --version prints the version.

Global options

You can put these before or after the subcommand.

OptionDefaultMeaning
--config <PATH>$SCALWS_CONFIG, else /etc/scalws/scalws.yamlConfiguration file. Commands that talk to the server read it to find the admin socket. check and backup use it as their input.
--socket <PATH>from the configurationAdmin socket. Overrides the one in the configuration.
--address <ADDR>noneLoopback metrics listener, for example 127.0.0.1:9900. Use it for read-only views instead of the admin socket.
--remote <HOST:PORT>noneRemote admin API over mutual TLS. Requires --ca, --client-cert and --client-key.
--ca <PATH>noneCA (PEM) that issued the remote admin API’s server certificate.
--client-cert <PATH>noneOperator client certificate (PEM).
--client-key <PATH>noneOperator client key (PEM).

How scalwsctl finds the server

For a single server, scalwsctl tries these in order:

  1. --remote
  2. --address
  3. --socket
  4. the admin socket from the configuration: server.admin.socket, or <server.runtime_dir>/admin.sock (server.runtime_dir), which is /run/scalws/admin.sock by default
  5. the configuration’s metrics listener (server.metrics.address), if the admin socket is disabled; an unspecified address such as 0.0.0.0 is reached as 127.0.0.1

If neither the admin socket nor the metrics listener is enabled, you get:

scalwsctl: neither the admin socket nor the metrics listener is enabled

The admin socket is mode 0660 scalws:scalws, so run scalwsctl as root or as a member of group scalws (usermod -aG scalws alice).

The metrics listener has no authentication. It answers only the read-only views status, apps, diagnose, recommendations and explain, plus /metrics, /healthz, /readyz and the dashboard. Every other view, and every change, needs the admin socket or --remote. A change sent to the metrics listener is refused with ... (use the admin socket: --socket or server.admin.socket).

Under scalws-controller

Each instance has its own admin socket (<runtime_dir>/web-<n>/admin.sock) and its own metrics port (127.0.0.1:9910 + n by default, see instances.metrics_base_port). There is no socket at /run/scalws/admin.sock.

scalwsctl treats the controller as running when its configuration file exists and the control socket it names (controller.socket) is present. The controller configuration is /etc/scalws/controller.yaml; site, migrate, panel, limit and high accept --controller-config <PATH> to name another one. reload always uses /etc/scalws/controller.yaml.

While the controller runs, and no server is named with --socket, --address or --remote:

  • reload, and the reloads done by site, migrate and limit --persist, go through the controller, so every instance moves to the new configuration.
  • site and migrate edit the configuration the controller serves (controller.config) instead of --config.
  • limit and high reach every instance through the controller.
  • These commands also need at least one instance with an admin socket under server.runtime_dir; otherwise they stop with scalws-controller runs no instance with an admin socket under <dir>.

For views of a single instance, name it, for example scalwsctl --socket /run/scalws/web-0/admin.sock apps or scalwsctl --address 127.0.0.1:9910 status.

Offline commands and admin-API commands

Works without a running serverNeeds the running server (admin socket, --address or --remote)
check (config-test), detect, profiles, backup, restore, completion, site liststatus, apps, tenants, top, reload, rollback, certs, upstreams, history, diff, restart, scale, purge, recommendations, diagnose, explain, limit, high

site add, site remove and migrate write files offline and then reload the server. They need the server only for that reload, which you can skip with --no-reload or --dry-run.

Exit status

On failure, scalwsctl prints scalwsctl: <message> on stderr and exits with status 1. check and detect print their errors without that prefix. diagnose exits with 2 when a finding is critical. explain exits with 1 unless the answer’s status is ok.

Configuration commands

check (alias config-test)

scalwsctl check [PATH]

Validates a configuration and prepares it exactly as a reload would: it loads certificates, checks document roots, and includes the site files. The running server is not touched. PATH defaults to --config.

$ sudo scalwsctl check
/etc/scalws/scalws.yaml: OK (2 listener(s), 3 tenant(s), 4 application(s), 6 domain(s))

Warnings are printed as warning: <path>: <message>. Errors list every problem, each with its location:

/etc/scalws/scalws.yaml: 2 configuration error(s):
  - tenants[0].applications[1].domains[0]: domain "shop.example.com" is already used by shop/web
  - server.acme.contact[0]: use mailto:address

detect

scalwsctl detect <PATH> [--json]

Detects the framework in an application directory and proposes an application configuration. Nothing is written. The output lists each matching profile with its confidence and evidence (+ matched, - not matched), marks the best one <- proposed, and prints a proposed applications entry. Values that could not be determined are left as FILL_IN_ placeholders. Exits with 1 when the path is not a directory or when no profile matches (no profile matched; configure the runtime manually).

profiles

scalwsctl profiles

Lists the built-in application profiles, one per line with its name and title, for example wordpress WordPress. The built-in profiles are wordpress, woocommerce, laravel, php, nextjs, node, django, flask, fastapi and python.

Sites

Site files are short YAML files, one per site, in the directory named by the main configuration’s include (by default /etc/scalws/sites.d/*.yaml). See Sites and the site file reference.

site and migrate write into the directory of the first include entry that contains a *, named <domain> plus that entry’s prefix and suffix (for sites.d/*.yaml: sites.d/<domain>.yaml). Without such an entry they stop with:

/etc/scalws/scalws.yaml has no `include:` for site files; add `include: sites.d/*.yaml` and create /etc/scalws/sites.d

Both accept:

OptionDefaultMeaning
--controller-config <PATH>/etc/scalws/controller.yamlWhen this file exists and the control socket it names is present, the configuration it serves (controller.config) is the one edited, and reloads go through scalws-controller.

site add

scalwsctl site add <DOMAIN> [--root PATH] [RUNTIME] [OPTIONS]

Writes <domain>.yaml next to the other site files, checks the whole configuration as a reload would, and reloads the server. The domain is written in lower case without a trailing dot. A domain that is already served (by a site file or by scalws.yaml), also as an alias or a www. name, is refused before anything is written: <domain> is already served by <tenant>/<app> (<file>). If the check fails, the file is removed (the new site does not pass the check, nothing was kept). If the server refuses the reload, the file is removed again.

Runtime (give at most one; the site is static when you give none):

OptionMeaning
--phpManaged PHP-FPM pool. Refused while scalws-controller runs; use --php-fastcgi there.
--php-fastcgi <ADDRESS>PHP through an existing PHP-FPM pool, for example unix:/run/php/php8.3-fpm.sock.
--node <COMMAND>Node.js command, for example "node server.js". The application must listen on $PORT. Refused while scalws-controller runs.
--python <MODULE:APP>Python ASGI application, for example main:app. Refused while scalws-controller runs.
--proxy <URL>Reverse proxy to an upstream, for example http://127.0.0.1:3000.
--redirect <URL>Redirect every request to another origin, for example https://example.org.

Other options:

OptionMeaning
--root <PATH>Document root: static files, PHP scripts, or the application directory. Default for a static or PHP site: /srv/www/<domain>. A missing directory is created, and a static site gets a placeholder index.html.
--www none|alias|redirectHow to handle www.<domain>. Default: redirect for a domain with exactly one dot that does not start with www., such as example.com, otherwise none.
--alias <DOMAIN>A further domain for the site. Repeatable.
--tls auto|manualauto gets a certificate through ACME. manual uses the TLS listeners’ certificates. Default: auto when server.acme.enabled is set.
--tenant <NAME>The tenant the site belongs to. Default: derived from the domain (shop.example.com becomes shop-example-com).
--htaccessRead the site’s .htaccess files.
--https-redirectRedirect plain HTTP to HTTPS (writes https_redirect: true).
--dry-runPrint the file that would be written and change nothing.
--no-reloadWrite and check the file, but do not reload the server.
$ sudo scalwsctl site add blog.example.com --root /srv/www/blog --php-fastcgi unix:/run/php/php8.3-fpm.sock --https-redirect
added blog.example.com → /etc/scalws/sites.d/blog.example.com.yaml
reloaded: ...

While scalws-controller runs, the managed runtimes are refused before anything is written, because every instance would start its own pool (ADR-0036):

scalwsctl: scalws-controller cannot run a managed PHP-FPM pool; use --php-fastcgi unix:/run/php/php8.3-fpm.sock (an existing pool) (ADR-0036)

--node and --python are refused the same way. Start those applications yourself (for example with systemd) and use --proxy http://127.0.0.1:PORT. When the controller is installed but not running, managed runtimes are accepted.

site list

scalwsctl site list [--json]

Lists every application with its tenant, domains, runtime and the file that defines it. This works offline.

TENANT            APP               DOMAINS               RUNTIME  FILE
blog-example-com  blog-example-com  blog.example.com      php      /etc/scalws/sites.d/blog.example.com.yaml
default           welcome           localhost, 127.0.0.1  respond  /etc/scalws/scalws.yaml

site remove

scalwsctl site remove <DOMAIN> [--no-reload]

Renames the site’s file to a hidden backup next to it (.<file>.removed-<time>), checks the configuration, and reloads. If the check or the reload fails, the file is put back. Only a domain that has a site file of its own can be removed this way. A domain defined in scalws.yaml gives <domain> is served by <tenant/app>, not by a site file of its own; edit it there.

migrate nginx, migrate apache

scalwsctl migrate nginx  [PATH] [OPTIONS]    # PATH default: /etc/nginx/nginx.conf
scalwsctl migrate apache [PATH] [OPTIONS]    # default: /etc/apache2/apache2.conf, else /etc/httpd/conf/httpd.conf

Reads every server block (nginx) or <VirtualHost> (Apache), including the files they include, and writes one ScalWS site file per site. The source configuration is only read. Directives that are not converted are written into the file as # TODO: lines.

OptionMeaning
--dry-runPrint the sites and files that would be written. Change nothing.
--overwriteReplace sites ScalWS already serves without asking. Conflicts with --skip-existing.
--skip-existingKeep sites ScalWS already serves without asking.
-y, --yesDo not ask before writing.
--no-reloadWrite and check, but do not reload the server.

On a terminal, migrate asks about each domain ScalWS already serves (Overwrite? [y]es / [n]o / [a]ll / n[o]ne:) and, unless --yes is given, once more before writing (Write N site file(s)? [y/N]). Only a domain’s own site file can be replaced; it is kept as .<name>.replaced-<time>. A domain defined in scalws.yaml or in another site’s file is skipped with a note.

Without a terminal, conflicts must be decided with --overwrite or --skip-existing. Otherwise nothing is written:

already served by ScalWS: shop.example.com (...)
run interactively, or pass --overwrite or --skip-existing

All files of a run are checked together like a reload. If the check or the reload fails, every file of the run is removed and replaced files are restored. The conversion rules are in Sites.

Views (read-only)

All views accept --json for the raw admin-API answer, except top and diff.

status

The server’s version, configuration generation, uptime, listeners and features.

scalwsd 1.0.0  generation 4  up 86400s
  listeners: http 192.0.2.10:80, https 192.0.2.10:443
  3 tenant(s), 4 application(s)
  cgroups on, cache on, optimizer on, AI advisor off
  admin socket: /run/scalws/admin.sock
  metrics: 127.0.0.1:9900

DRAINING appears after the uptime while the server shuts down.

apps

Applications with their runtime state, workers (ready/target [min-max]), in-flight requests, requests in the last window, p95 latency and findings. Findings are marked ! for a warning and !! for critical.

APPLICATION                  RUNTIME  STATE           WORKERS IN FLIGHT  REQ/WIN   P95 MS  FINDINGS
shop/web                     php      ready      4/4 [2-16]         3      812       41  ok
acme/api                     node     ready        2/2 [1-4]         3      120       13  saturated!

tenants

Tenants with memory in MiB, CPU seconds and their applications. This view needs the admin socket.

top

scalwsctl top [--interval N] [--once]

A refreshing apps view. --interval is the seconds between refreshes (default 2). --once prints once and exits.

certs

ACME-managed certificates: application, state (ok, pending, error, missing), days left, names, and the last error with the retry delay. With no ACME application, it prints no ACME-managed applications (application acme: true, server.acme.enabled). Needs the admin socket.

upstreams

scalwsctl upstreams [--json]
scalwsctl upstreams show <POOL> [--json]
scalwsctl upstreams set <POOL> <ADDR[=WEIGHT]>...

Upstream pools with one row per member: weight, active requests, consecutive failures and state (ok, or ejected while passive health has taken the member out). A pool without members prints one row with -; with no pools it prints no upstream pools configured.

$ scalwsctl upstreams show api
POOL                 MEMBER                       WEIGHT ACTIVE FAILURES STATE
api                  192.0.2.5:8080                    2      1        0 ok

set replaces the members of a pool with members_from: api, without a reload, and prints the resulting pool. The weight defaults to 1 and must be 1 to 1000:

$ scalwsctl upstreams set api 192.0.2.5:8080=2 192.0.2.6:8080 '[::1]:8080=3'

A pool that is not members_from: api is refused, an unknown pool is reported as such, and under scalws-controller set is refused (change the pool through the controller’s source). Needs the admin socket.

history and diff

scalwsctl history [--json]
scalwsctl diff [--from G] [--to H]

history lists the configurations the server keeps (generation, time, source, and <- active on the active one). The server keeps them in memory: at most 10, starting with the one it started with, so the list begins again after every restart of scalwsd. diff shows the differences between two of them. The default is from the previous generation to the active one. Both need the admin socket.

recommendations

scalwsctl recommendations [--last N] [--json]

The optimizer’s state and decisions per application: mode, requests per window, 5xx %, p95, in flight and workers, followed by the last N decisions (default 20), for example #1 acme/api scale_up applied workers 1 -> 2 with the reason on the next line.

diagnose

scalwsctl diagnose [TENANT/APP] [--json]

Diagnostic findings for all applications. With TENANT/APP, it also shows that application’s recent failed and slow requests. Exits with 2 if any finding is critical. See Troubleshooting.

acme/api  120 requests, p95 2048 ms
  [CRITICAL] upstream_timeouts: 25% of requests timed out
      - 30 of 120 requests
      possibly caused by: cpu_pressure
      -> raise workers

Recent failed or slow requests (1):
  1760000000 GET /slow -> 504 headers 3000 ms, total 3000 ms, upstream timeout

explain

scalwsctl explain <TENANT/APP> [--refresh] [--wait SECONDS] [--json]

Shows the optional local AI advisor’s explanation of an application’s diagnostics. The advice is advisory only. --refresh asks for a fresh explanation first, which needs the admin socket. --wait is how long to wait for a pending explanation (default 120). Exits with 1 unless the answer’s status is ok. With the advisor disabled (server.ai.enabled), you get AI advisor disabled (server.ai.enabled).

Changes to the running server

CommandWhat it doesOutput
reloadRe-reads the configuration file: validate, prepare, swap. On an error the active configuration stays. Goes through scalws-controller while it runs (see Under scalws-controller), unless --socket, --address or --remote names a server.reloaded: {"generation":5}
rollback [--generation G]Re-applies an earlier configuration from the server’s in-memory history. The default is the one before the active one. The configuration file is not changed, so the next reload reads the file again.rolled back: {"generation":6}
restart <TENANT/APP>Restarts an application without dropping requests: a new instance first, then the swap.restarted shop/web: ...
scale <TENANT/APP> <WORKERS>Sets the worker count of a managed pool, within workers.min..max.scaled acme/api: {"from":2,"to":4,...}
purge --host H --path P, purge --host H --prefix P, purge --tag T, purge --allPurges the response cache.purged

TENANT/APP must have exactly one /. Otherwise: scalwsctl: "x": use tenant/app.

rollback with nothing earlier in the history fails with no earlier configuration in the history. It acts on one scalwsd: under scalws-controller it reaches only the instance you name, not the controller’s generation. To go back under the controller, see Upgrading.

scale changes an application pool’s workers. To change the number of server instances under the controller, use scalws-controller scale.

Tenant resources: limit and high

These commands need cgroup v2 enforcement (server.cgroups.mode). See Operations.

scalwsctl limit cpu|ram|io <USER> [VALUE] [--clear] [--persist] [--controller-config PATH]
scalwsctl high  cpu|ram|io [-n|--top N] [--interval-ms MS] [--json] [--controller-config PATH]

limit applies a limit to a tenant’s cgroup at once. USER is the tenant’s name or its Unix user; a user that runs several tenants gives user "x" runs several tenants (...); name the tenant. Values:

ResourceValue
cpuPercent of one CPU: 50%; 200% is two CPUs.
ramPercent of the host’s memory (30%, at most 100), or a size (2GiB).
ioShare of a default tenant’s disk time (50%, at most 10000%), or a rate for reads and for writes each (20MiB/s). An I/O share and an I/O rate are separate limits; setting one keeps the other.

Sizes follow the value formats.

  • The limit lasts until scalwsd restarts. Under the controller it lasts until the controller restarts. It survives reloads.
  • --persist also writes the limit to the tenant’s resources in the configuration. The previous file is kept as <file>.bak.<time>. Then the server reloads. The edit is refused if it would change anything besides that tenant’s limits.
  • --clear removes the limit set at run time. With --persist, it is also removed from the configuration.

You must give either a value or --clear: give a value (e.g. 50%) or --clear. Both together give give a value or --clear, not both.

high shows the tenants using the most of a resource right now. It samples over --interval-ms (default 1000) and shows at most --top tenants (default 10). Limits set at run time are marked *:

Memory in use (host: 8.0 GiB)

TENANT  USER      RAM  OF HOST      LIMIT
------  ----  -------  -------  ---------
shop    ali   512 MiB    6.2 %  1.0 GiB *
blog    -         0 B    0.0 %          -

* limit set at run time, not in the configuration

While the controller runs, limits go to every running instance through the controller, the controller gives them to instances that start later, and --persist reloads through the controller. A server you name with --socket, --address or --remote takes precedence.

Installed aliases

The package installs symlinks to scalwsctl. scalwsctl checks the name it was called by:

AliasSame as
scalws-limitcpu <user> <value>scalwsctl limit cpu <user> <value>
scalws-limitram <user> <value>scalwsctl limit ram <user> <value>
scalws-limitio <user> <value>scalwsctl limit io <user> <value>
scalws-highcpuscalwsctl high cpu
scalws-highramscalwsctl high ram
scalws-highioscalwsctl high io
scalws-limitcpu ali 50%
scalws-limitram ali 2GiB --persist
scalws-limitcpu ali --clear
scalws-highcpu -n 5 --json

Panel

scalwsctl panel detect
scalwsctl panel sync
scalwsctl panel status [--json]
scalwsctl panel enable cpanel|directadmin|plesk [--dry-run]
scalwsctl panel disable [--restore-php]

Serves a hosting control panel’s domains with ScalWS; see control panels for what it changes and what it does not support.

CommandWhat it does
panel detectPrints cpanel, directadmin or plesk when that panel is installed here, else none.
panel enable <panel>Hands ports 80 and 443 from the panel’s web server (cPanel: Apache; DirectAdmin: the CustomBuild web server; Plesk: nginx and Apache) to ScalWS (needs root). Refuses without the panel, when panel mode is already on, when another program than the panel’s web server holds port 80 or 443, or when a TLS listener has its own certificates_file. Enables scalws-controller at boot, reloads it after the restart and waits until it answers HTTP on port 80. A failed step undoes the earlier ones (also stopping the controller’s instances) and the message says whether the panel’s web server serves port 80 again. --dry-run prints the plan and changes nothing.
panel syncReads the panel’s domains and certificates, writes the site files under /etc/scalws/sites.d/panel/, checks the configuration and reloads once. Fails with panel mode is not enabled before enable.
panel statusThe last sync: time, sites, reload, skipped domains, warnings and the error of a failed sync. --json prints the status file. Prints panel mode is off without panel mode.
panel disableGives the ports back to the panel’s web server (stops and disables scalws-controller, terminates its instances, starts the web server and checks port 80) and removes the panel lines and the generated files (needs root). PHP-FPM, cPanel’s FileProtect and an include entry you had before stay. --restore-php also puts back the recorded PHP handlers (on Plesk only on a domain or service plan that still has its recorded id; one deleted and created again is a warning).

Backup and restore

scalwsctl backup <OUT>
scalwsctl restore <DIR> [--apply]

backup copies the configuration file (--config), its site files (every file its include matches), and the files they reference (TLS certificates and keys, tls_ca_file bundles) into a new directory, under files/<absolute path>, with a manifest.json. The target must not exist, or must be empty. Output: backed up N file(s) to <OUT>.

Not in a backup: document roots, application data, ACME state (account and certificates), controller.yaml, and the console’s state and credentials.

restore is a dry run unless you pass --apply. It shows each file of the backup as unchanged, changed or new. With --apply, replaced files are kept as <file>.bak.<time>, and the restored files get their original mode (and the replaced file’s owner). Then the restored configuration is validated. Apply it with scalwsctl reload (or scalws-controller reload). restore only writes the files in the backup: a site file added after the backup stays where it is.

Shell completion

scalwsctl completion bash|zsh|fish|elvish|powershell

The package already installs completions for bash, zsh and fish.

See also