# Kubernetes

URL: https://scalws.com/docs/guide/kubernetes/
Source: `docs/guide/kubernetes.md` in the ScalWS repository

ScalWS runs in Kubernetes as the web server of an application: it serves static files,
proxies to other Services and runs PHP through a PHP-FPM sidecar in the same pod. This
guide covers the container image, the Helm chart, the plain manifests and the settings that
matter inside a pod.

ScalWS is also an ingress controller for `networking.k8s.io/v1` Ingress resources and the
Gateway API (Gateway, HTTPRoute), as a mode of the chart; see
[ingress-controller.md](https://scalws.com/docs/guide/ingress-controller/) and [gateway-api.md](https://scalws.com/docs/guide/gateway-api/).
Operators and multi-node
coordination are not part of this release. The multi-node design is in
[ADR-0021](https://scalws.com/docs/adr/0021-multi-node-control-plane/).

> **Note:** the container image `ghcr.io/xsmtx/scalws` and the Helm chart
> `oci://ghcr.io/xsmtx/charts/scalws` are not yet published for public download. Until they
> are, the steps on this page describe how the Kubernetes deployment works but cannot be
> followed as written; the Linux packages are the supported way to install 1.0.0.

## Image

The image is `ghcr.io/xsmtx/scalws:<version>`, built for amd64 and arm64. It is distroless
(no shell), runs as user 65532 and contains `/usr/bin/scalwsd` and `/usr/bin/scalwsctl`.
Because there is no shell, `kubectl exec <pod> -- scalwsctl status` works but
`kubectl exec <pod> -- sh` does not.
`SCALWS_CONFIG` points at `/etc/scalws/scalws.yaml`, which is this file:

```yaml
# Configuration baked into the ScalWS container image: a static site from /srv/www on
# port 8080, probes on 8081. Mount your own file and point SCALWS_CONFIG at it (the Helm
# chart does). Domains other than localhost need their own configuration.
server:
  listeners:
    - name: http
      address: "0.0.0.0:8080"
  health:
    address: "0.0.0.0:8081"
  reload:
    watch: true
  limits:
    shutdown_delay: 5s
  cgroups:
    mode: "off"
  runtime_dir: /run/scalws
  logging:
    format: json
tenants:
  - name: site
    applications:
      - name: web
        root: /srv/www
        domains: [localhost]
        runtime: {type: static}
```

To try it:

```sh
docker run -p 8080:8080 -p 8081:8081 --read-only --tmpfs /run/scalws:uid=65532 \
    ghcr.io/xsmtx/scalws:1.0.0-rc.1
```

## Helm chart

Install the chart from the OCI registry:

```sh
helm install web oci://ghcr.io/xsmtx/charts/scalws --version 1.0.0-rc.1 \
    --set domains={www.example.com} --set content.image=registry.example.com/my-site:1.0
```

`content.image` is an image holding your files; an init container copies them into
`/srv/www`. Without it the pod serves whatever is in `/srv/www` of the ScalWS image. The
chart creates a Deployment, a Service and a ConfigMap with the generated configuration,
plus a PodDisruptionBudget when `replicaCount` is above 1 or autoscaling is enabled, and
the optional objects below.

The main values (all of them are in `deploy/helm/scalws/values.yaml`):

| Value | Default | Meaning |
|---|---|---|
| `replicaCount` | `2` | Number of pods. |
| `domains` | `[scalws.local]` | Domains of the default site, used when `tenants` and `sites` are empty. Together with the domains of the `tenants` applications it is also the TLS `server_names` and the Certificate's `dnsNames`. Domains of `sites` files are not read: list them here for TLS. |
| `tenants` | `[]` | Applications in the `tenants:` form of `scalws.yaml`. Empty: one site serving `/srv/www` for `domains`. |
| `sites` | `{}` | Site files, name to content (the `include` mechanism of [site files](https://scalws.com/docs/sites/)). |
| `server` | `{}` | Extra `server:` settings merged over the chart's (limits, `trusted_proxies`, ...). |
| `existingConfigMap` | `""` | Use a ConfigMap made elsewhere (key `scalws.yaml`, site files `site-*.yaml`). The chart renders no configuration then, so it refuses `sites` and `serviceMonitor.enabled` together with it. |
| `rollOnConfigChange` | `false` | `false`: configuration changes are reloaded in place; settings that need a restart apply only after `kubectl rollout restart` (see [Configuration changes](#configuration-changes)). `true`: a change rolls the Deployment instead; with `tls.enabled` the watcher stays on for certificates. |
| `shutdownDelaySeconds` | `5` | After SIGTERM, keep serving while reporting not ready. |
| `shutdownTimeoutSeconds` | `30` | Time to drain connections afterwards. |
| `tls.enabled` | `false` | Serve HTTPS in the pod. |
| `tls.secretName` | `""` | Secret with `tls.crt` and `tls.key`. Default: `<fullname>-tls`. |
| `tls.certManager.issuerRef` | `{}` | For example `{name: letsencrypt, kind: ClusterIssuer}`. When set, the chart creates the Certificate. |
| `php.enabled` | `false` | Add a PHP-FPM sidecar. |
| `php.image` | `php:8.3-fpm` | Image of the sidecar. It must contain the PHP CLI (`php`), which the probes run. |
| `php.port` | `9000` | FPM port (on `127.0.0.1`). |
| `php.readOnlyRootFilesystem` | `false` | Read-only root filesystem for the sidecar. |
| `php.resources` | requests `cpu: 100m`, `memory: 64Mi` | Resources of the sidecar. |
| `content.image` | `""` | Image whose `content.path` is copied to `/srv/www` (needs `sh` and `cp`). |
| `content.path` | `/app` | Directory inside `content.image`. |
| `content.existingClaim` | `""` | An existing PersistentVolumeClaim mounted at `/srv/www`. With more than one replica the volume must be ReadOnlyMany or ReadWriteMany, because the pods run on different nodes. |
| `resources` | requests `cpu: 100m`, `memory: 128Mi`; no limit | Resources of the ScalWS container. Memory grows with the requests in flight and the size of proxied responses (about 330 MiB for 256 concurrent 100 KiB responses); a memory limit, if you set one, needs that headroom or scalwsd is OOM-killed under load. |
| `autoscaling.enabled` | `false` | Create a HorizontalPodAutoscaler. |
| `autoscaling.minReplicas`, `autoscaling.maxReplicas` | `2`, `10` | Its bounds. |
| `autoscaling.targetCPUUtilizationPercentage` | `70` | Its CPU target. |
| `serviceMonitor.enabled` | `false` | Prometheus Operator ServiceMonitor scraping `/metrics` on the probe port (sets `server.health.metrics`) through a separate ClusterIP Service `<name>-metrics`, so the main Service never exposes it. |
| `serviceMonitor.interval` | `30s` | Scrape interval. |

A PHP application with its code in your own image:

```yaml
domains: [app.example.com]
php:
  enabled: true
  image: php:8.3-fpm
content:
  image: registry.example.com/my-php-app:1.0
  path: /app
```

## Plain manifests

If you do not use Helm, `deploy/kubernetes/static/scalws.yaml` and
`deploy/kubernetes/php/scalws.yaml` are rendered from the chart with the `values.yaml` next
to each of them. Apply them as they are or edit them. `deploy/check.sh` verifies that they
match the chart; `deploy/check.sh --write` re-renders them.

## Pod settings

These `server:` settings are what the image and the chart use inside a pod. The health
listener is described after the list.

```yaml
server:
  reload:
    watch: true
  health:
    address: "0.0.0.0:8081"
    metrics: true
  limits:
    shutdown_delay: 5s
  cgroups:
    mode: "off"
  runtime_dir: /run/scalws
```

* `reload.watch` reloads the configuration when the mounted files change. ConfigMaps and
  Secrets are updated by swapping a symlink, which the watcher follows. An invalid change
  keeps the running configuration and is logged once.
* `limits.shutdown_delay` is the time after SIGTERM during which `/readyz` answers 503 while
  requests are still served, so Services and load balancers stop sending traffic first. Set
  `terminationGracePeriodSeconds` to the delay plus the shutdown timeout plus 5 seconds;
  the chart does this from `shutdownDelaySeconds` and `shutdownTimeoutSeconds`.
* `cgroups.mode: "off"` because the container runtime already enforces limits. With the
  default `auto`, scalwsd logs a warning in a pod.
* `runtime_dir` needs a writable directory (an `emptyDir` when the root filesystem is
  read-only).

Both scalwsd and scalwsctl read `SCALWS_CONFIG`. The order is `--config`, then
`SCALWS_CONFIG`, then `/etc/scalws/scalws.yaml`.

### Health listener

`health.address` serves `/healthz` (liveness) and `/readyz` (readiness) for the kubelet.
With `metrics: true` it also serves `/metrics`. Everything else answers 404. It is a plain
HTTP listener separate from the metrics listener (see
[the metrics listener](https://scalws.com/docs/guide/logging-metrics/#the-metrics-listener)), which keeps its loopback
default.

## Configuration changes

With `rollOnConfigChange: false` (the default) a `helm upgrade` updates the ConfigMap and
each pod reloads it in place. A reload cannot apply settings that need a restart
(listeners, `limits.max_connections`, `health.address`, `event_loops`, cache sizes and the
others listed under [Reload or restart?](https://scalws.com/docs/guide/configuration/#reload-or-restart)): it is
refused with "configuration change requires a restart" and the pods keep the old
configuration. Apply such a change with

```sh
kubectl rollout restart deployment/web-scalws
```

or set `rollOnConfigChange: true`, which rolls the Deployment on every configuration change.

## TLS

By default the pod speaks plain HTTP and TLS ends at an ingress or load balancer. Set
`server.trusted_proxies` (through `server:` in the values) to the addresses of that
ingress, so that client addresses and per-client limits are right.

To serve HTTPS in the pod, set `tls.enabled: true` and provide a Secret with `tls.crt` and
`tls.key` (named by `tls.secretName`). With cert-manager, set `tls.certManager.issuerRef`
and the chart creates the Certificate for `domains`. Renewed certificates are picked up
without a restart, also when `rollOnConfigChange` is `true`.

## PHP

With `php.enabled` the pod gets a PHP-FPM sidecar. Both containers mount the same
`/srv/www`, filled from `content.image` (an image with `sh` and `cp`) or taken from
`content.existingClaim`. PHP-FPM listens on `127.0.0.1:9000` only, so it is reachable from
the pod and from nowhere else. On shutdown the sidecar keeps running through the drain, so
requests in flight still finish. The sidecar has exec readiness and liveness probes that
connect to that port from inside the container (with a 1 second connect timeout), so the
pod is not ready before PHP-FPM accepts connections, and a PHP-FPM whose listener is closed
is restarted. The liveness probe cannot detect hung workers behind an open listener.
The image of `php.image` needs the PHP CLI (`php`) for these probes.

The PHP extensions are those of the sidecar image. For other extensions build your own image
`FROM php:8.3-fpm` and set `php.image`.

## Ingress controller

With `--set ingressController.enabled=true` the chart installs `scalws-ingress` next to
`scalwsd` in every pod, an IngressClass (`scalws`), the RBAC it needs and a `LoadBalancer`
Service. Ingress resources and the common `nginx.ingress.kubernetes.io/*` annotations
become ScalWS configuration; endpoint changes reach the pods without a reload. Install,
annotations, differences from ingress-nginx, troubleshooting and the migration checklist
are in [ingress-controller.md](https://scalws.com/docs/guide/ingress-controller/). A comparison with ingress-nginx and
Traefik on one node is in
[the Kubernetes ingress benchmark](https://scalws.com/docs/benchmarks/2026-10-14-xeon-k8s-ingress/).

### Gateway API

`--set ingressController.gatewayAPI.enabled=true` makes the same controller serve
GatewayClass, Gateway, HTTPRoute and ReferenceGrant resources too, and creates the
GatewayClass `scalws`; `ingressController.ingress.enabled=false` (`--ingress=false`) turns
the Ingress input off for a Gateway-only install. The chart never installs the Gateway API
CRDs; install them first. Both inputs share the pods, the Service address and one
ownership rule for hosts: the oldest claim wins, and an Ingress and an HTTPRoute never
serve the same host. Install, listeners and ports, attachment, ownership, TLS, status and
the conformance result are in [gateway-api.md](https://scalws.com/docs/guide/gateway-api/).

## Limits per pod

Per-client limits, tenant limits and the response cache are kept per pod. With N replicas
the effective limit is up to N times the configured value, and each pod has its own cache.
Size the limits accordingly.

## Not supported in pods

* ACME: scalwsd warns about it in a pod. Use cert-manager.
* Managed runtimes and containers: run Node.js, Python and other applications as their own
  Deployments and proxy to their Services.
* Cgroups: use `cgroups.mode: "off"` and the pod's resources.
* `scalws-controller`: scale with replicas and a HorizontalPodAutoscaler. Under the
  controller scalwsd ignores `server.health.address`, `server.reload.watch` and
  `limits.shutdown_delay`.

## Operations

Run the admin client inside a pod:

```sh
kubectl exec deploy/web-scalws -c scalws -- scalwsctl status
```

The Deployment is named `<release>-scalws` unless the release name already contains
`scalws` or you set `fullnameOverride`. Logs are JSON on stdout (`kubectl logs`).
Evictions such as `kubectl drain` respect the PodDisruptionBudget (created when
`replicaCount` is above 1 or autoscaling is enabled); rolling updates follow the
Deployment's `maxUnavailable` and `maxSurge` instead. Each pod drains as described under
[Pod settings](#pod-settings).