ScalWS
Get started
Get started

Decision records

ADR-0018: ACME certificate automation

  • Status: Accepted (2026-10-05)

Context

M10: ACME automation. Tenants’ domains need certificates without manual PEM handling; renewal must not drop traffic and must reuse the validated reload path (ADR-0006).

Decision

  • Client: instant-acme 0.8 (pure Rust, rustls + aws-lc-rs, RFC 8555, used in production by several projects). Certificate expiry is read with x509-parser.
  • Challenge: HTTP-01 only. The pipeline answers GET /.well-known/acme-challenge/<token> on plain-HTTP listeners for tokens of pending orders, before routing; unknown tokens fall through to the application. TLS-ALPN-01 and DNS-01 (needed for wildcards) are not implemented.
  • Configuration:
    • server.acme: enabled, directory (default Let’s Encrypt production), contact, terms_of_service_agreed (must be true), state_dir (default /var/lib/scalws/acme), renew_before (default 30 d), check_interval (default 12 h), ca_bundle (extra root for test CAs such as Pebble).
    • application.acme: true — one certificate per application covering all its domains (wildcards and IP addresses are rejected).
    • listener.tls.acme: true — serve managed certificates on that listener; such a listener may have no static certificates. Static certificates win on name conflicts.
  • Storage: state_dir/account.json (credentials, 0600) and state_dir/certs/<tenant>/<app>/{cert.pem,key.pem,meta.json} (key 0600), written atomically (temporary file + rename). meta.json records the names and the expiry.
  • Lifecycle: a manager task checks at start, every check_interval and after each configuration change. An application needs an order when it has no certificate, its domains changed, or the certificate expires within renew_before. After issuance the controller re-applies the active configuration (source acme): the new files are loaded by the normal prepare step and swapped in atomically — existing connections keep their certificate, new handshakes get the new one. Failed orders back off per application (5 min doubling to 24 h). Applications without a certificate yet are simply not served over TLS (or get the listener’s default certificate).
  • Observability: scalws_acme_orders_total{result}, scalws_acme_certificate_expiry_seconds{tenant,app}, log target scalws::acme, and scalwsctl certs / GET /v1/certs.

Consequences

  • Port 80 must reach a plain-HTTP listener for HTTP-01.
  • Verified end to end against Pebble (scripts/acme-e2e.sh), not against Let’s Encrypt.