Decision records

ADR-0016: Local AI advisor

• Status: Accepted (2026-10-05)

Context

Handoff §9/M8: a local LLM summarises telemetry, correlates symptoms, explains likely root causes and suggests safe investigations. It is not the optimizer. Operator chooses the model (≈3B–8B quantized instruct by default, nothing hard-coded); schema-constrained JSON output; compact aggregated input; redaction of credentials, cookies, secret query parameters and bodies; the server and deterministic optimisation are unaffected when the model is down, slow or disabled; bounded queues, strict timeouts, low priority. §8: never execute LLM-generated commands; AI never alters authentication, TLS, WAF decisions or cache rules.

Decision

• Disabled by default (server.ai.enabled: false).
• Provider: one OpenAI-compatible chat-completions endpoint (llama.cpp server, Ollama, vLLM, LM Studio, …) over plain HTTP/1.1; model and endpoint are configuration. The endpoint must be loopback unless allow_remote: true; an API key is read from an environment variable named in the configuration, never from the YAML itself. HTTPS endpoints are supported since 2026-10-06 (system roots plus an optional ca_bundle).
• Input (scalws-ai::AdvisoryInput): for one application, the diagnosis (evidence and findings), up to 10 recent failed/slow requests, the last optimizer decisions and a runtime summary — nothing else. Every string passes the redactor: query strings, user:password@, Bearer/Basic credentials, key=value secrets (password, token, secret, key, session, auth…), JWTs, e-mail addresses and long opaque tokens in path segments are replaced with [redacted]. Request bodies, headers, cookies, source code and database content are never collected. Input is capped at 16 KiB.
• Output: response_format JSON schema is sent; the answer is still validated strictly (deny_unknown_fields, bounded lengths and counts, likely_causes may only reference finding codes present in the input, confidence ∈ low/medium/high). Invalid output is discarded and counted; nothing partial is shown.
• Execution: one worker task, a bounded queue (max_queue, default 4; full → 429), per-call timeout (timeout, default 60 s), per-application minimum interval between automatic requests (min_interval, default 10 min). Automatic requests are enqueued when a new critical or warning finding appears; operators can request one at any time. The worker never blocks request handling; when the queue is full or the model fails, nothing else changes. The model process itself should run with low CPU/GPU priority (operator’s service unit; documented).
• Output use: advisory text only — stored per application, served at GET /explain?app= (POST to request a new one) on the loopback admin listener and by scalwsctl explain. Suggested investigation steps are displayed, never executed. The advisor has no write access to configuration, rules or runtimes.
• Metrics: scalws_ai_requests_total{result} (ok, invalid, timeout, error, rejected), scalws_ai_queue_depth.

Consequences

• Advice quality depends on the operator’s model; validation guarantees shape and references, not correctness. Every answer is labelled with the model name and the time.