Users' guide
Gateway API
scalws-ingress also serves the Kubernetes Gateway API: GatewayClass, Gateway, HTTPRoute
and ReferenceGrant (gateway.networking.k8s.io), at the level of the GATEWAY-HTTP
conformance profile. It is the same controller, in the same pod, as the
Ingress controller: both inputs are translated together into one
configuration of the scalwsd next to it, and they can run alone or together. This guide
covers the install, the GatewayClass and its features, how Gateway listeners map onto the
pod’s ports, attachment and host ownership, the translation of HTTPRoute with every
difference from the API’s wording, TLS, what fails closed, status, Events and metrics, the
conformance result and migration.
Design record: ADR-0052. The Ingress input, the shared parts of the controller (channels, isolation, start-up) and the data plane are described in ingress-controller.md, load-balancing.md and configuration.md.
Note: the container image
ghcr.io/xsmtx/scalwsand the Helm chartoci://ghcr.io/xsmtx/charts/scalwsare not yet published for public download. Until they are, the steps on this page describe how the Kubernetes deployment works but cannot be followed as written; the Linux packages are the supported way to install 1.0.0.
Install
The CRDs
The Gateway API resources are CustomResourceDefinitions that the cluster must have before the controller can serve them. Install the standard channel of v1.6.3:
kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.3/standard-install.yamlThe file’s SHA-256 is
356c2b8e286a964dc20995435659ad1095386329974793af10bbfe3eff977fe5. Supported bundles are
the standard channel v1.3.0 to v1.6.x; with another bundle (or the experimental channel)
the controller logs a warning at start and serves best effort. Only the fields of the
standard channel are read.
The chart never installs the CRDs, and has no option to. They are cluster-wide and shared by every Gateway API implementation in the cluster, and Helm deletes the CRDs it owns when a release is uninstalled, which would delete every Gateway and HTTPRoute of every implementation. Install and upgrade them as their own step, as you would for any shared cluster API.
Chart values
The Gateway API is a part of the chart’s ingress controller mode. It is in chart versions after 1.0.0-rc.1:
helm install ingress oci://ghcr.io/xsmtx/charts/scalws --version <chart version> \
--set ingressController.enabled=true \
--set ingressController.gatewayAPI.enabled=true| Value | Default | Meaning |
|---|---|---|
ingressController.gatewayAPI.enabled | false | Serve Gateway API resources (--gateway-api). |
ingressController.gatewayAPI.controllerName | scalws.com/gateway-controller | The spec.controllerName of the GatewayClasses this release serves (--gateway-controller-name). |
ingressController.gatewayAPI.gatewayClass.create | true | Create a GatewayClass with that controllerName. |
ingressController.gatewayAPI.gatewayClass.name | scalws | Its name. |
ingressController.gatewayAPI.extraPorts | [] | Extra Gateway listener ports, for example [{port: 8080, protocol: HTTP, containerPort: 9080}]. See Listeners and ports. |
ingressController.ingress.enabled | true | Serve Ingress resources (--ingress). false makes a Gateway-only install; the IngressClass and the Ingress RBAC are then left out. |
ingressController.crossNamespacePaths | false | Applies to both inputs, see Ownership. |
ingressController.publishService | the release’s Service | Its address goes into Gateway.status.addresses as well as the Ingress status. |
The other ingressController.* values (kind, watchNamespace, defaultCertificate,
service, resources) work as in Ingress mode. At least
one of ingress.enabled and gatewayAPI.enabled must be on.
With gatewayClass.create the chart checks that the CRDs are present and fails with the
install command above when they are not; set gatewayClass.create=false to install the
chart first and create the GatewayClass yourself. Two releases in one cluster need
different controllerName values; with the same value both would serve the same Gateways
and write their status. deploy/kubernetes/gateway/ holds manifests rendered from the
chart for both inputs with port 8080 as an extra port (deploy/check.sh keeps them in
sync).
Check the install
kubectl get gatewayclass scalws # ACCEPTED True
kubectl get pods -l app.kubernetes.io/instance=ingress
kubectl get service ingress-scalws # EXTERNAL-IPA first Gateway, the usual HTTP-to-HTTPS redirect and a route to a Service:
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata: {name: shop, namespace: shop}
spec:
gatewayClassName: scalws
listeners:
- {name: http, port: 80, protocol: HTTP, hostname: shop.example.com}
- name: https
port: 443
protocol: HTTPS
hostname: shop.example.com
tls: {certificateRefs: [{name: shop-example-com}]}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: redirect, namespace: shop}
spec:
parentRefs: [{name: shop, sectionName: http}]
rules:
- filters: [{type: RequestRedirect, requestRedirect: {scheme: https, statusCode: 301}}]
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: web, namespace: shop}
spec:
parentRefs: [{name: shop, sectionName: https}]
rules:
- backendRefs: [{name: web, port: 80}]kubectl get gateway -n shop shop # PROGRAMMED True, ADDRESS of the Service
kubectl get httproute -n shop # then: kubectl describe httproute -n shop web
curl -i -H 'Host: shop.example.com' http://192.0.2.10/ # 301 to https://shop.example.com:443/
curl -k --resolve shop.example.com:443:192.0.2.10 https://shop.example.com/(192.0.2.10 stands for the Service’s address.) The controller generates two applications
in the tenant gateway, one per scalwsd listener:
- name: g_http_h_shop_example_com
listeners: [http]
domains: [shop.example.com]
routes: [{path: /, runtime: {type: redirect, scheme: https, port: 443, status: 301}}]
- name: g_https_h_shop_example_com
listeners: [https]
domains: [shop.example.com]
routes: [{path: /, runtime: {type: proxy, upstream: {pool: p_shop_web_80}}}]Gateway applications are named g_<scalwsd listener>_ plus the Ingress-style host name
(g_<listener>_default for the catch-all); pools are p_<namespace>_<service>_<port> and
are shared with Ingresses that use the same Service port with the same settings.
GatewayClass and features
Every GatewayClass whose spec.controllerName is the controller’s
(--gateway-controller-name) is served and gets Accepted: True. A class with
parametersRef gets Accepted: False, reason InvalidParameters, and its Gateways are not
served: there are no class parameters.
The controller lists what it supports in GatewayClass.status.supportedFeatures, sorted by
name; the conformance suite reads the list there, so the claim itself is tested:
| Feature | Basis |
|---|---|
Gateway, HTTPRoute, ReferenceGrant | core |
GatewayHTTPListenerIsolation | listener-scoped hosts and the most-specific-listener rule |
GatewayPort8080 | only when port 8080 is an extra HTTP port (extraPorts) |
HTTPRoute303RedirectStatusCode, HTTPRoute307RedirectStatusCode, HTTPRoute308RedirectStatusCode | redirect statuses |
HTTPRouteBackendProtocolWebSocket | appProtocol: kubernetes.io/ws |
HTTPRouteBackendTimeout, HTTPRouteRequestTimeout | timeouts.backendRequest, timeouts.request |
HTTPRouteHostRewrite, HTTPRoutePathRewrite | filter URLRewrite |
HTTPRouteMethodMatching, HTTPRouteQueryParamMatching | method and queryParams matches |
HTTPRoutePathRedirect, HTTPRoutePortRedirect, HTTPRouteSchemeRedirect | filter RequestRedirect |
HTTPRouteResponseHeaderModification | filter ResponseHeaderModifier |
Not claimed: HTTPRouteRequestMirror and its variants, HTTPRouteCORS,
HTTPRouteBackendRequestHeaderModification, HTTPRouteParentRefPort,
HTTPRouteDestinationPortMatching, HTTPRouteBackendProtocolH2C,
GatewayStaticAddresses, GatewayAddressEmpty, GatewayInfrastructure,
GatewayHTTPSListenerDetectMisdirectedRequests,
GatewayRouteHostnameIntersectionPrecedence, the client certificate features, the
experimental features (retries, session persistence, named rules), BackendTLSPolicy,
ListenerSet, GRPCRoute, TLSRoute, TCPRoute and UDPRoute.
Listeners and ports
scalwsd cannot add or remove listeners without a restart, and the controller never
restarts it. The pod’s listeners are therefore fixed by the chart: http (Service port 80)
and https (Service port 443), plus one p<port> listener per extraPorts entry. The
controller learns the mapping from --gateway-listener PORT=NAME (the chart passes
80=http, 443=https and one per extra port); a port is HTTPS when its scalwsd listener
has tls. A Gateway listener is placed on the scalwsd listener of its port:
| Gateway listener | Result |
|---|---|
port exposed, protocol the port’s protocol (HTTP or HTTPS) | Served on that scalwsd listener. |
port not exposed | Accepted: False, reason PortUnavailable; the message lists the exposed ports. |
port exposed with the other protocol (HTTP on 443) | Accepted: False, reason PortUnavailable, “port 443 of this deployment serves HTTPS”. |
protocol TLS, TCP, UDP or unknown | Accepted: False, reason UnsupportedProtocol, supportedKinds: []. |
To serve another port, add it to extraPorts: each entry adds a scalwsd listener on
containerPort, a container port, a Service port and the flag. protocol is HTTP or
HTTPS; port may not be 80 or 443, and containerPort may not be one the chart uses
(8080, 8443, 8081, 10254). Extra ports serve Gateways only, never Ingresses.
Every generated Gateway application names the one scalwsd listener it serves (the
application key listeners), so one host
can have a redirect on port 80 and a backend on port 443. Gateway applications have no
implicit HTTPS redirect: in the Gateway API the redirect is an explicit route on the HTTP
listener, as in the example above. A request that no rule of a host matches gets 404.
Shared address
Every Gateway of a ScalWS GatewayClass is served by the controller’s pods and has the
address of its Service (--publish-service): load balancer ingress addresses, else
external IPs, else the cluster IP, each with its type (IPAddress or Hostname). Until
the Service has an address the Gateway is Programmed: False, reason AddressNotAssigned.
A Gateway with spec.addresses is served, but stays Programmed: False
(AddressNotAssigned) unless every requested address is one of the Service’s; an address
type other than IPAddress and Hostname makes it Accepted: False, reason
UnsupportedAddress.
What this costs: two Gateways on port 80 share one IP address and port, so nothing in a
request says which Gateway it was meant for; only the host tells them apart. Hosts are
therefore owned (next sections): Gateways of different namespaces cannot serve the same
host, and the first owner keeps it. Restrict who may create Gateways of the class:
grant create on gateways (gateway.networking.k8s.io) only to the teams that should
own hosts on the shared address, as you would for a shared ingress controller. Routes
attach under each Gateway’s allowedRoutes, so application teams need only HTTPRoutes.
Per-Gateway addresses (one Service per Gateway) are planned for the next phase (2d). Until
then the conformance test HTTPRouteMultipleGateways cannot pass, see
Conformance.
Attaching routes
A Gateway is served when its gatewayClassName names a served GatewayClass. Its listeners
are checked in this order: protocol and port (above); allowedRoutes.kinds (only
HTTPRoute of group gateway.networking.k8s.io; other kinds are left out of
supportedKinds and, when any were asked for, ResolvedRefs: False, reason
InvalidRouteKinds); TLS; listeners of the Gateway with the same port and hostname as an
earlier one get Conflicted: True, reason HostnameConflict.
An HTTPRoute attaches through each parentRefs entry that names a served Gateway (group
gateway.networking.k8s.io and kind Gateway, the defaults; the namespace defaults to the
route’s) to every listener of it that:
- matches
sectionNameandportwhen they are set, - is accepted,
- allows the route’s namespace (
allowedRoutes.namespaces.from:Same, the default,All,Selectorwith a label selector on the Namespace, orNone) and its kind, - has a hostname that intersects the route’s
hostnames.
Parent references to Gateways of other namespaces need no ReferenceGrant; the listener’s
allowedRoutes decides. The result is the route’s Accepted condition for that parent:
| Situation | Accepted | Reason |
|---|---|---|
| Attached to at least one listener | True | Accepted |
sectionName or port match no listener | False | NoMatchingParent |
| Listeners match, but none allows the namespace or kind, or they are conflicted | False | NotAllowedByListeners |
| Listeners allow it, but no hostname intersects | False | NoMatchingListenerHostname |
Every intersecting host is refused by the configuration (for example *.com) | False | UnsupportedValue |
| Every host it would serve belongs to another owner | False | HostnameConflict, or NotAllowedByListeners when every one is under an older wildcard of another namespace |
| Every rule is unsupported | False | UnsupportedValue |
| Left out because the configuration would not validate with it | False | InvalidConfiguration |
HostnameConflict and InvalidConfiguration are this implementation’s reasons; the API
allows them.
Hostnames and listener isolation
The hosts a route serves through a listener are the intersection of the listener’s
hostname and the route’s hostnames: an empty side matches everything (both empty: the
catch-all of that listener). A wildcard *.example.com covers one or more labels below
example.com, as the API defines, and becomes the domain **.example.com; a route
*.example.com under a listener *.shop.example.com serves **.shop.example.com.
A host is served only through the most specific listener of the same Gateway and port
whose hostname matches it: a route attached to a broad listener never answers for a host
that a more specific listener of that Gateway names (GatewayHTTPListenerIsolation). Every
explicit listener hostname gets an application even without routes, so such a host answers
404 instead of falling through to a broader one.
Across hosts the most specific domain wins (exact, then *. one label, then the longest
**. suffix, then the catch-all), and only that host’s routes are used: routes of a
broader hostname are not merged into a more specific one. So a request for
api.example.com with routes for api.example.com and for *.example.com only ever sees
the routes of api.example.com; a path those routes lack answers 404. The API leaves this
to the implementation (GatewayRouteHostnameIntersectionPrecedence is not claimed).
Ownership across Ingress and Gateway API
The two inputs share the listeners and hosts of the pod, so one rule decides who serves a
host on a scalwsd listener. Claims:
| Claimant | Owner | Claim time |
|---|---|---|
Ingress rule for a host (or a rule without host, or spec.defaultBackend: the catch-all *) | the Ingress namespace, on http and https together | creationTimestamp of the oldest Ingress of that namespace with the host |
Gateway listener with an explicit hostname | that Gateway (the Gateways of one namespace together) | the Gateway’s creationTimestamp |
| HTTPRoute host through a listener | the Gateway and the route’s namespace | the route’s creationTimestamp |
- The oldest claim wins; ties go to Ingress, then Gateway, then HTTPRoute, then
namespace/name. A loser never takes traffic: an Ingress gets a
PathConflictEvent; an HTTPRoute that lost some hosts on a parent staysAcceptedfor the others and gets aHostnameConflictEvent naming the hosts; one that lost all of them getsAccepted: False. Host patterns are compared as written, so*.example.comanda.example.comare different hosts, except for the next rule. - A host under an older wildcard of another namespace is refused. When an owner of one
namespace holds
*.example.com(or**.example.com), a newer claim of another namespace ona.example.comis not served: an Ingress getsPathConflictnaming the owner; an HTTPRoute gets aHostnameConflictEvent naming the hosts, andAccepted: Falsewith reasonNotAllowedByListenerswhen it loses all its hosts that way. Owners of the same namespace are not refused by their own wildcard, also across inputs: one team may combine its Ingress and Gateway API hosts under its wildcard. - Ingress and HTTPRoute never share a host, with or without
--cross-namespace-paths: their precedence rules, regex mode, HTTPS redirect defaults and policies differ and cannot be merged into one application. This is what makes a migration predictable, see Migration. - Inside a Gateway, a host belongs to the namespace of its oldest attached route.
Routes of other namespaces for that host are not served there (as for Ingresses of
several namespaces). This matters for the common hostname-less listener with
allowedRoutes: All: without the rule any namespace could add a path to another team’s host, or catch with a broad prefix the paths it does not serve. - Gateways of one namespace share their hosts: two Gateways of a namespace may have
the same listener hostname on one port, and the routes attached through either merge
into the host under the namespace rule above; a match two routes share keeps the oldest
(
PathConflict). Gateways of different namespaces with the same explicit listener hostname on one port: the newer listener getsConflicted: True, reasonHostnameConflict(the message names the owning Gateway), and serves nothing. The same holds for a listener hostname owned by an Ingress. Hostname-less listeners claim nothing by themselves; their routes claim the hosts they use. - A Gateway delegates through its own listener hostnames. When a Gateway owns an
explicit listener hostname, the routes its
allowedRoutesadmit (of any namespace) may serve that host, even when an Ingress of the Gateway’s namespace holds an older wildcard above it: the Gateway’s own claim already passed the wildcard rule. The namespace rule among the routes still applies. - An HTTPRoute that is not served keeps its claims (all rules unsupported, or left out
by isolation), and its matches answer
500, as an unserved Ingress keeps its paths, so refusing a route never hands its users to a newer one.
--cross-namespace-paths (chart crossNamespacePaths) lifts only the namespace rules
inside one input: Ingresses of several namespaces merge into a host, routes of every
namespace a listener allows merge their rules into a host of the Gateways of one namespace,
and their wildcard refusals among each other are lifted. Refusals between Ingress and
Gateway API owners, and between Gateways of different namespaces, stay. Turn it on only
when every namespace that can attach routes is trusted with every host.
Translation of HTTPRoute
Each matches entry of each rule becomes one route of the application of each host the
route serves (a rule without matches is one route on prefix /).
| HTTPRoute | ScalWS route |
|---|---|
path Exact | path_type: exact |
path PathPrefix | path_type: prefix on whole segments; a trailing / is ignored (/match/ is /match), as the API says |
path RegularExpression | path_type: regex, written (?:<value>)$, so anchored at both ends; linear-time syntax only (no look-around, no back-references): an expression the engine refuses makes the rule unsupported |
method | methods: [<method>] |
headers Exact, RegularExpression | headers with value, or regex: "^(?:<value>)$"; names lower-cased; only the first entry of a name counts (API rule) |
queryParams Exact, RegularExpression | query with value or regex: "^(?:<value>)$"; only the first entry of a name counts |
filter RequestHeaderModifier | request_headers (set, add, remove); set of Host becomes rewrite.host |
filter ResponseHeaderModifier | response_headers |
filter RequestRedirect | redirect runtime: scheme, hostname as host, path.replaceFullPath as path, path.replacePrefixMatch as prefix, statusCode (301 by default, 302, 303, 307, 308), port (see below) |
filter URLRewrite | rewrite.host from hostname; rewrite.path from replaceFullPath; rewrite.prefix from replacePrefixMatch |
timeouts.request | route request_timeout; 0s is no limit |
timeouts.backendRequest | proxy response_timeout; 0s is one year (the longest value; the data plane refuses 0) |
one backendRef | proxy runtime with upstream: {pool} |
several backendRefs | weighted backends: [{pool, weight}] |
an invalid backendRef | its share answers 500 (a {status: 500, weight} entry); all invalid, all weights 0, or no backendRefs and no redirect: the route answers 500 |
Proxy routes use the controller’s defaults, as for Ingresses: preserve_host: true,
websocket: true, connect_timeout: 5s, response_timeout: 60s, pools with retries: 2
(see Backends) and members from the EndpointSlices.
Differences from the API’s wording and from other implementations:
- Timeouts bound the response head.
timeouts.requestandtimeouts.backendRequestlimit the time until the response head is received, not the whole response body. Atimeouts.requestabove 60 s needs atimeouts.backendRequestat least as long, because the defaultresponse_timeoutof 60 s still applies. - Redirects always carry a port: the given
port; else the well-known port of a givenscheme(443 forhttps, 80 forhttp); else the listener’s port. Without it the data plane would use the request’s port, which inside the pod is the container port. $and?inreplaceFullPathofURLRewriteare literal. They are written as$$and%3F, so neither starts a capture reference nor a query.- Weights are reduced and scaled. The data plane takes weights from 0 to 1000; the
weights are divided by their greatest common divisor and, when the largest is still above
1000, scaled to 1000 with every non-zero weight kept at 1 or more. When that changes a
ratio the route gets a
WeightRoundedEvent. - No fallback to a broader hostname (see Hostnames).
- Regular expressions are matched by a linear-time engine (ADR-0050); the paths are anchored at both ends.
- ExternalName Services are refused, see below. The Ingress input still serves them.
Backends
A backendRef is valid when its group is "" and its kind Service (the defaults), the
Service and the port exist, the reference is allowed (same namespace, or a ReferenceGrant),
the Service is not of type ExternalName, and the port’s appProtocol is empty, http or
kubernetes.io/ws. Members are the ready, serving and not terminating addresses of the
Service’s EndpointSlices for the port; a Service without ready endpoints answers 503,
which the API allows, and the route gets a NoEndpoints Event. Problems set
ResolvedRefs: False on every parent of the route:
| Cause | Reason |
|---|---|
| group or kind not a core Service (ServiceImport too) | InvalidKind |
ExternalName Service | InvalidKind, “ExternalName Services are not supported” |
| Service or port missing | BackendNotFound |
| another namespace without a ReferenceGrant | RefNotPermitted |
appProtocol kubernetes.io/h2c, kubernetes.io/wss, https or another value | UnsupportedProtocol |
ExternalName Services are refused because an ExternalName can point at a Service of another namespace or at a node-local address, past ReferenceGrant, and the API has no way to allow them per namespace. HTTPS and h2c backends need BackendTLSPolicy and HTTP/2 to backends, which are not part of this release.
A backendRef to another namespace needs a ReferenceGrant in that namespace:
apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata: {name: shop-routes, namespace: backends}
spec:
from: [{group: gateway.networking.k8s.io, kind: HTTPRoute, namespace: shop}]
to: [{group: "", kind: Service}] # add name: to allow one Service onlyA deleted or changed ReferenceGrant takes effect at the next synchronisation.
Precedence
The rules of all routes of a host are ordered as the API defines: route
creationTimestamp, then namespace/name, then rule index, then match index. The data
plane then tries them in the API’s precedence: exact path, longest prefix, then among equal
paths a method match first, then more header matches, then more query matches, then the
earlier rule. Regular expression paths come after the prefixes. Two matches with the same
path, method, headers and query: the first wins, the later one is not served and gets a
PathConflict Event.
TLS
An HTTPS listener (tls.mode: Terminate, the default) takes its certificateRefs: core
Secrets of type kubernetes.io/tls, checked with the code scalwsd loads certificates
with. The first usable one is served. A Secret in another namespace needs a ReferenceGrant
in that namespace with from: [{group: gateway.networking.k8s.io, kind: Gateway, namespace: <gateway namespace>}] and to: [{group: "", kind: Secret}]. Problems give
ResolvedRefs: False:
| Cause | Reason |
|---|---|
Secret missing, wrong type, unparsable, key does not match, a group or kind other than core Secret, no certificateRefs | InvalidCertificateRef |
| another namespace without a ReferenceGrant | RefNotPermitted |
A listener without a usable certificate is Programmed: False, reason Invalid, and serves
nothing. Certificates go to the memory volume, as for Ingresses, and onto the scalwsd
listener of the port with these server names:
- A listener with a
hostnameserves its certificate for that name.*.example.comstays*.example.comfor SNI: an X.509 wildcard covers one label, so a client asking fora.b.example.comgets the default certificate. That is a limit of X.509, not of ScalWS. - A listener without
hostnameserves its certificate for the hosts its routes serve (only those its routes own, never the certificate’s other names, which may belong to another owner). Names the certificate does not cover are left out with aTLSSecretEvent. - A listener whose hostname belongs to another owner (
Conflicted: True) adds no certificate; the owner’s certificate serves that name. - Default certificate. On each HTTPS port, the oldest hostname-less listener whose
routes own the catch-all
*there (routes without hostnames whose catch-all no older Ingress or Gateway route holds) provides the default certificate, served for an absent or unknown SNI name. When every Gateway HTTPS listener on the port is in one namespace and no Ingress certificate is on it (one tenant), the oldest hostname-less listener whose routes serve any host provides it. Its certificate is written as an unnamed default entry when it gets no name (server_names: [], see tls.md). On a port shared by several namespaces, a listener whose routes all have hostnames does not: its certificate is never shown to other tenants’ clients. A configured--default-ssl-certificate(chartdefaultCertificate) always wins, also on extra HTTPS ports; without either, the controller’s generated certificateScalWS Ingress Controller Fake Certificatestays the default.
On the main HTTPS port the Ingress certificates come first, then the Gateway certificates
(Gateways oldest first, then listener order); a server name keeps the first certificate
that names it. When two Gateways provide certificates for the same name on a port, the
older Gateway’s is used and both listeners get OverlappingTLSConfig: True, reason
OverlappingCertificates. HTTPS listeners of one Gateway on one port whose hostnames
overlap (no hostname overlaps every hostname) get OverlappingTLSConfig: True, reason
OverlappingHostnames, as the API requires; it is informational.
A wildcard certificate SAN directly under one label (*.org) is accepted for serving
example.org, as Go clients and RFC 6125 do; browsers refuse such certificates.
cert-manager works as for Ingresses: the controller picks up the Secret when it is written
and renewed.
Fail closed
Access-related settings that ScalWS cannot honour make the affected part stop serving instead of serving without them:
| Setting | Result | Why |
|---|---|---|
Client certificate validation (a Gateway spec.tls.frontend entry for the listener’s port, or a listener tls field the controller does not know, such as frontendValidation) | listener Accepted: False, reason UnsupportedValue | Serving without validating clients would expose backends the owner meant to protect. |
Listener tls.options, tls.mode: Passthrough | listener Accepted: False, reason UnsupportedValue | Options usually tighten TLS; ignoring them would weaken it. |
Filters RequestMirror, CORS, ExternalAuth, ExtensionRef, unknown filter types, any filter on a backendRef | the rule’s matches answer 500 | No exact counterpart; see below. |
RequestHeaderModifier that adds or removes Host, or changes a hop-by-hop or framing header | the rule’s matches answer 500 | The data plane forbids these changes. |
| A field the controller does not know inside a match or a filter, an unknown enum value | the rule’s matches answer 500 | A newer field may narrow matching; ignoring it would widen what the rule matches. |
| A path or header regex the engine refuses | the rule’s matches answer 500 |
An unsupported rule is not dropped: a dropped /admin rule would let a broader rule such
as / serve the request unprotected. Each of its matches becomes a route answering 500
with its own key, and the route’s parent status gets PartiallyInvalid: True, reason
UnsupportedValue, with a message that starts with Dropped Rule and names the rule
indexes and the cause (the prefix the API requires). When every rule is unsupported the
parent is Accepted: False, reason UnsupportedValue, and the matches still answer 500.
An UnsupportedFilter or UnsupportedField Event carries the detail.
CORS and RequestMirror are not supported in this release: the data plane’s cors cannot
express the API’s * forms, and it has no mirroring. Unknown fields outside matches and
filters (rule-level experimental fields, Gateway spec.infrastructure, Gateway
spec.tls.backend) are ignored with an UnsupportedField Event; a backend that requires a
client certificate refuses the connection, so ignoring spec.tls.backend exposes nothing.
spec.infrastructure.parametersRef makes the Gateway Accepted: False, reason
InvalidParameters.
No value of a Gateway resource is formatted into configuration text: hostnames, paths,
header names and values, regular expressions and redirect fields go into typed fields and
through the same validation scalwsd runs. One bad HTTPRoute is left out by the
isolation that Ingresses get (its
matches answer 500 and it gets Accepted: False, reason InvalidConfiguration);
Gateways are never left out, their problems are found before translation.
Status
The leader pod (the Lease of the Ingress controller)
writes the status of GatewayClasses, Gateways and HTTPRoutes of this controller, after the
reload, and only when it changed. Every condition carries observedGeneration and keeps
its lastTransitionTime while its status does not change. GatewayClass and Gateway status
is written with merge patches of the fields the controller owns. HTTPRoute.status.parents
is shared with other controllers: the controller reads it, replaces only its own entries
(its controllerName), keeps the others, removes its entries for parents the route no
longer references, and writes the list with the resourceVersion it read; a conflict is
retried at the next report. The API allows at most 32 parent entries; a route that would
exceed it gets no status write and a warning in the log.
| Object | Condition | True reasons | False reasons |
|---|---|---|---|
| GatewayClass | Accepted | Accepted | InvalidParameters |
| Gateway | Accepted | Accepted; ListenersNotValid when some listeners are invalid | ListenersNotValid (no listener valid), InvalidParameters, UnsupportedAddress |
| Gateway | Programmed | Programmed | Invalid, AddressNotAssigned, Pending |
| Listener | Accepted | Accepted | PortUnavailable, UnsupportedProtocol, UnsupportedValue |
| Listener | ResolvedRefs | ResolvedRefs | InvalidCertificateRef, RefNotPermitted, InvalidRouteKinds |
| Listener | Conflicted | HostnameConflict | NoConflicts |
| Listener | Programmed | Programmed | Invalid, Pending |
| Listener | OverlappingTLSConfig | OverlappingHostnames, OverlappingCertificates | not set |
| HTTPRoute parent | Accepted | Accepted | see Attaching routes |
| HTTPRoute parent | ResolvedRefs | ResolvedRefs | InvalidKind, BackendNotFound, RefNotPermitted, UnsupportedProtocol |
| HTTPRoute parent | PartiallyInvalid | UnsupportedValue (Dropped Rule ...) | not set |
Pending means scalwsd refused the last configuration change and the controller is
retrying; the previous configuration keeps serving. Listener status also has
supportedKinds (HTTPRoute, or none for an unsupported protocol) and attachedRoutes,
the number of routes accepted on that listener (counted also when the listener is not
programmed, as the API says). GatewayClass.status.supportedFeatures is the list above.
Events
Warning Events on the HTTPRoute or the Gateway, emitted by the leader once while the
problem lasts (after a leader change an Event may be emitted again once). Conditions carry
the state; Events carry the detail that does not fit a condition message.
| Reason | Meaning |
|---|---|
HostnameConflict | Hosts of the route are not served on a parent because another owner holds them; the message names the hosts. |
PathConflict | A match lost to an older equal match of another route; it is not served. |
ServiceNotFound | A referenced Service or its port is missing (ResolvedRefs: False, BackendNotFound); its share answers 500. |
NoEndpoints | The Service has no ready endpoints; its share answers 503. |
UnsupportedFilter | A filter, a regular expression or a header change cannot be served; the rule answers 500. |
UnsupportedField | An unknown field: inside a match or filter the rule answers 500; elsewhere it is ignored. |
WeightRounded | The backendRefs weights were scaled and a ratio changed. |
TLSSecret | Detail of a certificate problem (a name the certificate does not cover, a name already served by another certificate); never Secret content. |
InvalidConfiguration | The route was left out because the configuration did not validate with it. |
Metrics and readiness
On :10254, next to the Ingress metrics, which
count both inputs (scalws_ingress_syncs_total, scalws_ingress_reload_failures_total):
| Metric | Meaning |
|---|---|
scalws_ingress_gateway_api_available | 1 while the Gateway API watches run (the CRDs are served), else 0. |
scalws_ingress_gateways{state} | Gateways of this controller, programmed or invalid. |
scalws_ingress_httproutes{state} | HTTPRoutes of this controller, accepted, partial (some rules dropped) or rejected. |
scalws_ingress_status_writes_total{kind, result} | Status writes by kind (GatewayClass, Gateway, HTTPRoute) and result: ok, conflict or error. |
The controller does not need the CRDs to start. With --gateway-api it checks the API
group gateway.networking.k8s.io at start and every 60 s:
- CRDs missing: a warning in the log, the gauge at 0, and the pod serves Ingresses alone; a missing install never keeps an Ingress-serving pod unready.
- CRDs present at start: the Gateway API caches are part of the readiness wait, without a time limit. A pod whose RBAC lacks a Gateway API kind stays unready (and logs the list error once per kind) rather than serving a configuration without the Gateways.
- CRDs installed while running: the watches start and the pod stays ready. Structural changes are held until the first lists arrive, at most 60 s; after that they are applied without the Gateway input until the lists complete.
- CRDs removed: after two discovery checks in a row that find them missing, confirmed
by reading the
gateways.gateway.networking.k8s.ioCRD, the Gateway input stops and the pod keeps serving Ingresses.
Flags of scalws-ingress run
Added to the Ingress controller flags:
| Flag | Default | Meaning |
|---|---|---|
--ingress | true | Serve Ingress resources; --ingress=false for a Gateway-only install. |
--gateway-api | off | Serve GatewayClass, Gateway, HTTPRoute and ReferenceGrant. |
--gateway-controller-name | scalws.com/gateway-controller | The GatewayClass spec.controllerName to serve. |
--gateway-listener | 80=http, 443=https | PORT=NAME, repeated: a Gateway listener port and the scalwsd listener that serves it. |
--ingress=false without --gateway-api is refused at start. --cross-namespace-paths,
--publish-service, --default-ssl-certificate and --watch-namespace apply to both
inputs. scalws-ingress bootstrap has no new flag: the extra listeners are in the base
configuration the chart writes.
RBAC
With gatewayAPI.enabled the chart adds:
get,list,watchongateways,httproutesandreferencegrants, andget,update,patchongateways/statusandhttproutes/status(ClusterRole, or a Role inwatchNamespace);- cluster-wide:
get,list,watchongatewayclassesandget,update,patchongatewayclasses/status;get,list,watchonnamespacesfor theallowedRoutesselectors (withwatchNamespace, only that Namespace byresourceNames);geton the one CustomResourceDefinitiongateways.gateway.networking.k8s.io(byresourceNames), to check the bundle version and to confirm an uninstall.
With watchNamespace only the Gateways and routes of that namespace are served; the
GatewayClass read stays cluster-wide. The trust boundary of the shared address is RBAC on
gateways, see Shared address.
Conformance
The official conformance suite of Gateway API v1.6.3, profile GATEWAY-HTTP, runs in kind against the chart in Gateway mode with port 8080 as an extra HTTP port. Result:
- Core:
partial. 35 tests passed, none failed, one skipped:HTTPRouteMultipleGateways. That test gives two hostname-less Gateways each its own route for/and expects different backends through each Gateway’s own address. On the shared address a request carries nothing that tells the two Gateways apart (see Shared address), so it cannot pass. Per-Gateway addresses, planned as phase 2d, will make the core resultsuccess. - Extended: 17 of 17 passed: every test of the claimed extended features.
The run reads the features from GatewayClass.status.supportedFeatures and writes the
report standard-<version>-default-report.yaml. CI runs it in the workflow
gateway-conformance on pull requests that touch the controller, the chart or the data
plane, nightly, and on release tags, and keeps the report. To run it yourself on a Linux
host with docker, kind, kubectl, helm and Go: conformance/run.sh, see
conformance/README.md in the repository.
Migration
From an Ingress of this controller to an HTTPRoute
An Ingress and an HTTPRoute never serve one host together, and the older claim wins, so a move is two steps with no gap:
- Create the Gateway (if needed) and the HTTPRoute for the host. While the Ingress exists
the route does not take the host: its parent status shows
Accepted: False, reasonHostnameConflict(orAccepted: Truewith aHostnameConflictEvent when it also has other hosts). Check everything else on the route:ResolvedRefs: True, noPartiallyInvalid. - Delete the Ingress (or remove its rule for the host). At the next synchronisation, about
a second later, the HTTPRoute owns the host and its
AcceptedturnsTrue.
A Gateway listener with that explicit hostname is newer than the Ingress and gets
Conflicted: True until step 2. Mind the defaults that differ: Ingress hosts with TLS get an
implicit HTTP-to-HTTPS redirect, a Gateway needs an explicit redirect route on the HTTP
listener; the Ingress input serves ExternalName Services, HTTPRoute does not; annotation
policies (basic authentication, source ranges, CORS) have no HTTPRoute counterpart in this
release and stay on Ingress.
From ingress-nginx
ingress-nginx has no Gateway API support; move first to this controller’s Ingress input
(migration checklist), then host by
host to HTTPRoutes as above, or write HTTPRoutes directly for new hosts. The common
annotations map to HTTPRoute as: rewrite-target to URLRewrite (replaceFullPath or
replacePrefixMatch; no capture groups), upstream-vhost to URLRewrite.hostname,
ssl-redirect to a RequestRedirect route on the HTTP listener, permanent-redirect to
RequestRedirect, canary-weight to weighted backendRefs, canary-by-header to a
header match, proxy-read-timeout to timeouts.backendRequest.
From another Gateway API implementation
Create a GatewayClass of this controller (the chart’s scalws), point a copy of each
Gateway at it with gatewayClassName, and attach the routes to it with a second
parentRefs entry: the route status keeps one entry per controller, so both
implementations report on the same route during the move. Then move DNS to the new
address and remove the old parent. Check before you move:
- Every Gateway gets the shared address; Gateways of different namespaces cannot share a
host, and a host belongs to the namespace of its oldest route unless
crossNamespacePathsis on. - Listeners must use the exposed ports (80, 443 and
extraPorts); other ports arePortUnavailable. - Rules with CORS, RequestMirror, ExternalAuth, extension filters or
backendReffilters answer500; ExternalName and HTTPS or h2c backends are refused. - Routes of a broader hostname are not merged into a more specific host.
- Timeouts bound the response head.
Troubleshooting
kubectl get gatewayclass,gateway -A
kubectl describe gateway -n <ns> <name> # listener conditions, attachedRoutes
kubectl describe httproute -n <ns> <name> # parent conditions, Events
kubectl get events -A --field-selector involvedObject.kind=HTTPRoute,type=Warning
kubectl logs deploy/ingress-scalws -c controller # discovery, status writes, reloads| Symptom | Likely cause | Check |
|---|---|---|
| GatewayClass has no status | The CRDs are missing or appeared less than a minute ago, --gateway-api is off, or controllerName differs from --gateway-controller-name. | scalws_ingress_gateway_api_available, the controller log. |
| Pod stays at 1/2 after enabling the Gateway API | The CRDs exist but the controller cannot list a Gateway API kind (RBAC). | Controller log, kubectl auth can-i list httproutes --as=system:serviceaccount:<ns>:<sa>. |
Listener Accepted: False, PortUnavailable | The port is not exposed, or it serves the other protocol. | The message lists the exposed ports; extraPorts. |
Listener Conflicted: True | Another Gateway of the same port has the same hostname in an older or other namespace, or an Ingress owns the host. | The message names the owner. |
Route Accepted: False, HostnameConflict or NotAllowedByListeners | Another owner holds the host, or an older wildcard of another namespace covers it, or the listeners do not allow the namespace. | HostnameConflict Events; the message. |
Route NoMatchingListenerHostname | The route’s hostnames and the listener’s hostname do not intersect. | Listener and route hostnames. |
500 on some paths | The rule is unsupported (PartiallyInvalid), or a backendRef is invalid (ResolvedRefs: False). | Parent conditions, UnsupportedFilter and UnsupportedField Events. |
503 | No ready endpoints. | NoEndpoints Events, scalwsctl upstreams. |
404 for a path a broader hostname’s route serves | Only the most specific host’s routes are used. | Add the path to a route of the specific host. |
Gateway Programmed: False, AddressNotAssigned | The Service has no address yet, or spec.addresses asks for one it does not have. | kubectl get service ingress-scalws. |
| Wrong certificate | The listener’s Secret is unusable, a name is not covered, or another Gateway’s older certificate holds the name. | ResolvedRefs, OverlappingTLSConfig, TLSSecret Events. |
| Status not updated | No leader, or writes fail. | kubectl get lease, scalws_ingress_status_writes_total{result="error"}. |
Not supported
- GRPCRoute, TLSRoute, TCPRoute, UDPRoute; TLS passthrough; listeners of protocol
TLS,TCPorUDP. - Per-Gateway addresses, Deployments or Services (planned: phase 2d).
- ListenerSet: parent references to a ListenerSet get no status from this controller.
- BackendTLSPolicy, HTTPS and h2c backends, ExternalName backends.
- RequestMirror, CORS, ExternalAuth, extension filters, filters on
backendRefs. - Experimental-channel fields (retries, session persistence, rule names) and Mesh (GAMMA).
- Installing the CRDs.