Users' guide

Gateway API

scalws-ingress also serves the Kubernetes Gateway API: GatewayClass, Gateway, HTTPRoute and ReferenceGrant (gateway.networking.k8s.io), at the level of the GATEWAY-HTTP conformance profile. It is the same controller, in the same pod, as the Ingress controller: both inputs are translated together into one configuration of the scalwsd next to it, and they can run alone or together. This guide covers the install, the GatewayClass and its features, how Gateway listeners map onto the pod’s ports, attachment and host ownership, the translation of HTTPRoute with every difference from the API’s wording, TLS, what fails closed, status, Events and metrics, the conformance result and migration.

Design record: ADR-0052. The Ingress input, the shared parts of the controller (channels, isolation, start-up) and the data plane are described in ingress-controller.md, load-balancing.md and configuration.md.

Note: the container image ghcr.io/xsmtx/scalws and the Helm chart oci://ghcr.io/xsmtx/charts/scalws are not yet published for public download. Until they are, the steps on this page describe how the Kubernetes deployment works but cannot be followed as written; the Linux packages are the supported way to install 1.0.0.

Install

The CRDs

The Gateway API resources are CustomResourceDefinitions that the cluster must have before the controller can serve them. Install the standard channel of v1.6.3:

kubectl apply --server-side -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.6.3/standard-install.yaml

The file’s SHA-256 is 356c2b8e286a964dc20995435659ad1095386329974793af10bbfe3eff977fe5. Supported bundles are the standard channel v1.3.0 to v1.6.x; with another bundle (or the experimental channel) the controller logs a warning at start and serves best effort. Only the fields of the standard channel are read.

The chart never installs the CRDs, and has no option to. They are cluster-wide and shared by every Gateway API implementation in the cluster, and Helm deletes the CRDs it owns when a release is uninstalled, which would delete every Gateway and HTTPRoute of every implementation. Install and upgrade them as their own step, as you would for any shared cluster API.

Chart values

The Gateway API is a part of the chart’s ingress controller mode. It is in chart versions after 1.0.0-rc.1:

helm install ingress oci://ghcr.io/xsmtx/charts/scalws --version <chart version> \
    --set ingressController.enabled=true \
    --set ingressController.gatewayAPI.enabled=true
ValueDefaultMeaning
ingressController.gatewayAPI.enabledfalseServe Gateway API resources (--gateway-api).
ingressController.gatewayAPI.controllerNamescalws.com/gateway-controllerThe spec.controllerName of the GatewayClasses this release serves (--gateway-controller-name).
ingressController.gatewayAPI.gatewayClass.createtrueCreate a GatewayClass with that controllerName.
ingressController.gatewayAPI.gatewayClass.namescalwsIts name.
ingressController.gatewayAPI.extraPorts[]Extra Gateway listener ports, for example [{port: 8080, protocol: HTTP, containerPort: 9080}]. See Listeners and ports.
ingressController.ingress.enabledtrueServe Ingress resources (--ingress). false makes a Gateway-only install; the IngressClass and the Ingress RBAC are then left out.
ingressController.crossNamespacePathsfalseApplies to both inputs, see Ownership.
ingressController.publishServicethe release’s ServiceIts address goes into Gateway.status.addresses as well as the Ingress status.

The other ingressController.* values (kind, watchNamespace, defaultCertificate, service, resources) work as in Ingress mode. At least one of ingress.enabled and gatewayAPI.enabled must be on.

With gatewayClass.create the chart checks that the CRDs are present and fails with the install command above when they are not; set gatewayClass.create=false to install the chart first and create the GatewayClass yourself. Two releases in one cluster need different controllerName values; with the same value both would serve the same Gateways and write their status. deploy/kubernetes/gateway/ holds manifests rendered from the chart for both inputs with port 8080 as an extra port (deploy/check.sh keeps them in sync).

Check the install

kubectl get gatewayclass scalws          # ACCEPTED True
kubectl get pods -l app.kubernetes.io/instance=ingress
kubectl get service ingress-scalws       # EXTERNAL-IP

A first Gateway, the usual HTTP-to-HTTPS redirect and a route to a Service:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata: {name: shop, namespace: shop}
spec:
  gatewayClassName: scalws
  listeners:
    - {name: http, port: 80, protocol: HTTP, hostname: shop.example.com}
    - name: https
      port: 443
      protocol: HTTPS
      hostname: shop.example.com
      tls: {certificateRefs: [{name: shop-example-com}]}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: redirect, namespace: shop}
spec:
  parentRefs: [{name: shop, sectionName: http}]
  rules:
    - filters: [{type: RequestRedirect, requestRedirect: {scheme: https, statusCode: 301}}]
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata: {name: web, namespace: shop}
spec:
  parentRefs: [{name: shop, sectionName: https}]
  rules:
    - backendRefs: [{name: web, port: 80}]
kubectl get gateway -n shop shop         # PROGRAMMED True, ADDRESS of the Service
kubectl get httproute -n shop            # then: kubectl describe httproute -n shop web
curl -i -H 'Host: shop.example.com' http://192.0.2.10/      # 301 to https://shop.example.com:443/
curl -k --resolve shop.example.com:443:192.0.2.10 https://shop.example.com/

(192.0.2.10 stands for the Service’s address.) The controller generates two applications in the tenant gateway, one per scalwsd listener:

- name: g_http_h_shop_example_com
  listeners: [http]
  domains: [shop.example.com]
  routes: [{path: /, runtime: {type: redirect, scheme: https, port: 443, status: 301}}]
- name: g_https_h_shop_example_com
  listeners: [https]
  domains: [shop.example.com]
  routes: [{path: /, runtime: {type: proxy, upstream: {pool: p_shop_web_80}}}]

Gateway applications are named g_<scalwsd listener>_ plus the Ingress-style host name (g_<listener>_default for the catch-all); pools are p_<namespace>_<service>_<port> and are shared with Ingresses that use the same Service port with the same settings.

GatewayClass and features

Every GatewayClass whose spec.controllerName is the controller’s (--gateway-controller-name) is served and gets Accepted: True. A class with parametersRef gets Accepted: False, reason InvalidParameters, and its Gateways are not served: there are no class parameters.

The controller lists what it supports in GatewayClass.status.supportedFeatures, sorted by name; the conformance suite reads the list there, so the claim itself is tested:

FeatureBasis
Gateway, HTTPRoute, ReferenceGrantcore
GatewayHTTPListenerIsolationlistener-scoped hosts and the most-specific-listener rule
GatewayPort8080only when port 8080 is an extra HTTP port (extraPorts)
HTTPRoute303RedirectStatusCode, HTTPRoute307RedirectStatusCode, HTTPRoute308RedirectStatusCoderedirect statuses
HTTPRouteBackendProtocolWebSocketappProtocol: kubernetes.io/ws
HTTPRouteBackendTimeout, HTTPRouteRequestTimeouttimeouts.backendRequest, timeouts.request
HTTPRouteHostRewrite, HTTPRoutePathRewritefilter URLRewrite
HTTPRouteMethodMatching, HTTPRouteQueryParamMatchingmethod and queryParams matches
HTTPRoutePathRedirect, HTTPRoutePortRedirect, HTTPRouteSchemeRedirectfilter RequestRedirect
HTTPRouteResponseHeaderModificationfilter ResponseHeaderModifier

Not claimed: HTTPRouteRequestMirror and its variants, HTTPRouteCORS, HTTPRouteBackendRequestHeaderModification, HTTPRouteParentRefPort, HTTPRouteDestinationPortMatching, HTTPRouteBackendProtocolH2C, GatewayStaticAddresses, GatewayAddressEmpty, GatewayInfrastructure, GatewayHTTPSListenerDetectMisdirectedRequests, GatewayRouteHostnameIntersectionPrecedence, the client certificate features, the experimental features (retries, session persistence, named rules), BackendTLSPolicy, ListenerSet, GRPCRoute, TLSRoute, TCPRoute and UDPRoute.

Listeners and ports

scalwsd cannot add or remove listeners without a restart, and the controller never restarts it. The pod’s listeners are therefore fixed by the chart: http (Service port 80) and https (Service port 443), plus one p<port> listener per extraPorts entry. The controller learns the mapping from --gateway-listener PORT=NAME (the chart passes 80=http, 443=https and one per extra port); a port is HTTPS when its scalwsd listener has tls. A Gateway listener is placed on the scalwsd listener of its port:

Gateway listenerResult
port exposed, protocol the port’s protocol (HTTP or HTTPS)Served on that scalwsd listener.
port not exposedAccepted: False, reason PortUnavailable; the message lists the exposed ports.
port exposed with the other protocol (HTTP on 443)Accepted: False, reason PortUnavailable, “port 443 of this deployment serves HTTPS”.
protocol TLS, TCP, UDP or unknownAccepted: False, reason UnsupportedProtocol, supportedKinds: [].

To serve another port, add it to extraPorts: each entry adds a scalwsd listener on containerPort, a container port, a Service port and the flag. protocol is HTTP or HTTPS; port may not be 80 or 443, and containerPort may not be one the chart uses (8080, 8443, 8081, 10254). Extra ports serve Gateways only, never Ingresses.

Every generated Gateway application names the one scalwsd listener it serves (the application key listeners), so one host can have a redirect on port 80 and a backend on port 443. Gateway applications have no implicit HTTPS redirect: in the Gateway API the redirect is an explicit route on the HTTP listener, as in the example above. A request that no rule of a host matches gets 404.

Shared address

Every Gateway of a ScalWS GatewayClass is served by the controller’s pods and has the address of its Service (--publish-service): load balancer ingress addresses, else external IPs, else the cluster IP, each with its type (IPAddress or Hostname). Until the Service has an address the Gateway is Programmed: False, reason AddressNotAssigned. A Gateway with spec.addresses is served, but stays Programmed: False (AddressNotAssigned) unless every requested address is one of the Service’s; an address type other than IPAddress and Hostname makes it Accepted: False, reason UnsupportedAddress.

What this costs: two Gateways on port 80 share one IP address and port, so nothing in a request says which Gateway it was meant for; only the host tells them apart. Hosts are therefore owned (next sections): Gateways of different namespaces cannot serve the same host, and the first owner keeps it. Restrict who may create Gateways of the class: grant create on gateways (gateway.networking.k8s.io) only to the teams that should own hosts on the shared address, as you would for a shared ingress controller. Routes attach under each Gateway’s allowedRoutes, so application teams need only HTTPRoutes.

Per-Gateway addresses (one Service per Gateway) are planned for the next phase (2d). Until then the conformance test HTTPRouteMultipleGateways cannot pass, see Conformance.

Attaching routes

A Gateway is served when its gatewayClassName names a served GatewayClass. Its listeners are checked in this order: protocol and port (above); allowedRoutes.kinds (only HTTPRoute of group gateway.networking.k8s.io; other kinds are left out of supportedKinds and, when any were asked for, ResolvedRefs: False, reason InvalidRouteKinds); TLS; listeners of the Gateway with the same port and hostname as an earlier one get Conflicted: True, reason HostnameConflict.

An HTTPRoute attaches through each parentRefs entry that names a served Gateway (group gateway.networking.k8s.io and kind Gateway, the defaults; the namespace defaults to the route’s) to every listener of it that:

  • matches sectionName and port when they are set,
  • is accepted,
  • allows the route’s namespace (allowedRoutes.namespaces.from: Same, the default, All, Selector with a label selector on the Namespace, or None) and its kind,
  • has a hostname that intersects the route’s hostnames.

Parent references to Gateways of other namespaces need no ReferenceGrant; the listener’s allowedRoutes decides. The result is the route’s Accepted condition for that parent:

SituationAcceptedReason
Attached to at least one listenerTrueAccepted
sectionName or port match no listenerFalseNoMatchingParent
Listeners match, but none allows the namespace or kind, or they are conflictedFalseNotAllowedByListeners
Listeners allow it, but no hostname intersectsFalseNoMatchingListenerHostname
Every intersecting host is refused by the configuration (for example *.com)FalseUnsupportedValue
Every host it would serve belongs to another ownerFalseHostnameConflict, or NotAllowedByListeners when every one is under an older wildcard of another namespace
Every rule is unsupportedFalseUnsupportedValue
Left out because the configuration would not validate with itFalseInvalidConfiguration

HostnameConflict and InvalidConfiguration are this implementation’s reasons; the API allows them.

Hostnames and listener isolation

The hosts a route serves through a listener are the intersection of the listener’s hostname and the route’s hostnames: an empty side matches everything (both empty: the catch-all of that listener). A wildcard *.example.com covers one or more labels below example.com, as the API defines, and becomes the domain **.example.com; a route *.example.com under a listener *.shop.example.com serves **.shop.example.com.

A host is served only through the most specific listener of the same Gateway and port whose hostname matches it: a route attached to a broad listener never answers for a host that a more specific listener of that Gateway names (GatewayHTTPListenerIsolation). Every explicit listener hostname gets an application even without routes, so such a host answers 404 instead of falling through to a broader one.

Across hosts the most specific domain wins (exact, then *. one label, then the longest **. suffix, then the catch-all), and only that host’s routes are used: routes of a broader hostname are not merged into a more specific one. So a request for api.example.com with routes for api.example.com and for *.example.com only ever sees the routes of api.example.com; a path those routes lack answers 404. The API leaves this to the implementation (GatewayRouteHostnameIntersectionPrecedence is not claimed).

Ownership across Ingress and Gateway API

The two inputs share the listeners and hosts of the pod, so one rule decides who serves a host on a scalwsd listener. Claims:

ClaimantOwnerClaim time
Ingress rule for a host (or a rule without host, or spec.defaultBackend: the catch-all *)the Ingress namespace, on http and https togethercreationTimestamp of the oldest Ingress of that namespace with the host
Gateway listener with an explicit hostnamethat Gateway (the Gateways of one namespace together)the Gateway’s creationTimestamp
HTTPRoute host through a listenerthe Gateway and the route’s namespacethe route’s creationTimestamp
  • The oldest claim wins; ties go to Ingress, then Gateway, then HTTPRoute, then namespace/name. A loser never takes traffic: an Ingress gets a PathConflict Event; an HTTPRoute that lost some hosts on a parent stays Accepted for the others and gets a HostnameConflict Event naming the hosts; one that lost all of them gets Accepted: False. Host patterns are compared as written, so *.example.com and a.example.com are different hosts, except for the next rule.
  • A host under an older wildcard of another namespace is refused. When an owner of one namespace holds *.example.com (or **.example.com), a newer claim of another namespace on a.example.com is not served: an Ingress gets PathConflict naming the owner; an HTTPRoute gets a HostnameConflict Event naming the hosts, and Accepted: False with reason NotAllowedByListeners when it loses all its hosts that way. Owners of the same namespace are not refused by their own wildcard, also across inputs: one team may combine its Ingress and Gateway API hosts under its wildcard.
  • Ingress and HTTPRoute never share a host, with or without --cross-namespace-paths: their precedence rules, regex mode, HTTPS redirect defaults and policies differ and cannot be merged into one application. This is what makes a migration predictable, see Migration.
  • Inside a Gateway, a host belongs to the namespace of its oldest attached route. Routes of other namespaces for that host are not served there (as for Ingresses of several namespaces). This matters for the common hostname-less listener with allowedRoutes: All: without the rule any namespace could add a path to another team’s host, or catch with a broad prefix the paths it does not serve.
  • Gateways of one namespace share their hosts: two Gateways of a namespace may have the same listener hostname on one port, and the routes attached through either merge into the host under the namespace rule above; a match two routes share keeps the oldest (PathConflict). Gateways of different namespaces with the same explicit listener hostname on one port: the newer listener gets Conflicted: True, reason HostnameConflict (the message names the owning Gateway), and serves nothing. The same holds for a listener hostname owned by an Ingress. Hostname-less listeners claim nothing by themselves; their routes claim the hosts they use.
  • A Gateway delegates through its own listener hostnames. When a Gateway owns an explicit listener hostname, the routes its allowedRoutes admit (of any namespace) may serve that host, even when an Ingress of the Gateway’s namespace holds an older wildcard above it: the Gateway’s own claim already passed the wildcard rule. The namespace rule among the routes still applies.
  • An HTTPRoute that is not served keeps its claims (all rules unsupported, or left out by isolation), and its matches answer 500, as an unserved Ingress keeps its paths, so refusing a route never hands its users to a newer one.

--cross-namespace-paths (chart crossNamespacePaths) lifts only the namespace rules inside one input: Ingresses of several namespaces merge into a host, routes of every namespace a listener allows merge their rules into a host of the Gateways of one namespace, and their wildcard refusals among each other are lifted. Refusals between Ingress and Gateway API owners, and between Gateways of different namespaces, stay. Turn it on only when every namespace that can attach routes is trusted with every host.

Translation of HTTPRoute

Each matches entry of each rule becomes one route of the application of each host the route serves (a rule without matches is one route on prefix /).

HTTPRouteScalWS route
path Exactpath_type: exact
path PathPrefixpath_type: prefix on whole segments; a trailing / is ignored (/match/ is /match), as the API says
path RegularExpressionpath_type: regex, written (?:<value>)$, so anchored at both ends; linear-time syntax only (no look-around, no back-references): an expression the engine refuses makes the rule unsupported
methodmethods: [<method>]
headers Exact, RegularExpressionheaders with value, or regex: "^(?:<value>)$"; names lower-cased; only the first entry of a name counts (API rule)
queryParams Exact, RegularExpressionquery with value or regex: "^(?:<value>)$"; only the first entry of a name counts
filter RequestHeaderModifierrequest_headers (set, add, remove); set of Host becomes rewrite.host
filter ResponseHeaderModifierresponse_headers
filter RequestRedirectredirect runtime: scheme, hostname as host, path.replaceFullPath as path, path.replacePrefixMatch as prefix, statusCode (301 by default, 302, 303, 307, 308), port (see below)
filter URLRewriterewrite.host from hostname; rewrite.path from replaceFullPath; rewrite.prefix from replacePrefixMatch
timeouts.requestroute request_timeout; 0s is no limit
timeouts.backendRequestproxy response_timeout; 0s is one year (the longest value; the data plane refuses 0)
one backendRefproxy runtime with upstream: {pool}
several backendRefsweighted backends: [{pool, weight}]
an invalid backendRefits share answers 500 (a {status: 500, weight} entry); all invalid, all weights 0, or no backendRefs and no redirect: the route answers 500

Proxy routes use the controller’s defaults, as for Ingresses: preserve_host: true, websocket: true, connect_timeout: 5s, response_timeout: 60s, pools with retries: 2 (see Backends) and members from the EndpointSlices.

Differences from the API’s wording and from other implementations:

  • Timeouts bound the response head. timeouts.request and timeouts.backendRequest limit the time until the response head is received, not the whole response body. A timeouts.request above 60 s needs a timeouts.backendRequest at least as long, because the default response_timeout of 60 s still applies.
  • Redirects always carry a port: the given port; else the well-known port of a given scheme (443 for https, 80 for http); else the listener’s port. Without it the data plane would use the request’s port, which inside the pod is the container port.
  • $ and ? in replaceFullPath of URLRewrite are literal. They are written as $$ and %3F, so neither starts a capture reference nor a query.
  • Weights are reduced and scaled. The data plane takes weights from 0 to 1000; the weights are divided by their greatest common divisor and, when the largest is still above 1000, scaled to 1000 with every non-zero weight kept at 1 or more. When that changes a ratio the route gets a WeightRounded Event.
  • No fallback to a broader hostname (see Hostnames).
  • Regular expressions are matched by a linear-time engine (ADR-0050); the paths are anchored at both ends.
  • ExternalName Services are refused, see below. The Ingress input still serves them.

Backends

A backendRef is valid when its group is "" and its kind Service (the defaults), the Service and the port exist, the reference is allowed (same namespace, or a ReferenceGrant), the Service is not of type ExternalName, and the port’s appProtocol is empty, http or kubernetes.io/ws. Members are the ready, serving and not terminating addresses of the Service’s EndpointSlices for the port; a Service without ready endpoints answers 503, which the API allows, and the route gets a NoEndpoints Event. Problems set ResolvedRefs: False on every parent of the route:

CauseReason
group or kind not a core Service (ServiceImport too)InvalidKind
ExternalName ServiceInvalidKind, “ExternalName Services are not supported”
Service or port missingBackendNotFound
another namespace without a ReferenceGrantRefNotPermitted
appProtocol kubernetes.io/h2c, kubernetes.io/wss, https or another valueUnsupportedProtocol

ExternalName Services are refused because an ExternalName can point at a Service of another namespace or at a node-local address, past ReferenceGrant, and the API has no way to allow them per namespace. HTTPS and h2c backends need BackendTLSPolicy and HTTP/2 to backends, which are not part of this release.

A backendRef to another namespace needs a ReferenceGrant in that namespace:

apiVersion: gateway.networking.k8s.io/v1beta1
kind: ReferenceGrant
metadata: {name: shop-routes, namespace: backends}
spec:
  from: [{group: gateway.networking.k8s.io, kind: HTTPRoute, namespace: shop}]
  to: [{group: "", kind: Service}]          # add name: to allow one Service only

A deleted or changed ReferenceGrant takes effect at the next synchronisation.

Precedence

The rules of all routes of a host are ordered as the API defines: route creationTimestamp, then namespace/name, then rule index, then match index. The data plane then tries them in the API’s precedence: exact path, longest prefix, then among equal paths a method match first, then more header matches, then more query matches, then the earlier rule. Regular expression paths come after the prefixes. Two matches with the same path, method, headers and query: the first wins, the later one is not served and gets a PathConflict Event.

TLS

An HTTPS listener (tls.mode: Terminate, the default) takes its certificateRefs: core Secrets of type kubernetes.io/tls, checked with the code scalwsd loads certificates with. The first usable one is served. A Secret in another namespace needs a ReferenceGrant in that namespace with from: [{group: gateway.networking.k8s.io, kind: Gateway, namespace: <gateway namespace>}] and to: [{group: "", kind: Secret}]. Problems give ResolvedRefs: False:

CauseReason
Secret missing, wrong type, unparsable, key does not match, a group or kind other than core Secret, no certificateRefsInvalidCertificateRef
another namespace without a ReferenceGrantRefNotPermitted

A listener without a usable certificate is Programmed: False, reason Invalid, and serves nothing. Certificates go to the memory volume, as for Ingresses, and onto the scalwsd listener of the port with these server names:

  • A listener with a hostname serves its certificate for that name. *.example.com stays *.example.com for SNI: an X.509 wildcard covers one label, so a client asking for a.b.example.com gets the default certificate. That is a limit of X.509, not of ScalWS.
  • A listener without hostname serves its certificate for the hosts its routes serve (only those its routes own, never the certificate’s other names, which may belong to another owner). Names the certificate does not cover are left out with a TLSSecret Event.
  • A listener whose hostname belongs to another owner (Conflicted: True) adds no certificate; the owner’s certificate serves that name.
  • Default certificate. On each HTTPS port, the oldest hostname-less listener whose routes own the catch-all * there (routes without hostnames whose catch-all no older Ingress or Gateway route holds) provides the default certificate, served for an absent or unknown SNI name. When every Gateway HTTPS listener on the port is in one namespace and no Ingress certificate is on it (one tenant), the oldest hostname-less listener whose routes serve any host provides it. Its certificate is written as an unnamed default entry when it gets no name (server_names: [], see tls.md). On a port shared by several namespaces, a listener whose routes all have hostnames does not: its certificate is never shown to other tenants’ clients. A configured --default-ssl-certificate (chart defaultCertificate) always wins, also on extra HTTPS ports; without either, the controller’s generated certificate ScalWS Ingress Controller Fake Certificate stays the default.

On the main HTTPS port the Ingress certificates come first, then the Gateway certificates (Gateways oldest first, then listener order); a server name keeps the first certificate that names it. When two Gateways provide certificates for the same name on a port, the older Gateway’s is used and both listeners get OverlappingTLSConfig: True, reason OverlappingCertificates. HTTPS listeners of one Gateway on one port whose hostnames overlap (no hostname overlaps every hostname) get OverlappingTLSConfig: True, reason OverlappingHostnames, as the API requires; it is informational.

A wildcard certificate SAN directly under one label (*.org) is accepted for serving example.org, as Go clients and RFC 6125 do; browsers refuse such certificates. cert-manager works as for Ingresses: the controller picks up the Secret when it is written and renewed.

Fail closed

Access-related settings that ScalWS cannot honour make the affected part stop serving instead of serving without them:

SettingResultWhy
Client certificate validation (a Gateway spec.tls.frontend entry for the listener’s port, or a listener tls field the controller does not know, such as frontendValidation)listener Accepted: False, reason UnsupportedValueServing without validating clients would expose backends the owner meant to protect.
Listener tls.options, tls.mode: Passthroughlistener Accepted: False, reason UnsupportedValueOptions usually tighten TLS; ignoring them would weaken it.
Filters RequestMirror, CORS, ExternalAuth, ExtensionRef, unknown filter types, any filter on a backendRefthe rule’s matches answer 500No exact counterpart; see below.
RequestHeaderModifier that adds or removes Host, or changes a hop-by-hop or framing headerthe rule’s matches answer 500The data plane forbids these changes.
A field the controller does not know inside a match or a filter, an unknown enum valuethe rule’s matches answer 500A newer field may narrow matching; ignoring it would widen what the rule matches.
A path or header regex the engine refusesthe rule’s matches answer 500

An unsupported rule is not dropped: a dropped /admin rule would let a broader rule such as / serve the request unprotected. Each of its matches becomes a route answering 500 with its own key, and the route’s parent status gets PartiallyInvalid: True, reason UnsupportedValue, with a message that starts with Dropped Rule and names the rule indexes and the cause (the prefix the API requires). When every rule is unsupported the parent is Accepted: False, reason UnsupportedValue, and the matches still answer 500. An UnsupportedFilter or UnsupportedField Event carries the detail.

CORS and RequestMirror are not supported in this release: the data plane’s cors cannot express the API’s * forms, and it has no mirroring. Unknown fields outside matches and filters (rule-level experimental fields, Gateway spec.infrastructure, Gateway spec.tls.backend) are ignored with an UnsupportedField Event; a backend that requires a client certificate refuses the connection, so ignoring spec.tls.backend exposes nothing. spec.infrastructure.parametersRef makes the Gateway Accepted: False, reason InvalidParameters.

No value of a Gateway resource is formatted into configuration text: hostnames, paths, header names and values, regular expressions and redirect fields go into typed fields and through the same validation scalwsd runs. One bad HTTPRoute is left out by the isolation that Ingresses get (its matches answer 500 and it gets Accepted: False, reason InvalidConfiguration); Gateways are never left out, their problems are found before translation.

Status

The leader pod (the Lease of the Ingress controller) writes the status of GatewayClasses, Gateways and HTTPRoutes of this controller, after the reload, and only when it changed. Every condition carries observedGeneration and keeps its lastTransitionTime while its status does not change. GatewayClass and Gateway status is written with merge patches of the fields the controller owns. HTTPRoute.status.parents is shared with other controllers: the controller reads it, replaces only its own entries (its controllerName), keeps the others, removes its entries for parents the route no longer references, and writes the list with the resourceVersion it read; a conflict is retried at the next report. The API allows at most 32 parent entries; a route that would exceed it gets no status write and a warning in the log.

ObjectConditionTrue reasonsFalse reasons
GatewayClassAcceptedAcceptedInvalidParameters
GatewayAcceptedAccepted; ListenersNotValid when some listeners are invalidListenersNotValid (no listener valid), InvalidParameters, UnsupportedAddress
GatewayProgrammedProgrammedInvalid, AddressNotAssigned, Pending
ListenerAcceptedAcceptedPortUnavailable, UnsupportedProtocol, UnsupportedValue
ListenerResolvedRefsResolvedRefsInvalidCertificateRef, RefNotPermitted, InvalidRouteKinds
ListenerConflictedHostnameConflictNoConflicts
ListenerProgrammedProgrammedInvalid, Pending
ListenerOverlappingTLSConfigOverlappingHostnames, OverlappingCertificatesnot set
HTTPRoute parentAcceptedAcceptedsee Attaching routes
HTTPRoute parentResolvedRefsResolvedRefsInvalidKind, BackendNotFound, RefNotPermitted, UnsupportedProtocol
HTTPRoute parentPartiallyInvalidUnsupportedValue (Dropped Rule ...)not set

Pending means scalwsd refused the last configuration change and the controller is retrying; the previous configuration keeps serving. Listener status also has supportedKinds (HTTPRoute, or none for an unsupported protocol) and attachedRoutes, the number of routes accepted on that listener (counted also when the listener is not programmed, as the API says). GatewayClass.status.supportedFeatures is the list above.

Events

Warning Events on the HTTPRoute or the Gateway, emitted by the leader once while the problem lasts (after a leader change an Event may be emitted again once). Conditions carry the state; Events carry the detail that does not fit a condition message.

ReasonMeaning
HostnameConflictHosts of the route are not served on a parent because another owner holds them; the message names the hosts.
PathConflictA match lost to an older equal match of another route; it is not served.
ServiceNotFoundA referenced Service or its port is missing (ResolvedRefs: False, BackendNotFound); its share answers 500.
NoEndpointsThe Service has no ready endpoints; its share answers 503.
UnsupportedFilterA filter, a regular expression or a header change cannot be served; the rule answers 500.
UnsupportedFieldAn unknown field: inside a match or filter the rule answers 500; elsewhere it is ignored.
WeightRoundedThe backendRefs weights were scaled and a ratio changed.
TLSSecretDetail of a certificate problem (a name the certificate does not cover, a name already served by another certificate); never Secret content.
InvalidConfigurationThe route was left out because the configuration did not validate with it.

Metrics and readiness

On :10254, next to the Ingress metrics, which count both inputs (scalws_ingress_syncs_total, scalws_ingress_reload_failures_total):

MetricMeaning
scalws_ingress_gateway_api_available1 while the Gateway API watches run (the CRDs are served), else 0.
scalws_ingress_gateways{state}Gateways of this controller, programmed or invalid.
scalws_ingress_httproutes{state}HTTPRoutes of this controller, accepted, partial (some rules dropped) or rejected.
scalws_ingress_status_writes_total{kind, result}Status writes by kind (GatewayClass, Gateway, HTTPRoute) and result: ok, conflict or error.

The controller does not need the CRDs to start. With --gateway-api it checks the API group gateway.networking.k8s.io at start and every 60 s:

  • CRDs missing: a warning in the log, the gauge at 0, and the pod serves Ingresses alone; a missing install never keeps an Ingress-serving pod unready.
  • CRDs present at start: the Gateway API caches are part of the readiness wait, without a time limit. A pod whose RBAC lacks a Gateway API kind stays unready (and logs the list error once per kind) rather than serving a configuration without the Gateways.
  • CRDs installed while running: the watches start and the pod stays ready. Structural changes are held until the first lists arrive, at most 60 s; after that they are applied without the Gateway input until the lists complete.
  • CRDs removed: after two discovery checks in a row that find them missing, confirmed by reading the gateways.gateway.networking.k8s.io CRD, the Gateway input stops and the pod keeps serving Ingresses.

Flags of scalws-ingress run

Added to the Ingress controller flags:

FlagDefaultMeaning
--ingresstrueServe Ingress resources; --ingress=false for a Gateway-only install.
--gateway-apioffServe GatewayClass, Gateway, HTTPRoute and ReferenceGrant.
--gateway-controller-namescalws.com/gateway-controllerThe GatewayClass spec.controllerName to serve.
--gateway-listener80=http, 443=httpsPORT=NAME, repeated: a Gateway listener port and the scalwsd listener that serves it.

--ingress=false without --gateway-api is refused at start. --cross-namespace-paths, --publish-service, --default-ssl-certificate and --watch-namespace apply to both inputs. scalws-ingress bootstrap has no new flag: the extra listeners are in the base configuration the chart writes.

RBAC

With gatewayAPI.enabled the chart adds:

  • get, list, watch on gateways, httproutes and referencegrants, and get, update, patch on gateways/status and httproutes/status (ClusterRole, or a Role in watchNamespace);
  • cluster-wide: get, list, watch on gatewayclasses and get, update, patch on gatewayclasses/status; get, list, watch on namespaces for the allowedRoutes selectors (with watchNamespace, only that Namespace by resourceNames); get on the one CustomResourceDefinition gateways.gateway.networking.k8s.io (by resourceNames), to check the bundle version and to confirm an uninstall.

With watchNamespace only the Gateways and routes of that namespace are served; the GatewayClass read stays cluster-wide. The trust boundary of the shared address is RBAC on gateways, see Shared address.

Conformance

The official conformance suite of Gateway API v1.6.3, profile GATEWAY-HTTP, runs in kind against the chart in Gateway mode with port 8080 as an extra HTTP port. Result:

  • Core: partial. 35 tests passed, none failed, one skipped: HTTPRouteMultipleGateways. That test gives two hostname-less Gateways each its own route for / and expects different backends through each Gateway’s own address. On the shared address a request carries nothing that tells the two Gateways apart (see Shared address), so it cannot pass. Per-Gateway addresses, planned as phase 2d, will make the core result success.
  • Extended: 17 of 17 passed: every test of the claimed extended features.

The run reads the features from GatewayClass.status.supportedFeatures and writes the report standard-<version>-default-report.yaml. CI runs it in the workflow gateway-conformance on pull requests that touch the controller, the chart or the data plane, nightly, and on release tags, and keeps the report. To run it yourself on a Linux host with docker, kind, kubectl, helm and Go: conformance/run.sh, see conformance/README.md in the repository.

Migration

From an Ingress of this controller to an HTTPRoute

An Ingress and an HTTPRoute never serve one host together, and the older claim wins, so a move is two steps with no gap:

  1. Create the Gateway (if needed) and the HTTPRoute for the host. While the Ingress exists the route does not take the host: its parent status shows Accepted: False, reason HostnameConflict (or Accepted: True with a HostnameConflict Event when it also has other hosts). Check everything else on the route: ResolvedRefs: True, no PartiallyInvalid.
  2. Delete the Ingress (or remove its rule for the host). At the next synchronisation, about a second later, the HTTPRoute owns the host and its Accepted turns True.

A Gateway listener with that explicit hostname is newer than the Ingress and gets Conflicted: True until step 2. Mind the defaults that differ: Ingress hosts with TLS get an implicit HTTP-to-HTTPS redirect, a Gateway needs an explicit redirect route on the HTTP listener; the Ingress input serves ExternalName Services, HTTPRoute does not; annotation policies (basic authentication, source ranges, CORS) have no HTTPRoute counterpart in this release and stay on Ingress.

From ingress-nginx

ingress-nginx has no Gateway API support; move first to this controller’s Ingress input (migration checklist), then host by host to HTTPRoutes as above, or write HTTPRoutes directly for new hosts. The common annotations map to HTTPRoute as: rewrite-target to URLRewrite (replaceFullPath or replacePrefixMatch; no capture groups), upstream-vhost to URLRewrite.hostname, ssl-redirect to a RequestRedirect route on the HTTP listener, permanent-redirect to RequestRedirect, canary-weight to weighted backendRefs, canary-by-header to a header match, proxy-read-timeout to timeouts.backendRequest.

From another Gateway API implementation

Create a GatewayClass of this controller (the chart’s scalws), point a copy of each Gateway at it with gatewayClassName, and attach the routes to it with a second parentRefs entry: the route status keeps one entry per controller, so both implementations report on the same route during the move. Then move DNS to the new address and remove the old parent. Check before you move:

  • Every Gateway gets the shared address; Gateways of different namespaces cannot share a host, and a host belongs to the namespace of its oldest route unless crossNamespacePaths is on.
  • Listeners must use the exposed ports (80, 443 and extraPorts); other ports are PortUnavailable.
  • Rules with CORS, RequestMirror, ExternalAuth, extension filters or backendRef filters answer 500; ExternalName and HTTPS or h2c backends are refused.
  • Routes of a broader hostname are not merged into a more specific host.
  • Timeouts bound the response head.

Troubleshooting

kubectl get gatewayclass,gateway -A
kubectl describe gateway -n <ns> <name>          # listener conditions, attachedRoutes
kubectl describe httproute -n <ns> <name>        # parent conditions, Events
kubectl get events -A --field-selector involvedObject.kind=HTTPRoute,type=Warning
kubectl logs deploy/ingress-scalws -c controller # discovery, status writes, reloads
SymptomLikely causeCheck
GatewayClass has no statusThe CRDs are missing or appeared less than a minute ago, --gateway-api is off, or controllerName differs from --gateway-controller-name.scalws_ingress_gateway_api_available, the controller log.
Pod stays at 1/2 after enabling the Gateway APIThe CRDs exist but the controller cannot list a Gateway API kind (RBAC).Controller log, kubectl auth can-i list httproutes --as=system:serviceaccount:<ns>:<sa>.
Listener Accepted: False, PortUnavailableThe port is not exposed, or it serves the other protocol.The message lists the exposed ports; extraPorts.
Listener Conflicted: TrueAnother Gateway of the same port has the same hostname in an older or other namespace, or an Ingress owns the host.The message names the owner.
Route Accepted: False, HostnameConflict or NotAllowedByListenersAnother owner holds the host, or an older wildcard of another namespace covers it, or the listeners do not allow the namespace.HostnameConflict Events; the message.
Route NoMatchingListenerHostnameThe route’s hostnames and the listener’s hostname do not intersect.Listener and route hostnames.
500 on some pathsThe rule is unsupported (PartiallyInvalid), or a backendRef is invalid (ResolvedRefs: False).Parent conditions, UnsupportedFilter and UnsupportedField Events.
503No ready endpoints.NoEndpoints Events, scalwsctl upstreams.
404 for a path a broader hostname’s route servesOnly the most specific host’s routes are used.Add the path to a route of the specific host.
Gateway Programmed: False, AddressNotAssignedThe Service has no address yet, or spec.addresses asks for one it does not have.kubectl get service ingress-scalws.
Wrong certificateThe listener’s Secret is unusable, a name is not covered, or another Gateway’s older certificate holds the name.ResolvedRefs, OverlappingTLSConfig, TLSSecret Events.
Status not updatedNo leader, or writes fail.kubectl get lease, scalws_ingress_status_writes_total{result="error"}.

Not supported

  • GRPCRoute, TLSRoute, TCPRoute, UDPRoute; TLS passthrough; listeners of protocol TLS, TCP or UDP.
  • Per-Gateway addresses, Deployments or Services (planned: phase 2d).
  • ListenerSet: parent references to a ListenerSet get no status from this controller.
  • BackendTLSPolicy, HTTPS and h2c backends, ExternalName backends.
  • RequestMirror, CORS, ExternalAuth, extension filters, filters on backendRefs.
  • Experimental-channel fields (retries, session persistence, rule names) and Mesh (GAMMA).
  • Installing the CRDs.