The 2020 edition exposed services to the outside world with the Nginx Ingress controller, a daemon on every node terminating ports 80 and 443. That controller is gone. The Kubernetes project retired ingress-nginx in 2026 after the IngressNightmare vulnerabilities made its annotation-driven configuration model untenable, and the broader industry had already moved to its successor. The Ingress API it served was always a bit of a kludge, a thin resource with the real configuration smuggled in through controller-specific annotations. The replacement is the Gateway API, a standard, and the Cilium you installed when you built the cluster already implements it.

This series rebuilds my 2020 Apress book, Advanced Platform Development with Kubernetes, for 2026. The approach behind it comes from building and running data platforms in production for more than twenty years.

§What Changed, and Why

Ingress had one resource and a pile of annotations. You wrote a few lines of real YAML, and then everything that mattered, the rewrites, the timeouts, the headers, the TLS behavior, the rate limits, went into nginx.ingress.kubernetes.io/... annotations that meant nothing to any other controller. Your manifests stopped being portable the moment you used them, because they were half standard resource and half vendor dialect. Worse, that annotation surface was where IngressNightmare lived: configuration smuggled as strings, parsed by a privileged controller, turned into a remote code execution path. The model was a security liability and a lock-in vector at the same time.

The Gateway API replaces it with a small set of real, typed resources, where the things you used to express as magic strings are now actual fields with actual schemas. A controller validates them, an agent can reason about them, and they mean the same thing on Cilium, Envoy Gateway, Istio, or Kong. It is a graduated standard with multiple implementations: the configuration is portable because the API is the contract rather than the controller.

§Who Owns What: the Gateway API Model

The part of the Gateway API that looks like extra ceremony at first is worth understanding before you apply anything. The API splits a single concern, “expose a service,” across resources owned by different people, and that split maps to how a platform is actually run and secured.

A GatewayClass names an implementation. Cilium registers one called cilium, and that is infrastructure-level: the cluster operator picks the controller, once.

A Gateway declares the listeners, the ports, the protocols, the TLS. This is the platform team’s resource, the front door itself. It lives in a namespace you control, it holds the certificates, and it decides which namespaces are even allowed to attach routes to it.

An HTTPRoute declares routing rules, this hostname and path go to that Service. This belongs to the application teams, in their own namespaces, next to the services they expose. They never touch the Gateway or its TLS; they just attach a route to a front door the platform team published.

That separation is least privilege expressed as an API. An app team can publish a route without holding permission to rewrite the cluster’s TLS or its listeners. The platform team can change the certificate strategy without coordinating with every app. A ReferenceGrant resource gates the cross-namespace references explicitly, so nothing reaches across a namespace boundary unless someone deliberately allowed it. Ingress had none of this; it was one resource that either you could write or you could not.

§Install the Gateway API CRDs

The Gateway API resource types are not bundled with Kubernetes or Cilium; you install the CRDs yourself. Use the standard channel, matching the version your Cilium release supports (check the Cilium release notes; this uses v1.2.1).

kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/v1.2.1/standard-install.yaml

§Turn On Gateway API in Cilium

Enable the feature in Cilium and restart its components so the operator begins watching Gateway resources. Cilium’s Gateway API support builds on the kube-proxy replacement and L7 proxy you already turned on, which is why this slots in without a second data path.

cilium upgrade --set gatewayAPI.enabled=true

kubectl -n kube-system rollout restart deployment/cilium-operator
kubectl -n kube-system rollout restart daemonset/cilium

Cilium creates a GatewayClass named cilium for its controller. That is the class your Gateways reference.

kubectl get gatewayclass

NAME     CONTROLLER                     ACCEPTED   AGE
cilium   io.cilium/gateway-controller   True       30s

§Give the Gateway an Address You Own

Here is the part that is genuinely different on a cluster you own, and the part most tutorials skip because they assume a cloud is standing by to hand you a load balancer. A Gateway needs a reachable IP. In a managed cloud, Cilium creates a LoadBalancer Service and the cloud provisions one of its load balancers for you, automatically. That works, and it also quietly reintroduces exactly the per-provider dependency this series is built to avoid: a cloud load balancer is a billed, vendor-specific resource your platform now leans on. Fine if you are on that cloud to stay, worth naming plainly as a dependency.

The owned-infrastructure answer is to let Cilium itself hand out and announce the address. Define a CiliumLoadBalancerIPPool with addresses that route to your nodes. (The Cilium API group for these resources moves between releases, v2alpha1 to v2; match the version you installed.)

apiVersion: cilium.io/v2alpha1
kind: CiliumLoadBalancerIPPool
metadata:
  name: gateway-pool
spec:
  blocks:
    - cidr: "203.0.113.0/29"

That assigns an address, but something still has to make the network deliver traffic for it to your nodes. Two ways, depending on where you run.

On a shared layer-2 network, bare metal or a colo rack where your nodes share a broadcast domain, turn on L2 announcements and Cilium will answer ARP for the address, so the local network routes it to a node.

cilium upgrade --set l2announcements.enabled=true

apiVersion: cilium.io/v2alpha1
kind: CiliumL2AnnouncementPolicy
metadata:
  name: gateway-l2
spec:
  loadBalancerIPs: true
  interfaces:
    - ^eth0$
  nodeSelector:
    matchLabels:
      kubernetes.io/os: linux

On a routed network where the nodes do not share a layer-2 segment, Cilium speaks BGP instead, advertising the address to your router or top-of-rack switch with a CiliumBGPPeeringPolicy. Either way the result is the same: a stable IP that reaches your Gateway, owned by you, announced by your own networking layer. Point a DNS record at it and the front door has an address.

§Automatic TLS with cert-manager

cert-manager automates free TLS certificates from Let’s Encrypt, and it understands the Gateway API directly now. Install it from its release manifest (use a current version), then turn on its Gateway API support.

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.18.0/cert-manager.yaml

# enable Gateway API support on the controller
# (on older releases this is --feature-gates=ExperimentalGatewayAPISupport=true)
kubectl -n cert-manager patch deployment cert-manager --type=json \
  -p='[{"op":"add","path":"/spec/template/spec/containers/0/args/-","value":"--enable-gateway-api"}]'

Define a ClusterIssuer for Let’s Encrypt. The HTTP-01 challenge proves you control a hostname by serving a token over port 80, and cert-manager does it through the Gateway: it creates a temporary HTTPRoute for the challenge path, lets Let’s Encrypt fetch it, and tears it down.

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: [email protected]
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
      - http01:
          gatewayHTTPRoute:
            parentRefs:
              - name: platform-gateway
                namespace: gateway
                kind: Gateway

HTTP-01 validates one concrete hostname at a time, which is right for a handful of services. Once you are publishing many, nifi., kib., auth., and so on, you will want a single *.apk8s.dev wildcard, and a wildcard cannot be proven over HTTP-01. That needs the DNS-01 challenge, where cert-manager proves control by writing a TXT record through your DNS provider’s API. It is a second solver on the same issuer, with a provider token in a Secret.

    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cloudflare-api-token
              key: token

Use HTTP-01 per-host while there are few services, and switch to a DNS-01 wildcard once the list grows. Both are free and both auto-renew.

§Define the Gateway

The Gateway declares the listeners. The cert-manager.io/cluster-issuer annotation tells cert-manager to watch this Gateway, read its hostnames, and populate the TLS secret automatically.

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: platform-gateway
  namespace: gateway
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-prod
spec:
  gatewayClassName: cilium
  listeners:
    - name: http
      protocol: HTTP
      port: 80
      hostname: "echo.apk8s.dev"
      allowedRoutes:
        namespaces:
          from: All
    - name: https
      protocol: HTTPS
      port: 443
      hostname: "echo.apk8s.dev"
      tls:
        mode: Terminate
        certificateRefs:
          - name: echo-apk8s-tls
      allowedRoutes:
        namespaces:
          from: All

The allowedRoutes.namespaces.from: All is the policy decision that lets app teams in any namespace attach routes to this front door. Tighten it to a label selector when you want to restrict which namespaces may publish. cert-manager sees the annotation, requests a certificate for echo.apk8s.dev, completes the challenge through the Gateway, and writes the result into the echo-apk8s-tls secret the HTTPS listener references.

§Route a Real Service

Deploy an actual service and put it behind the Gateway. A small Deployment and Service in their own namespace stand in for any app team’s workload.

apiVersion: v1
kind: Namespace
metadata: { name: demo }
---
apiVersion: apps/v1
kind: Deployment
metadata: { name: echo, namespace: demo }
spec:
  replicas: 2
  selector: { matchLabels: { app: echo } }
  template:
    metadata: { labels: { app: echo } }
    spec:
      containers:
        - name: echo
          image: ealen/echo-server:latest
          ports: [{ containerPort: 80 }]
---
apiVersion: v1
kind: Service
metadata: { name: echo, namespace: demo }
spec:
  selector: { app: echo }
  ports: [{ port: 80, targetPort: 80 }]

The HTTPRoute lives in demo, next to the service it exposes, and attaches to the platform Gateway in the gateway namespace. Because the route and its backend Service are in the same namespace, no ReferenceGrant is needed; you only need one of those when a route points at a Service in a different namespace.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: echo
  namespace: demo
spec:
  parentRefs:
    - name: platform-gateway
      namespace: gateway
  hostnames:
    - "echo.apk8s.dev"
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: echo
          port: 80

§Verify End to End

The Gateway should report an address, and the certificate should go Ready.

kubectl get gateway -n gateway
kubectl get certificate -n gateway

NAME               CLASS    ADDRESS          PROGRAMMED   AGE
platform-gateway   cilium   203.0.113.1      True         3m

NAME             READY   SECRET           AGE
echo-apk8s-tls   True    echo-apk8s-tls   2m

With DNS pointing echo.apk8s.dev at that address, the service answers over HTTPS with a real, auto-renewing Let’s Encrypt certificate. Confirm the certificate chain, not just that it loads.

curl -v https://echo.apk8s.dev 2>&1 | grep -E "issuer|subject|HTTP/"

*  subject: CN=echo.apk8s.dev
*  issuer: C=US; O=Let's Encrypt; CN=R13
< HTTP/2 200

That is a real chain from Let’s Encrypt, terminated at your Gateway and routed to a pod the scheduler placed.

§Operating the Front Door

A managed ingress hides what a front door actually does day to day. Owning it means those capabilities are yours, in the open, and the Gateway API exposes them as plain fields rather than annotation incantations.

App teams extend it without touching it. This is the operate-it heart of the model. A new service goes live by adding an HTTPRoute in its own namespace pointing at the shared Gateway, no platform-team change, no controller restart, no edit to the front door itself. The Gateway’s allowedRoutes is the policy; everything else is self-service. That is how a platform scales to many teams without the networking config becoming one contended file.

Split traffic for safe rollouts. Ingress could not do weighted routing without controller-specific annotations; the Gateway API does it with a field. Send ten percent of traffic to a canary and the rest to stable, then shift the weights as confidence grows.

      backendRefs:
        - name: echo
          port: 80
          weight: 90
        - name: echo-canary
          port: 80
          weight: 10

The same matches block routes by header or path when you want to, so a request carrying a particular header can be steered to a new version while everyone else stays on the old one.

See the traffic. The Hubble observability you enabled with Cilium shows flows through the Gateway, so when a route misbehaves you watch the actual packets rather than guess. hubble observe --namespace demo shows requests arriving and where they go.

Renewal takes care of itself. cert-manager renews each certificate well before expiry and rewrites the secret the Gateway reads, with no restart. Confirm it any time with kubectl get certificate -A; a READY of True and a future renewal time is all you need to see. The 2020 edition’s manual TLS secret wiring is gone.

§When Something Is Wrong

The Gateway has no address and PROGRAMMED is False. The IP pool or its announcement is missing. Confirm the CiliumLoadBalancerIPPool exists and, on layer 2, that l2announcements.enabled=true and the CiliumL2AnnouncementPolicy are in place. Without one of those, Cilium assigns no reachable address.

The certificate is stuck and never goes Ready. The ACME challenge cannot complete. For HTTP-01, Let’s Encrypt must reach port 80 at the hostname, so check that DNS already points at the Gateway address and that port 80 is open; the challenge runs over HTTP before HTTPS exists. If the name is a wildcard, HTTP-01 cannot work at all and you need the DNS-01 solver. kubectl describe certificate and the Order/Challenge resources cert-manager creates spell out which step failed.

The route does not attach, or you get a 404 from the Gateway. Either the Gateway’s allowedRoutes does not permit the route’s namespace, or the route’s hostname does not match a listener hostname. kubectl describe httproute shows the attachment status and any rejection reason. A backend Service in another namespace also needs a ReferenceGrant allowing it.

§What You Gained

The whole Nginx Ingress chapter from the original book, the DaemonSet, the config maps, the controller-specific annotations, the manual TLS secret wiring, collapses into a handful of small typed resources and an annotation. The routing is portable across any Gateway API implementation, the configuration is real fields instead of magic strings, app teams publish their own routes without touching the front door, and the certificates renew themselves. The security model that produced IngressNightmare is replaced by one built on least privilege.

The platform now has a front door with automatic TLS. Next I start putting services behind it, beginning with the data layer the rest of this series runs on: PostgreSQL, run by an operator, on the storage you just built.