The platform is about to grow tools that people log into: a dataflow canvas, a BI front end, dashboards. You do not want a separate username and password for each of them, and you do not want to rent identity from Okta or Auth0, billed per user, for something this foundational. One identity provider gives every tool single sign-on, central user management, and multi-factor auth from one place. That provider is Keycloak, and it runs on your cluster, backed by the Postgres you already operate and reached through the gateway.

This series rebuilds my 2020 Apress book, Advanced Platform Development with Kubernetes, for 2026. The approach behind it comes from building and running data platforms in production for more than twenty years.

§Why Self-Host Identity

Identity is the wrong place to take on lock-in, and it is the place people most often do. Every tool you add authenticates against it, every user lives in it, and the roles and group memberships that govern who can see what accumulate inside it. That makes the identity provider the single hardest thing to migrate off later: moving it means re-integrating every application and re-homing every user at once. Okta, Auth0, and Cognito know this, which is why they meter you per active user, on OpenID Connect, a protocol that is an open standard so that you do not have to pay rent on it. You are paying a subscription to speak a free language.

Keycloak is the open answer, and a mature one. It is a CNCF project now, the modern Quarkus-based distribution starts in a second or two rather than the old WildFly-based one the 2020 book used, and it does everything you would otherwise buy: single sign-on across applications over OIDC and SAML, social and LDAP federation, multi-factor authentication including passkeys, fine-grained roles, and a full admin UI. It is stateless, keeping all of its state in a database, so it slots straight onto the Postgres cluster already running and inherits that cluster’s failover and backups for free. The thing you would least want to lose is protected by infrastructure you already built.

§A Database for Keycloak

CloudNativePG manages databases declaratively, so ask the cluster for a keycloak database rather than creating one by hand.

apiVersion: postgresql.cnpg.io/v1
kind: Database
metadata:
  name: keycloak
  namespace: data
spec:
  cluster:
    name: platform-pg
  name: keycloak
  owner: app

§Deploy Keycloak

Keycloak needs an admin bootstrap password of its own. Put it in a secret, then deploy. Because Keycloak is stateless, this is an ordinary Deployment pointed at the keycloak database, told it sits behind a proxy that terminates TLS, and given its public hostname.

kubectl create namespace iam

kubectl -n iam create secret generic keycloak-admin \
  --from-literal=password='change-this-admin-password'

apiVersion: apps/v1
kind: Deployment
metadata:
  name: keycloak
  namespace: iam
spec:
  replicas: 1
  selector:
    matchLabels:
      app: keycloak
  template:
    metadata:
      labels:
        app: keycloak
    spec:
      containers:
        - name: keycloak
          image: quay.io/keycloak/keycloak:26.1   # pin a current version
          args: ["start"]
          env:
            - { name: KC_DB, value: postgres }
            - { name: KC_DB_URL, value: "jdbc:postgresql://platform-pg-rw.data:5432/keycloak" }
            - { name: KC_DB_USERNAME, value: app }
            - name: KC_DB_PASSWORD
              valueFrom:
                secretKeyRef: { name: platform-pg-app, key: password }
            - { name: KC_HOSTNAME, value: "https://auth.apk8s.dev" }
            - { name: KC_PROXY_HEADERS, value: xforwarded }
            - { name: KC_HTTP_ENABLED, value: "true" }
            - { name: KC_HEALTH_ENABLED, value: "true" }
            - { name: KC_BOOTSTRAP_ADMIN_USERNAME, value: admin }
            - name: KC_BOOTSTRAP_ADMIN_PASSWORD
              valueFrom:
                secretKeyRef: { name: keycloak-admin, key: password }
          ports:
            - { name: http, containerPort: 8080 }
            - { name: management, containerPort: 9000 }
          readinessProbe:
            httpGet: { path: /health/ready, port: 9000 }
            initialDelaySeconds: 30
            periodSeconds: 10
---
apiVersion: v1
kind: Service
metadata:
  name: keycloak
  namespace: iam
spec:
  selector:
    app: keycloak
  ports:
    - { name: http, port: 8080, targetPort: 8080 }

KC_HTTP_ENABLED=true lets Keycloak serve plain HTTP inside the cluster because the gateway terminates TLS, and KC_PROXY_HEADERS=xforwarded tells it to trust the forwarded protocol and host so it builds correct redirect URLs. Those two settings are where most behind-a-proxy Keycloak problems come from; get them wrong and logins half-work, redirecting to the wrong scheme or host. The health endpoint lives on the management port, 9000.

This is a single replica, right for development. For production you run several, and Keycloak forms a cluster so sessions and caches are shared; the Keycloak Operator handles that clustering for you and adds a Keycloak custom resource and declarative realm imports, which is the path to take once identity is central to the platform. The deployment here is plain so the moving parts are visible; the operator is the same shape with the wiring done for you.

§Expose It Through the Gateway

An HTTPRoute puts the admin console and the OIDC endpoints at auth.apk8s.dev.

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: keycloak
  namespace: iam
spec:
  parentRefs:
    - name: platform-gateway
      namespace: gateway
  hostnames:
    - "auth.apk8s.dev"
  rules:
    - backendRefs:
        - name: keycloak
          port: 8080

With DNS pointed at the gateway, the admin console is at https://auth.apk8s.dev, and you log in with admin and the bootstrap password.

§Create a Realm and a Client

A realm is an isolated set of users, roles, and applications. Create one for the platform, then register each application as an OpenID Connect client inside it.

In the admin console, create a realm named platform. Inside it, create your users, or federate an existing LDAP or social login. Then add a client for each tool. The next post wires NiFi to this, so create that client now:

• Client ID: nifi
• Client type: OpenID Connect, confidential (it has a client secret)
• Valid redirect URIs: https://nifi.apk8s.dev/*
• Copy the generated client secret; the application uses it to complete the login handshake.

Every additional tool, Superset, OpenSearch Dashboards, Jupyter, gets its own client the same way, all pointed at the same realm and discovery URL:

https://auth.apk8s.dev/realms/platform/.well-known/openid-configuration

That single URL is how applications find Keycloak’s login, token, and key endpoints.

§How a Tool Logs In

It is worth walking the handshake once, because every user-facing tool from here on uses it and the troubleshooting later depends on understanding it. When a user opens NiFi, NiFi sees no session and redirects the browser to Keycloak’s login endpoint, named in that discovery document, carrying its client ID and the redirect URI it wants to come back to. The user authenticates to Keycloak, the one place credentials and multi-factor live, and Keycloak redirects the browser back to NiFi’s redirect URI with a short-lived authorization code. NiFi then exchanges that code, plus its client secret, directly with Keycloak for tokens: an ID token saying who the user is, and an access token carrying their roles. NiFi trusts it because it is signed by Keycloak’s key, which NiFi fetched from the discovery document.

The user typed a password once, at Keycloak, and every tool that follows trusts the result. The redirect URI is the security boundary: Keycloak only ever sends the code back to a URI the client registered, which is why the redirect URI list on each client has to be exact.

§Realm as Code

Clicking through the admin console is fine to learn the model and wrong as the source of truth. A realm, with its clients, roles, and settings, exports to a single JSON document, and that document is what belongs in version control. The Keycloak Operator imports it declaratively with a KeycloakRealmImport resource, so the entire identity configuration is applied from a manifest like everything else on the platform.

apiVersion: k8s.keycloak.org/v2alpha1
kind: KeycloakRealmImport
metadata:
  name: platform-realm
  namespace: iam
spec:
  keycloakCRName: keycloak
  realm:
    realm: platform
    enabled: true
    clients:
      - clientId: nifi
        redirectUris: ["https://nifi.apk8s.dev/*"]
        # ...

Now adding a tool to the platform is a reviewed change to a file, not a remembered sequence of clicks in a UI, and standing the realm back up on a fresh cluster is one apply.

§Operating Identity

A hosted identity provider does several things behind its bill: it federates your directory, enforces your MFA policy, stays available, and keeps backups. Own it and those are yours, and Keycloak does each.

Federate existing users. You rarely start empty. Keycloak’s user federation connects an existing LDAP or Active Directory so those users log in with the credentials they already have, and social login (Google, GitHub) is a few fields in the same realm. Users do not have to be re-created to come under single sign-on.

Enforce policy in one place. Multi-factor authentication, password rules, session lifetimes, and brute-force detection are realm settings, applied to every client at once. Turning on passkeys or requiring MFA for an admin role happens once and covers every tool, which is the security upside of central identity over per-app logins.

Stay available, and recover. Production runs multiple Keycloak replicas clustered by the operator, so a pod loss does not log everyone out. And because all of its state lives in the keycloak database, the Postgres backups and point-in-time recovery from earlier already protect it. There is no separate identity backup to arrange; protecting the database protects identity.

Upgrade deliberately. New Keycloak versions migrate the database schema on startup, so you move versions one at a time with the database backed up first, the same measured upgrade discipline as the rest of the platform.

§When Something Is Wrong

Login redirects fail with “Invalid redirect URI.” The single most common OIDC error. The redirect URI the application sent does not exactly match one registered on its client. Match them precisely, including the scheme, host, and path pattern; a trailing-slash or http-versus-https difference is enough to reject it.

Logins redirect to the wrong host, or break behind the gateway. The proxy settings. Confirm KC_HOSTNAME is the public https://auth.apk8s.dev and KC_PROXY_HEADERS=xforwarded is set, so Keycloak builds URLs with the external host the gateway presents rather than its internal pod address.

Keycloak will not start, or hangs on boot. The database. Confirm it can reach platform-pg-rw.data:5432 and that the keycloak database and credentials exist; on first start it creates its schema, which fails silently if the connection is wrong.

Tokens are rejected as expired or not-yet-valid. Clock skew between Keycloak and the validating application. Token validity is time-sensitive, which is why the cluster build set up time sync on every node; confirm it is still running.

§What You Have

A single sign-on layer you own: one set of users and one login, with multi-factor auth and roles enforced in one place, exposed over OIDC for any tool to authenticate against, backed by your own Postgres and behind your own gateway. No per-user identity bill, and no critical dependency you would dread migrating off later.

Next I put it to work. NiFi is the first user-facing tool on the platform, and rather than its own local login, it authenticates against this Keycloak realm, the start of one identity across everything.