The platform stores, moves, and serves a lot of data, and the people who get value out of it, analysts, data scientists, engineers, need a place to work with it: write some Python, run a query, build a model, explore. The cloud sells this as SageMaker, Vertex AI Workbench, or a Databricks seat, metered by the hour and by the user. The open answer is a multi-user JupyterHub, and on the platform you have built it gives every person their own notebook environment, logged in through Keycloak, with direct access to the lakehouse and every other data source you own.

This series rebuilds my 2020 Apress book, Advanced Platform Development with Kubernetes, for 2026. The approach behind it comes from building and running data platforms in production for more than twenty years.

§A Data Lab You Own

A managed notebook service is convenient and it is metered per user and per compute hour. A team that adopts it widely gets a large bill, and the data the notebooks work on usually has to live in or near that vendor’s environment to perform. For a platform whose premise is that you own the data and the infrastructure, renting the place people explore that data back from a cloud is the wrong shape.

Self-hosted JupyterHub puts the lab on the platform. Each person gets a real, isolated environment, with their own storage and their own kernel, spun up on demand on your own nodes. The data they work with is already here, a service name away, so nothing is copied out to a vendor first. Because it runs on the cluster, a notebook has governed access to the running platform.

§Notebooks Are Where a Platform Proves Itself

A data platform’s pieces prove they connect in a notebook, and that has been true the whole time. In 2020 I ran every worked example of a platform like this from a self-hosted JupyterHub: generating a million fictional records from a notebook, loading them into object storage, projecting a schema over them, and querying the result back through a distributed SQL engine, joins across a NoSQL ring and an RDBMS included. Same notebook, whole platform. Standing that lab up, though, meant a hand-built user image, a hand-tended Hub, and glue at every seam.

The job is unchanged in 2026; the effort is not. The surrounding stack modernized, Presto became Trino and the warehouse became the Iceberg lakehouse, Elasticsearch became OpenSearch, MinIO became SeaweedFS, and JupyterLab went from 1.x to 4.x, while the Hub itself became one maintained chart. Two changes bite anyone following a 2020-era guide: the community notebook images moved from Docker Hub to quay.io/jupyter/, and the authenticator’s authorization model flipped its default, which gets its own treatment below because it produces a lab nobody can log in to.

§How JupyterHub Works on Kubernetes

JupyterHub is two parts worth understanding. The Hub is the always-on front door: it handles login, decides who is allowed in, and manages user servers. When a user logs in, the Hub uses a spawner to start a single-user notebook server just for them, and on Kubernetes that spawner is KubeSpawner, which launches a dedicated pod running JupyterLab. One user, one pod, isolated from the others, with its own CPU, memory, and storage. When the user leaves, the pod can be culled to free the resources, and their files persist on a volume that reattaches next time.

That model is what makes it multi-tenant and affordable at once. There is no big shared server everyone crowds onto; there is a small Hub and a pod per active user, scheduled across the cluster like any other workload, costing nothing when no one is working.

§Deploy With Zero-to-JupyterHub

The canonical install is the Zero-to-JupyterHub Helm chart, and this is one of the few places in this series where Helm is the idiomatic choice rather than a wrapper to avoid: the Hub, its proxy, the spawner configuration, and the scheduling pieces are a coordinated set the project itself maintains as a chart. The configuration lives in a values file, and each key changes how the lab behaves:

# values-jupyterhub.yaml
singleuser:
  image:
    name: registry.apk8s.dev/platform/datalab   # the custom lab image, below
    tag: "v1"
  storage:
    dynamic:
      storageClass: rook-ceph-block
    capacity: 10Gi
  cpu:
    limit: 2
    guarantee: 0.5
  memory:
    limit: 4G
    guarantee: 1G

cull:
  enabled: true
  timeout: 3600

singleuser.image is the environment every user gets, and pointing it at your own registry rather than a stock image is the difference between a demo and a lab; the image is where the team’s libraries live. storage.dynamic.storageClass gives each user a private volume provisioned from the platform’s Ceph storage on first login, and capacity bounds it; the volume outlives the pod, so a user’s work is durable while their compute is disposable. The cpu and memory pairs deserve their asymmetry: the guarantee is what the scheduler reserves, small because most notebook time is idle thinking, and the limit is the burst headroom a heavy query can use, so twenty users fit on hardware sized for a handful of simultaneous heavy ones. cull.enabled with a 3600-second timeout reclaims the pod of anyone idle for an hour, which is the setting that makes per-user pods affordable; their files persist, so culling costs them a thirty-second respawn, not their work.

kubectl create namespace lab
helm repo add jupyterhub https://hub.jupyter.org/helm-chart/
helm install datalab jupyterhub/jupyterhub -n lab -f values-jupyterhub.yaml
kubectl -n lab get pods

NAME                              READY   STATUS    RESTARTS   AGE
continuous-image-puller-8xkzq     1/1     Running   0          2m
hub-7d9c8b6f4-mm2lp               1/1     Running   0          2m
proxy-5f6d9b7c8-qq4rv             1/1     Running   0          2m

The hub and proxy are the standing cost of the lab, two small pods. Expose the proxy through the gateway at lab.apk8s.dev, and it joins the platform behind the same TLS as everything else.

§Log In Through Keycloak

Like every other tool in this series, the data lab keeps no users of its own. It authenticates against the platform’s Keycloak realm with the generic OAuth authenticator, so the same single sign-on that fronts NiFi, Superset, and the catalog fronts the notebooks too. In the platform realm, create a confidential OIDC client named jupyterhub with one valid redirect URI, https://lab.apk8s.dev/hub/oauth_callback, and note the generated client secret. Then wire it up in the values:

hub:
  config:
    JupyterHub:
      authenticator_class: generic-oauth
    GenericOAuthenticator:
      client_id: jupyterhub
      client_secret: "<from the Keycloak client>"
      oauth_callback_url: https://lab.apk8s.dev/hub/oauth_callback
      authorize_url: https://auth.apk8s.dev/realms/platform/protocol/openid-connect/auth
      token_url: https://auth.apk8s.dev/realms/platform/protocol/openid-connect/token
      userdata_url: https://auth.apk8s.dev/realms/platform/protocol/openid-connect/userinfo
      username_claim: preferred_username
      scope: ["openid", "email", "profile"]
      allow_all: true

The endpoint URLs are the realm’s standard OpenID Connect endpoints; username_claim: preferred_username maps the Keycloak username onto the Jupyter username, so the volume a user gets tomorrow matches the one they got today. The line that trips people is allow_all: true. In 2023, OAuthenticator 16 flipped its authorization default: authenticating is no longer authorizing, and a Hub configured like the old guides, with no allow rule at all, sends every valid Keycloak user to a 403. On this platform that default is actually the right instinct, just with a better rule than allow_all: put lab users in a Keycloak group and use claim_groups_key with allowed_groups so realm membership decides notebook access. allow_all: true is the development shortcut that says any authenticated platform user may in; either way, the decision now has to be explicit, and a login loop that ends in 403 means you have not made it.

A user clicks in, authenticates at Keycloak, and lands in their own freshly spawned lab. Watch the spawn happen:

kubectl -n lab get pods -w

jupyter-craig                     0/1     Pending   0          1s
jupyter-craig                     1/1     Running   0          24s

One pod, named for the user, plus a bound PVC named the same way. That pod is the user’s entire footprint on the cluster, and it is subject to every platform rule other pods live under.

§Build the Lab Image

The single-user image is where the lab earns its keep, because it decides what every notebook can reach. Build it from the community base and add the platform’s client libraries:

FROM quay.io/jupyter/minimal-notebook:2026-07-07

RUN pip install --no-cache-dir \
    trino \
    "psycopg[binary]" \
    sqlalchemy \
    opensearch-py \
    boto3 \
    pandas \
    pyarrow \
    matplotlib

minimal-notebook is the deliberate base: it carries JupyterLab and nothing speculative, so the image stays small and every library present is one the team chose. The additions map one-to-one onto the platform: trino for the lakehouse, psycopg and SQLAlchemy for Postgres, opensearch-py for the search cluster, boto3 for the S3 API on SeaweedFS, and pyarrow because Trino returns columnar results and pandas should receive them that way. Build it in the platform’s CI, push it to the platform registry, bump the tag in the values file, and every user’s next spawn has the new environment. The image is the lab’s shared environment, version-controlled like everything else.

§Notebooks That Reach the Platform

Because the user pods run inside the cluster, every platform service is reachable by its in-cluster name, with no port-forwarding and no data leaving the platform. The sensor readings the MQTT post started collecting two days ago are already queryable from a fresh notebook:

import trino, pandas as pd

conn = trino.dbapi.connect(
    host="trino.data", port=8080, user="craig", catalog="iceberg"
)
df = pd.read_sql("""
    SELECT device, avg(temp) / 1000.0 AS avg_temp_c, max(load) AS peak_load
    FROM analytics.sensor_readings
    WHERE collected > current_timestamp - INTERVAL '1' DAY
    GROUP BY device
""", conn)
df

    device  avg_temp_c  peak_load
0   lab-d1       46.81       0.31
1   lab-d2       48.02       0.44

That, a notebook joining the platform’s pieces, is the demonstration that took most of a chapter to stand up in 2020, reduced to eight lines against the 2026 stack. The same kernel can hit OpenSearch for an aggregation the lakehouse would grind on, pull an object from SeaweedFS with boto3, and write results back to Postgres, because the lab is inside the platform rather than a vendor’s copy of it.

For governed access to the Kubernetes API itself, when a notebook needs to launch a job or read a config, the spawned pods run under a ServiceAccount with a deliberately scoped Role, the least-privilege pattern from the platform’s security posture. The next post builds on this, using a notebook to prototype event-driven processing and then promote it to a Kubernetes Job.

§Profiles, Including GPUs

One image does not fit every task, and KubeSpawner’s profile list puts the choice at spawn time: the user picks an environment, and the override block adjusts the pod.

singleuser:
  profileList:
    - display_name: "Standard"
      description: "2 CPU / 4 GB, the default lab"
      default: true
    - display_name: "GPU training"
      description: "Lands on a GPU node; for model work"
      kubespawner_override:
        extra_resource_limits:
          nvidia.com/gpu: "1"
        tolerations:
          - key: dedicated
            value: gpu
            effect: NoSchedule

The GPU profile requests the nvidia.com/gpu resource, which only nodes with GPUs advertise, and tolerates the taint that keeps casual workloads off them, the same selector-and-taint pairing the platform uses everywhere placement matters. In 2020, a GPU notebook meant a hand-configured node and a user who knew exactly which environment to ask for; now an analyst picks a profile from a dropdown and the scheduler finds the GPU. Later in this series the cluster gains an on-prem GPU node, and this profile is how the lab reaches it.

§Operating the Lab

Storage that follows the user. Each user gets a dynamic Ceph volume that persists their work and reattaches on every login, so their environment is durable without anyone managing it. The platform’s storage backs it, and the platform’s backups protect it.

Quotas keep it affordable. A ResourceQuota on the lab namespace bounds what the whole lab can consume, so the guarantee-versus-limit oversubscription stays a scheduling strategy rather than an incident. With culling reclaiming idle pods, a team’s notebooks do not quietly hold a cluster’s worth of resources overnight.

Grow the lab through the image, not the pods. When the team needs a new library, it goes into the datalab image through CI, not pip install in a running notebook, which evaporates at the next spawn. The occasional in-notebook install is fine for an experiment; the image is where the environment actually lives.

§When Something Is Wrong

A user’s server never spawns. Usually storage or image. The per-user PVC cannot bind, which points at Rook, or the single-user image cannot be pulled; kubectl -n lab describe pod jupyter-<user> names which in its events. The spawn timeout is also worth raising for large images, and the chart’s continuous-image-puller exists precisely to pre-stage the image on every node.

Login succeeds at Keycloak, then 403s at the Hub. Authorization, not authentication: the OAuthenticator 16 default. Set allow_all: true or, better, the allowed_groups rule, as covered above.

Login fails or loops without a 403. The OIDC client. The redirect URI in Keycloak must match https://lab.apk8s.dev/hub/oauth_callback exactly, the client secret in the values must be current, and the realm endpoints must be reachable from the Hub pod.

A notebook cannot reach a data source. In-cluster name or credentials, not networking. Confirm the service name (trino.data, platform-pg-rw.data) and that the notebook has the credentials it needs; a Cilium network policy could also be denying the path, visible in Hubble.

Notebooks cannot launch jobs or read cluster resources. The ServiceAccount’s Role does not grant it. Scope the Role to exactly what the notebooks need rather than reaching for cluster-admin.

§The Lab in the AI Era

The data lab is the human window into the platform, and it is worth being precise about how that changes with capable models in the loop, because the answer is not “the notebook writes itself.” Two things actually change. Inside the lab, an assistant extension like Jupyter AI goes into the datalab image like any other library, pointed at whatever model the platform sanctions, and an analyst gets drafting help with the query and the plot without their data leaving the cluster. Outside the lab, agents do not need notebooks at all: the platform’s MCP layer gives a frontier model the same governed access to Trino and the catalog that the notebook user has, with the same identity, audit, and least-privilege posture. The lab and the MCP gateway are the same idea for two audiences, humans and agents respectively, each getting scoped access to the one platform instead of copies of its data. And when an agent’s analysis is worth keeping, a notebook is still the right artifact to ask it for: executable, reviewable, and versioned, which is more than can be said for a chat transcript.

§What You Have

A multi-user data lab on the platform: every person their own isolated JupyterLab on demand, logged in through single sign-on, with persistent personal storage, optional GPU profiles, and direct, governed access to the lakehouse, the databases, the search cluster, and the object store. It is the workbench a data team needs, owned by you, with no per-seat or per-hour cloud notebook bill and no data copied out to a vendor to use it.

Next I use that lab for real, prototyping an event-driven pipeline in a notebook and then promoting it to run as a Kubernetes Job, triggered by objects landing in storage.