The clearest reason to put your own hardware in the hybrid cluster is the GPU. Sustained AI work on rented cloud GPUs runs up a large bill, and a GPU you own pays for itself quickly when you actually use it. This is the final piece of the platform, and it is what makes the optional in-cluster AI, self-hosted model inference and embeddings, affordable: adding GPUs to Kubernetes, the modern way, which is far less work than the 2020 book’s version.

This series rebuilds my 2020 Apress book, Advanced Platform Development with Kubernetes, for 2026. The approach behind it comes from building and running data platforms in production for more than twenty years.

§Why On-Prem GPUs

The economics drive the decision. A capable GPU rented by the hour in a cloud, run continuously for the kind of work AI actually involves, costs more in a few months than buying the card outright. For bursty, occasional work the cloud is fine; for sustained inference, fine-tuning, or a team’s daily experimentation, owned hardware wins by a wide margin, and it keeps your data and your models on machines you control. This is the same self-hosting math as the rest of the series, and GPUs are where the cloud margin is highest.

That math is what makes the AI tier of this platform practical. Giving frontier models access through the MCP gateway is the primary use, and it runs against hosted models. The optional in-cluster pieces, running an open model with Ollama or vLLM, generating embeddings locally, serving a classic model, all want a GPU, and they only make sense if that GPU is not bleeding you on a cloud meter. Owning the GPU lets you leave self-hosted AI running.

The card itself can be consumer hardware, and for a lab or a small team it should be. In 2020 I used a GeForce GTX 1070 in a generic workstation and paused to note NVIDIA’s licensing: consumer GeForce drivers are not licensed for data-center deployment. That note still applies in 2026, unchanged. A GeForce card in your office serving your own experiments is what the hardware is for; an organization racking GPUs for production should read NVIDIA’s terms and price the data-center line. Between those poles is a wide band of small teams where a couple of prosumer cards do real work, which is exactly the situation this platform is built for.

§Six Pages Then, One Install Now

Nothing shows how far self-hosting has come like GPUs. In 2020, making one GPU workstation into a Kubernetes node took six pages of my book to document, because every step was manual and version-sensitive: a fresh Ubuntu 18.04 install; adding the ppa:graphics-drivers repository and NVIDIA’s own apt repositories; installing nvidia-driver-440, nvidia-container-runtime, and nvidia-modprobe at versions that had to agree with each other and with CUDA 10.2; loading kernel modules by hand; installing k3s with INSTALL_K3S_SKIP_START=true so it would not start before the next step; and then writing containerd’s config.toml yourself, registering the NVIDIA runtime with the correct runtime_type, before finally starting k3s. A mismatch anywhere produced a node that looked healthy and could not run GPU work. It was the right procedure for its time, and it was exactly the kind of expert-only ritual that made people say self-hosting was not worth it.

The NVIDIA GPU Operator absorbed all of it. On nodes it discovers GPUs on, the operator installs and manages the entire stack as containers: the kernel driver, the container toolkit that wires containers to the GPU, the device plugin that advertises GPUs to the scheduler, node labeling by hardware feature, and the monitoring exporter. The driver itself runs as a versioned container the operator upgrades and rolls back like any other workload, which retires the whole class of failure a hand-installed driver invited. This is the boring-infrastructure thesis at the hardest-to-configure corner of the platform: the version-sensitive expertise got encoded into an operator.

§Install the Operator

The operator installs by Helm, with one platform-specific wrinkle worth understanding rather than pasting. The GPU node in this cluster runs k3s, and k3s keeps containerd’s config and socket in its own paths, not the stock ones. The operator’s toolkit component edits containerd config to register the NVIDIA runtime, the same edit that was a hand-written config.toml in 2020, so it must be told where k3s keeps things:

helm repo add nvidia https://helm.ngc.nvidia.com/nvidia

helm install gpu-operator nvidia/gpu-operator \
  -n gpu-operator --create-namespace \
  --set toolkit.env[0].name=CONTAINERD_CONFIG \
  --set toolkit.env[0].value=/var/lib/rancher/k3s/agent/etc/containerd/config.toml \
  --set toolkit.env[1].name=CONTAINERD_SOCKET \
  --set toolkit.env[1].value=/run/k3s/containerd/containerd.sock

On a kubeadm cluster with stock containerd, the two --set pairs disappear and the defaults are right. Either way, the operator finds the GPU nodes on its own through feature discovery; the dedicated=gpu:NoSchedule taint from the hybrid-cluster post does not block it, because the operator’s pods tolerate it. Give it a few minutes on first install, since it is compiling and loading a kernel driver, then check the one line that matters:

kubectl describe node lab-gpu | grep -A2 'Capacity'

Capacity:
  cpu:                16
  nvidia.com/gpu:     1

nvidia.com/gpu appearing in the node’s capacity is the operator’s whole job condensed to one field: the scheduler now knows this node can run GPU work.

§Request a GPU Like Any Resource

A pod asks for a GPU the way it asks for CPU or memory, and the scheduler places it on a node that has one. The proof, then as now, is nvidia-smi from inside a container:

apiVersion: v1
kind: Pod
metadata: { name: gpu-check, namespace: gpu-operator }
spec:
  restartPolicy: Never
  tolerations:
    - { key: dedicated, value: gpu, effect: NoSchedule }
  containers:
    - name: cuda
      image: nvidia/cuda:12.8.0-base-ubuntu24.04
      command: ["nvidia-smi"]
      resources:
        limits:
          nvidia.com/gpu: 1

kubectl apply -f gpu-check.yaml
kubectl -n gpu-operator logs gpu-check

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 570.86.10    Driver Version: 570.86.10    CUDA Version: 12.8     |
|-------------------------------+----------------------+----------------------|
| GPU  Name         Persistence | Bus-Id        Disp.A | Volatile Uncorr. ECC |
|   0  GeForce RTX 4070 Ti  Off | 00000000:01:00.0 Off |                  N/A |
+-----------------------------------------------------------------------------+

In 2020, producing this same table was the payoff at the end of the six-page ritual, at driver 440 and CUDA 10.2. Here it is the first pod after one Helm install. The tolerations line is the one addition a GPU requester needs on this cluster, matching the taint that keeps everything else off the card, and nvidia.com/gpu: 1 in limits is the request itself; GPUs are always whole-number limits, never overcommitted by the scheduler, which is precisely why sharing them takes a mechanism of its own.

§Share a GPU: Time-Slicing and MIG

A GPU requested whole by one pod sits idle whenever that pod is not using it, which is wasteful for a card you paid for. Two mechanisms let multiple workloads share one. Time-slicing oversubscribes: the device plugin advertises one physical GPU as several schedulable ones, and the pods that land there take turns on the hardware with no memory isolation between them, which suits notebooks, development, and light inference where the workloads can be trusted to coexist. MIG, Multi-Instance GPU, available on the data-center cards, partitions one GPU into hardware-isolated instances with their own memory and compute slices, real isolation for real multi-tenancy.

Time-slicing is configured as a ConfigMap the device plugin reads, plus one patch telling the operator to use it:

apiVersion: v1
kind: ConfigMap
metadata:
  name: time-slicing-config
  namespace: gpu-operator
data:
  any: |-
    version: v1
    sharing:
      timeSlicing:
        resources:
          - name: nvidia.com/gpu
            replicas: 4

kubectl apply -f time-slicing-config.yaml
kubectl patch clusterpolicies.nvidia.com/cluster-policy --type merge \
  -p '{"spec":{"devicePlugin":{"config":{"name":"time-slicing-config","default":"any"}}}}'

replicas: 4 is the sharing factor: the one card now advertises as four, and four single-GPU pods schedule onto it. The key name any is the profile name, applied here as the default for every GPU node; a larger fleet can name per-node profiles and label nodes to select them. The verification is the same capacity field as before, now multiplied:

kubectl describe node lab-gpu | grep nvidia.com/gpu:
#   nvidia.com/gpu:  4

Sharing is what lets a single card serve a team. The data lab’s GPU profile, a couple of analysts’ notebooks, and a small inference service all draw on one time-sliced card, rather than each demanding a whole GPU that mostly idles. For affordability, this matters as much as owning the hardware, because the difference between “one workload per card” and “four” is the difference between needing four cards and needing one.

§Put It to Work: Self-Hosted Inference

With a GPU in the cluster, the optional AI pieces become practical, and they deploy like everything else here, as a manifest. An open model served by Ollama is a Deployment that requests the GPU and a Service the platform reaches by name:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: ollama
  namespace: ai
spec:
  replicas: 1
  selector:
    matchLabels: { app: ollama }
  template:
    metadata:
      labels: { app: ollama }
    spec:
      tolerations:
        - { key: dedicated, value: gpu, effect: NoSchedule }
      containers:
        - name: ollama
          image: ollama/ollama:0.9.5   # pin a current release
          ports:
            - containerPort: 11434
          resources:
            limits:
              nvidia.com/gpu: 1
          volumeMounts:
            - { name: models, mountPath: /root/.ollama }
      volumes:
        - name: models
          persistentVolumeClaim: { claimName: ollama-models }

The model cache lands on a Ceph volume so pulled models survive the pod, and the single GPU limit here is one of the time-sliced replicas, which for embedding generation and small-model inference is plenty. This is the serving layer behind the local embeddings and semantic search work I have written about: generating embeddings over your own data, continuously, without an API meter, is exactly the workload that justifies the card. Serving a classic model with KServe, or vLLM for heavier open-model inference, requests the hardware the same way.

The leverage in 2026 is still giving frontier models access and context through MCP; in-cluster inference is the supporting role, for the work you want entirely on your own hardware, the data you will not send out, the embeddings you generate constantly enough that an API would cost more than a card. Owned GPUs let you run that work continuously without watching a meter.

§Operating GPUs

Watch them. The GPU Operator includes DCGM and its exporter, publishing GPU utilization, memory, and temperature as Prometheus metrics, so the cards land on the same dashboards as the rest of the platform. Without those metrics you cannot tell whether a GPU is idle or saturated, and with a shared card that difference is the whole capacity plan.

Choose a sharing strategy deliberately. Time-slicing for cooperative, development, and light-inference workloads; MIG where you need isolation on a card that supports it; a whole GPU for a job that genuinely needs all of it. The strategy is configuration, not hardware, so revisit it as the workloads change.

Schedule with taints. GPU nodes are valuable and scarce, so they stay tainted, and only workloads that tolerate the taint may land there. Combined with the hybrid cluster’s region labels, GPU work finds the GPU node wherever in the world it physically sits, which is the pairing the last post built and this one collects on.

Let the operator own the driver. The upgrade path for the whole GPU stack is upgrading the operator, which rolls the driver container like any other versioned component. The one rule is to never mix in a host-installed driver alongside it; that conflict is the last surviving relative of the 2020 version-matching problem, and it is entirely avoidable by leaving the host bare.

§When Something Is Wrong

Pods requesting a GPU stay Pending. The capacity is not advertised. Check the operator’s pods in gpu-operator are healthy, especially the driver DaemonSet on first install, since it compiles against the node’s kernel; until the device plugin advertises nvidia.com/gpu, the scheduler has nowhere to place the pod. kubectl describe node shows whether the resource exists.

A pod runs but cannot see the GPU. The container toolkit wiring. nvidia-smi failing inside a pod that was scheduled to a GPU node points at the toolkit config, and on k3s the first suspects are the containerd paths from the install section; the toolkit pod’s logs say what it edited and where.

Time-slicing does not multiply the GPU. The device plugin did not pick up the config. Confirm the ConfigMap name and profile key match the ClusterPolicy patch, and watch the device plugin pod restart and re-advertise; the node capacity number is the ground truth.

Driver or CUDA mismatch after an upgrade. A host-installed driver is fighting the operator’s. Remove the host driver and let the operator reconcile; one owner for the driver, always.

§What You Have, and the Platform Whole

GPUs as schedulable, shareable capacity in the cluster, installed by an operator instead of a six-page driver ritual, monitored like everything else, and ready to run self-hosted inference and embeddings affordably because the hardware is yours. The cloud margin on GPUs, among the highest there is, is one you no longer pay for the work you run continuously.

That completes the platform. You started with a cluster on plain virtual machines and built, one liberally-licensed piece at a time, a foundation of runtime, network, storage, and observability; a data spine of databases, streaming, search, object storage, and a lakehouse; the tools people work in, an integration engine, a BI front end, a notebook lab, all behind one single sign-on; a metadata catalog and an MCP gateway that make the whole thing legible to AI; and now your own GPUs spanning cloud and hardware you own. Every piece is FOSS you can trust not to be relicensed out from under you, every piece runs on infrastructure you control, and the labor that used to require a team of experts is now an agent with this series as its frame of reference. The cloud vendors will rent you each piece of this on a meter, forever. You can build the whole thing, own it outright, and run it with an agent. That was the bet of the 2020 book, and in 2026 it holds.