Some work on a platform is a small piece of logic that should run on demand and cost nothing when idle: enrich a record, transform a payload, call out to a model, respond to an event. A full long-running service is too much for it, and the cloud answer is functions, Lambda and its kin, which are useful and locked to the provider. The self-hosted answer used to have an obvious open choice, and that choice changed its terms, so this post is about both the capability and why the project behind it matters. I build scale-to-zero serverless functions on Knative, on the cluster you own.

This series rebuilds my 2020 Apress book, Advanced Platform Development with Kubernetes, for 2026. The approach behind it comes from building and running data platforms in production for more than twenty years.

§Serverless Without the Lock-In

You write a function, the platform runs it when something calls it, scales it up under load, and scales it back to nothing when it is idle, so you pay for nothing while it waits. For event-driven transforms, the glue logic between systems, and the occasional job, that model fits.

The cloud version, Lambda and the rest, delivers it and binds three things to the provider: the runtime, the event sources, and the operational model, none of which move. For a platform built on owning its infrastructure, running this layer yourself keeps the functions, their triggers, and their scaling on your own cluster, where you can see them and where they cost you compute you already have rather than a per-invocation meter that adds up quietly.

§Why Not OpenFaaS Anymore

In 2020 I used OpenFaaS for this layer, and it was a good open choice at the time. I do not lead with it now because it is the same pattern this series keeps documenting. OpenFaaS moved key capabilities out of its Community Edition and into its paid tiers: scale-to-zero, running functions across multiple namespaces, and other features a real deployment wants ended up behind OpenFaaS Pro. The Community Edition still exists, but the parts that make serverless actually serverless, scaling to zero chief among them, are no longer in the free version.

This is the open-core squeeze rather than an outright relicense, and the effect is the same for the thesis of this platform: the open thing is a limited lead-in to the paid thing, and the capability you need is on the other side of a license. So the platform uses Knative, a CNCF project with broad backing, where scale-to-zero and the rest are simply the product, not a teaser for it. It is the same reasoning that put OpenSearch, Valkey, and SeaweedFS in this stack: build on the project whose full capability stays open.

§Install Knative Serving

Knative Serving installs from plain manifests, two for Knative itself and one for a networking layer:

KNATIVE=knative-v1.19.0   # pin the current release

kubectl apply -f https://github.com/knative/serving/releases/download/${KNATIVE}/serving-crds.yaml
kubectl apply -f https://github.com/knative/serving/releases/download/${KNATIVE}/serving-core.yaml
kubectl apply -f https://github.com/knative/net-kourier/releases/download/${KNATIVE}/kourier.yaml

The CRDs define the Service, Revision, and Route kinds the rest of this post uses. The core manifest runs Knative’s four brains in the knative-serving namespace: the controller reconciling those resources, the webhook validating them, the autoscaler making the scale decisions, and the activator, the component that makes zero possible, because something has to catch the request that arrives while zero pods exist. Kourier is the networking layer, an Envoy-based ingress built by the Knative project for exactly this job and nothing else; the platform’s Gateway API stack keeps fronting everything user-facing, with Kourier as the internal hop Knative controls. (A net-gateway-api plugin exists if you would rather Knative program the platform gateway directly; Kourier is the smaller, boring default.)

Two settings finish the install, both patches to Knative’s ConfigMaps:

kubectl patch configmap/config-network -n knative-serving --type merge \
  -p '{"data":{"ingress-class":"kourier.ingress.networking.knative.dev"}}'

kubectl patch configmap/config-domain -n knative-serving --type merge \
  -p '{"data":{"fn.apk8s.dev":""}}'

The first tells Knative to program Kourier. The second sets the domain scheme: every function gets a URL of the form <name>.<namespace>.fn.apk8s.dev, so one wildcard DNS record and one certificate cover every function the platform will ever deploy. That naming decision is the kind of thing that looks cosmetic and is actually structural; make it once, before the first function, and function URLs are predictable forever after.

kubectl -n knative-serving get pods

NAME                          READY   STATUS    RESTARTS   AGE
activator-58c9d7b95-x2m4p     1/1     Running   0          2m
autoscaler-7f96b6c8d-kk21q    1/1     Running   0          2m
controller-6b98f8d55-7rwlj    1/1     Running   0          2m
webhook-5d9c8f7b6-mv93z       1/1     Running   0          2m

§The First Function

A function is a container that answers HTTP, and this one does a job the platform actually has: enriching a record, here by scoring a text field, standing in for any per-record transform a flow needs.

func main() {
    http.HandleFunc("/", func(w http.ResponseWriter, r *http.Request) {
        var rec map[string]any
        if err := json.NewDecoder(r.Body).Decode(&rec); err != nil {
            http.Error(w, err.Error(), http.StatusBadRequest)
            return
        }
        rec["score"] = score(fmt.Sprint(rec["text"])) // the actual enrichment
        w.Header().Set("Content-Type", "application/json")
        if err := json.NewEncoder(w).Encode(rec); err != nil {
            log.Printf("encode: %v", err)
        }
    })
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Nothing Knative-specific appears in the code, which is the point: no SDK, no handler signature from a vendor, just HTTP. Build it into the same scratch-based image shape as the event workers, and deploy it as a Knative Service:

apiVersion: serving.knative.dev/v1
kind: Service
metadata:
  name: enrich
  namespace: functions
spec:
  template:
    metadata:
      annotations:
        autoscaling.knative.dev/min-scale: "0"
        autoscaling.knative.dev/max-scale: "10"
    spec:
      containers:
        - image: registry.apk8s.dev/platform/enrich:v1
          ports:
            - containerPort: 8080

min-scale: "0" is the economics of the whole layer in one annotation: this function may cease to exist while idle. max-scale: "10" is its blast-radius cap, the most of the cluster it may claim under load, and setting it is not optional hygiene; an unbounded function fed by a runaway caller will happily eat the nodes. The containerPort tells Knative where the container listens so its probes and routing agree with the code above.

kubectl apply -f enrich-ksvc.yaml
kubectl -n functions get ksvc enrich

NAME     URL                                     LATESTCREATED   READY
enrich   https://enrich.functions.fn.apk8s.dev   enrich-00001    True

Now watch the part that makes it serverless. With no traffic, the function has no pods:

kubectl -n functions get pods
# No resources found in functions namespace.

curl -s https://enrich.functions.fn.apk8s.dev \
  -d '{"text":"the platform gets more useful every week"}'

{"score":0.83,"text":"the platform gets more useful every week"}

The request took a moment longer than the next one will, and behind it the sequence ran: the request landed on the activator, which was standing in for the absent function; the autoscaler saw demand and scaled the revision to one; the pod came up and the request was released to it. Subsequent requests hit the running pod directly. Leave it idle and watch kubectl get pods -w: after the scale-to-zero grace period, the pod terminates and the function goes back to costing nothing. Zero to serving to zero, observable in two terminals.

Every deploy of the Service creates a Revision, and traffic is split across revisions declaratively:

  traffic:
    - revisionName: enrich-00001
      percent: 90
    - latestRevision: true
      percent: 10

That is a canary in four lines: the new revision takes ten percent until you move the numbers, and rollback is moving them back, no redeploy involved. The revisions themselves are immutable, so the version you roll back to is exactly the version that worked.

§Call It From the Platform

On this platform the natural caller is already here. The NiFi flows from earlier use an InvokeHTTP processor, and a Knative service is just an HTTP endpoint, so a dataflow can hand each record to the function for a transform the built-in processors do not cover, a sentiment score, an embedding, a custom enrichment, and route the result onward. The function scales up while the flow is busy and back to zero when it stops, so the custom transform costs nothing between runs.

The event-driven object processing from the last posts can call functions the same way, and so can anything else on the platform that speaks HTTP. The function becomes an owned, on-demand capability that any part of the platform can reach.

§Event-Driven Autoscaling With KEDA

Knative scales on HTTP traffic. Plenty of platform work is not HTTP-shaped: a consumer draining a Kafka topic, a worker watching a queue. For those, KEDA is the companion, a CNCF graduated project that scales ordinary Deployments on external signals, including down to zero. Be precise about the division of labor, because it is a common confusion: KEDA does not scale Knative services, and does not need to. Knative owns the request-driven functions; KEDA owns the queue-driven consumers; together they give everything on the platform the same property, running only while there is work.

helm repo add kedacore https://kedacore.github.io/charts
helm install keda kedacore/keda -n keda --create-namespace

A ScaledObject then attaches scaling behavior to a consumer Deployment, like the object-processing controller’s heavier cousin, a worker that processes a topic:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
  name: events-consumer
  namespace: functions
spec:
  scaleTargetRef:
    name: events-consumer        # the Deployment to scale
  minReplicaCount: 0
  maxReplicaCount: 20
  triggers:
    - type: kafka
      metadata:
        bootstrapServers: platform-kafka-kafka-bootstrap.kafka:9092
        consumerGroup: enrich
        topic: events
        lagThreshold: "100"

scaleTargetRef names the Deployment KEDA takes over; you stop setting replicas yourself, because KEDA owns it now. minReplicaCount: 0 is scale-to-zero for consumers, and lagThreshold: "100" is the sizing rule: one replica per hundred messages of consumer-group lag, up to the maxReplicaCount cap. The consumer scales with the backlog it exists to drain: idle when the topic is quiet, fanning out when a burst lands, gone again when it clears. The autoscaling signal is the platform’s own Kafka, not a cloud metric you rent access to.

§Operating Serverless

Scale-to-zero is the economics. The reason this layer is worth running is that idle functions cost nothing, so you can deploy many small capabilities without paying for them to wait. That is the model Lambda sells and OpenFaaS moved behind a paywall, running here on your own nodes for the cost of the capacity they use only while working.

Revisions are your rollouts. Every deploy is a revision, and traffic splitting across revisions gives you canary releases and instant rollback by shifting percentages, without a separate deployment tool. A bad version is reverted by moving traffic back rather than redeploying.

Cold starts are the tradeoff to manage. Scaling from zero means the first request after idle waits for a pod to start, and how long depends almost entirely on image size, which is why the ten-megabyte Go images from the last post matter here too. Where even that latency is unacceptable, set min-scale to one and keep a warm instance, trading a little idle cost for none of the wait. That is a per-function choice, made in an annotation.

§When Something Is Wrong

The function URL returns nothing or times out. The networking layer. Confirm the Kourier pods in kourier-system are running, that config-network names the Kourier ingress class, and that the wildcard DNS for the function domain points where Kourier listens; kubectl -n functions get ksvc showing READY True with a dead URL is almost always DNS or the ingress-class patch.

It never scales to zero. Min-scale or a lingering connection. Confirm the annotation is min-scale: "0" and that nothing is holding the function open with constant traffic; a health checker pinging the function every few seconds is enough to keep it alive forever.

KEDA does not scale on events. The trigger config. Confirm the bootstrap address and consumer group match what the consumer actually uses, and that there is real lag on the topic; KEDA scales on the signal it can see, and a wrong consumer group reads as zero lag forever.

The first request after idle is slow. A cold start, working as designed. If it is unacceptable for that function, raise min-scale to keep one instance warm, and check the image size before reaching for that lever.

§Functions Are Agent-Sized Units

Of everything on this platform, functions are the piece best matched to how AI agents actually produce code. A function is one handler, one image, one twenty-line resource: small enough to review in full, deployed as an immutable revision, canaried by percentage, and rolled back by arithmetic. That is the entire risk-management story an agent-written piece of production code needs, built into the runtime. My working pattern is exactly this: the agent gets the record shape and the transform requirement, produces the handler and the Service manifest, the code passes the same gates all Go here passes, and the revision takes ten percent of traffic while the dashboards decide the rest. And because idle functions cost nothing, the platform can afford to keep the experiments: an enrichment that runs weekly, a one-off transform kept for reruns, a half-proven idea at revision three. Scale-to-zero turns “should this exist” from a capacity question into a cleanup question, which is a much better question to have.

§What You Have

Scale-to-zero serverless functions on your own cluster: small capabilities that run on demand, cost nothing idle, scale on HTTP traffic through Knative or on Kafka lag through KEDA, roll out by revision with canary splits, and are called by the NiFi flows and event pipelines already on the platform. It is the serverless model the cloud meters and OpenFaaS gated, on a CNCF project that keeps the whole capability open, running on infrastructure you control.

That completes the platform’s compute and data layers. Before the final stretch, one accounting is owed: the 2020 book had a Blockchain chapter, and the next post is about what happened to it.