Postgres is the platform’s default database, and most of the time it is the right answer. Some workloads it is the wrong answer for: ingesting a firehose of writes that never stops, scaling linearly to many nodes, staying available for writes even when a node or a whole data center goes down. That is what Cassandra was built for, and it still earns its place in 2026 for those jobs. This post runs it on Kubernetes the modern way, which has changed, because the operator I used for this in 2020 no longer exists.

This series rebuilds my 2020 Apress book, Advanced Platform Development with Kubernetes, for 2026. The approach behind it comes from building and running data platforms in production for more than twenty years.

§Postgres First, Cassandra When You Must

I want to be careful here, because the easy mistake is to reach for a “web scale” database because it sounds impressive, and then live with the costs for a workload that never needed it. Postgres is the default for a reason: it is flexible, transactional, and handles far more load than people assume. You reach past it only when a workload has a shape Postgres genuinely struggles with.

Cassandra has one such shape, and it is specific. It is a distributed, peer-to-peer database with no master node, where every node can take writes, data is replicated across nodes by a partition key, and adding nodes increases capacity close to linearly. That design gives it sustained write throughput, the lineage here is Netflix running over a million writes per second; linear horizontal scaling, you add nodes and get more; and high availability, because with no master there is no single point whose loss stops writes, and you tune how many replicas must acknowledge a write. The tradeoffs are real, you give up the rich querying, joins, and strong transactions of a relational database, and you model your data around the queries you will run rather than around normalized tables. So: Postgres for operational data, Cassandra for the write-heavy, always-on, scale-out workload that demands it.

§The Operator Graveyard

Running Cassandra on Kubernetes in 2020 meant choosing an operator, and the natural choice then was Rook’s, the same Rook project this platform uses for Ceph. That is what I ran when I documented this in 2020: a three-node ring serving as the platform’s always-on store and as one catalog in a distributed SQL warehouse, where a single query joined records in an object store against rows in the ring, from a notebook. It worked, and it depended on an operator that no longer exists.

Rook removed it. The project narrowed its focus to storage, deprecated its Cassandra and other non-storage operators, and the Cassandra work graduated out into dedicated projects. An operator you might have built on is gone, the project-death flavor of the lock-in problem this series keeps running into, this time inside the Kubernetes ecosystem itself. The lesson is not “choose better six years ago”; it is that a platform survives these deaths when the thing that dies is a manager of open components rather than the component itself. The database was Apache Cassandra all along, the data and the protocol were standard, and moving to a new operator is an operations task, not a migration. Six years on, the replacements surpass what died, with repair and backup automation the 2020 operator never had. Even the disasters of this ecosystem trend toward easier.

There are two replacements, matching two databases. For Cassandra itself, K8ssandra and its cass-operator are the maintained, community-backed path, with repair, backup, and monitoring tooling built around them. For more throughput from the same data model, the Scylla Operator runs Scylla, a C++ reimplementation of Cassandra that speaks the same CQL protocol but pushes more operations per node with lower tail latency. You pick the database; the operator follows.

§Install the K8ssandra Operator

K8ssandra ships as a Helm chart, another of the few installs in this series where Helm is the project’s own first-class path rather than a wrapper. It depends on cert-manager for its webhook certificates, which this platform has run since the cluster build.

helm repo add k8ssandra https://helm.k8ssandra.io/stable
helm install k8ssandra-operator k8ssandra/k8ssandra-operator \
  -n k8ssandra-operator --create-namespace

kubectl -n k8ssandra-operator rollout status deploy/k8ssandra-operator

The operator watches for K8ssandraCluster resources across namespaces and manages everything below them: the StatefulSets, the seed services, the auth secrets, and the companion tools.

§Declare the Ring

A cluster is one resource, and its important decisions are topology and storage:

apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: platform-cassandra
  namespace: data
spec:
  cassandra:
    serverVersion: "5.0.2"        # a current Cassandra release
    datacenters:
      - metadata:
          name: dc1
        size: 3
        racks:
          - name: r1
          - name: r2
          - name: r3
        storageConfig:
          cassandraDataVolumeClaimSpec:
            storageClassName: rook-ceph-block
            accessModes: [ReadWriteOnce]
            resources:
              requests:
                storage: 100Gi
        resources:
          requests:
            cpu: "2"
            memory: 4Gi
          limits:
            memory: 4Gi

serverVersion selects genuine Apache Cassandra, the operator running the open database rather than a fork of it. size: 3 with three racks is the resilience decision: the operator spreads one node per rack, racks map to failure domains, and Cassandra’s replica placement then guarantees that the three copies of any partition live in three different racks, so no single node or rack loss removes all replicas. This is the same failure-domain thinking as Ceph’s failureDomain: host, applied one layer up. The storageConfig puts each node’s data on its own Ceph volume; for the highest write ceilings, local NVMe with a local-path storage class beats network storage, a trade of operational convenience for latency that heavy rings eventually make. The resources block pins memory to the same value at request and limit, because Cassandra is a JVM service that sizes its heap from its environment, and a pod that can be throttled into memory pressure mid-compaction is a pod that gets killed at the worst time.

Apply it and watch the ring form, rack by rack:

kubectl apply -f platform-cassandra.yaml
kubectl -n data get pods -l cassandra.datastax.com/cluster=platform-cassandra

NAME                                 READY   STATUS    RESTARTS   AGE
platform-cassandra-dc1-r1-sts-0      2/2     Running   0          6m
platform-cassandra-dc1-r2-sts-0      2/2     Running   0          4m
platform-cassandra-dc1-r3-sts-0      2/2     Running   0          2m

The proof the ring actually formed is Cassandra’s own view of itself:

kubectl -n data exec -it platform-cassandra-dc1-r1-sts-0 -c cassandra -- nodetool status

Datacenter: dc1
===============
Status=Up/Down |/ State=Normal/Leaving/Joining/Moving
--  Address       Load        Tokens  Owns    Rack
UN  10.42.1.15    112.3 KiB   16      100.0%  r1
UN  10.42.2.31    108.9 KiB   16      100.0%  r2
UN  10.42.3.22    110.1 KiB   16      100.0%  r3

Three nodes, all UN, up and normal, one per rack. That two-letter column is the first thing to read in any Cassandra incident for the next decade of this ring’s life.

§Keyspace, Table, and the Write Path

The operator generated a superuser credential as a Secret; that is the way in:

CASS_USER=$(kubectl -n data get secret platform-cassandra-superuser -o jsonpath='{.data.username}' | base64 -d)
CASS_PASS=$(kubectl -n data get secret platform-cassandra-superuser -o jsonpath='{.data.password}' | base64 -d)

kubectl -n data exec -it platform-cassandra-dc1-r1-sts-0 -c cassandra -- \
  cqlsh -u "$CASS_USER" -p "$CASS_PASS"

Cassandra rewards designing around your queries, which is the mental shift from relational modeling. You do not normalize and join; you decide what you will query and lay the data out so that query is a single fast lookup by partition key. The platform already has a firehose that fits: the sensor stream arriving through Kafka, readings that never stop and are always queried per device over recent time.

CREATE KEYSPACE sensors
  WITH replication = {'class': 'NetworkTopologyStrategy', 'dc1': 3};

CREATE TABLE sensors.readings (
  device_id text,
  ts        timestamp,
  temp      double,
  load      double,
  PRIMARY KEY (device_id, ts)
) WITH CLUSTERING ORDER BY (ts DESC);

The keyspace’s replication map is where the rack topology pays off: NetworkTopologyStrategy with 'dc1': 3 says three replicas in this datacenter, placed rack-aware, and it is the strategy to use even with one datacenter because it is the one that understands topology when you later add another. In the table, PRIMARY KEY (device_id, ts) is doing two different jobs: device_id is the partition key, deciding which nodes own a device’s data, and ts is the clustering column, sorting rows within the partition. CLUSTERING ORDER BY (ts DESC) stores newest-first, so “the last hour for device X” reads from the front of one partition on one replica set, which is the query this table exists to serve and the reason it is shaped this way.

CONSISTENCY LOCAL_QUORUM;
INSERT INTO sensors.readings (device_id, ts, temp, load)
  VALUES ('lab-d1', toTimestamp(now()), 46.8, 0.11);
SELECT * FROM sensors.readings WHERE device_id = 'lab-d1' LIMIT 3;

The CONSISTENCY line is the control a single-primary database does not offer. LOCAL_QUORUM means two of the three replicas must acknowledge, so any subsequent quorum read overlaps at least one replica that has the write: read-your-writes, while still tolerating a node down. Drop to ONE for maximum-throughput telemetry where a lost reading is noise; rise to ALL when you must not proceed without full durability and accept that one dead node blocks you. That dial, per query, is the operational meaning of “tunable consistency.”

§Joined Back Into the Warehouse

The cross-source join comes forward to the modern stack by giving Trino a Cassandra catalog:

# cassandra.properties, in the Trino catalog config
connector.name=cassandra
cassandra.contact-points=platform-cassandra-dc1-service.data
cassandra.load-policy.dc-aware.local-dc=dc1

contact-points aims at the datacenter service the operator maintains, and the dc-aware load policy keeps Trino’s requests local to dc1 rather than wandering to any future datacenter. With the catalog in place, one SQL statement joins the ring’s live readings against the lakehouse’s history, from a notebook. That is the same join I demonstrated in 2020 through Presto and the Rook operator, running today on none of the same software. The pieces died; the pattern did not.

§Or Scylla, the C++ Drop-In

If the workload needs more per node than Cassandra delivers, Scylla runs the same data model and the same CQL with a shard-per-core architecture that extracts more from each machine. The Scylla Operator installs from its own manifests, and a cluster is a ScyllaCluster:

apiVersion: scylla.scylladb.com/v1
kind: ScyllaCluster
metadata:
  name: platform-scylla
  namespace: data
spec:
  datacenter:
    name: dc1
    racks:
      - name: r1
        members: 3
        storage:
          capacity: 100Gi
          storageClassName: rook-ceph-block

The choice between them is throughput against ecosystem. Cassandra with K8ssandra has the broader tooling and the Apache governance; Scylla wrings more out of the hardware, and a serious Scylla deployment leans into that, with dedicated cores, NVMe, and its performance-tuning DaemonSet, machinery that only pays off when you actually need the ceiling. Both are CQL, so you can prototype on one and move to the other without rewriting your application, which is the same anti-lock-in property the whole platform selects for, applied between two open databases.

§Operating Cassandra

Repair, automatically. A distributed database where replicas can drift needs periodic repair to stay consistent, and doing it by hand is error-prone. K8ssandra runs Reaper, which schedules and manages repairs across the ring, so the maintenance that Cassandra operators most often neglect happens on its own.

Scale by adding nodes. Growing capacity is raising the datacenter size; the operator adds nodes and the ring rebalances ownership onto them. This is the linear scaling Cassandra is built for, and the operator handles the token management that made it fiddly by hand.

Replicate across data centers. Cassandra’s multi-datacenter replication keeps a live copy in another location, for disaster recovery or for serving reads near users, and the operator models it as another datacenter in the same K8ssandraCluster, with the keyspace’s replication map extended to match. A capability that is genuinely hard to build, native to the database, declared in YAML.

Back up to your object store. K8ssandra’s Medusa backs the cluster up to S3-compatible storage, which is the SeaweedFS object store you already run, so Cassandra’s backups land alongside Postgres’s and OpenSearch’s on infrastructure you own. And the operator exposes Prometheus metrics, so the ring’s health sits on the same dashboards as the rest of the platform.

§When Something Is Wrong

Nodes stay Pending. Storage. The data volumes cannot bind, pointing back at Rook; Cassandra nodes are heavy on disk and will not start without it. kubectl -n data get pvc shows the claims and their state.

A node shows DN in nodetool status. Down. The ring keeps serving as long as replica math allows, which is the availability you paid for; find out why the node died before the second one does. The pod’s events and the container log say whether it is the node, the volume, or the process.

Reads miss recent writes, or data seems inconsistent. Consistency level. A write at ONE and a read at ONE can miss each other by design; use LOCAL_QUORUM on both sides for read-your-writes, or run repair if replicas have drifted beyond what hinted handoff healed.

Queries slow down over time on a table with deletes. Tombstones. Cassandra marks deletes with tombstones that accumulate and slow reads until compaction clears them; a data model with heavy deletes is usually a model that should be redesigned around the write path, because Cassandra is built for data that arrives and expires, not data that churns.

A node is killed by the OOM killer. JVM heap against the pod memory limit, the same tuning concern as the other JVM services. The request-equals-limit pattern from the cluster manifest exists for this; if you changed it, change it back.

§The Data Model Is the Hard Part, and Agents Help

Running Cassandra stopped being the hard part somewhere in the last six years; the operator runs it better than most humans did. The hard part that remains is the data model, because a wrong partition key is a design error that performs fine in development and falls over at scale, hot partitions, unbounded rows, tombstone graveyards. This is where an AI agent is genuinely useful in a way that fits the leash: give it the queries the application will run, this post, and the schema conventions above, and have it propose the table designs, then review the partition keys the way you would review any consequential design. The feedback loop is unusually good because the failure modes are legible: nodetool status is text, the per-table statistics are text, and a hot partition shows up on the dashboards the operator already exports. I also point the platform’s MCP layer at the ring through Trino, which means an agent investigating “why is device lab-d1 noisy” queries the same catalog a human does, with the same governed access, and never needs credentials to the ring itself.

§What You Have

A distributed, write-heavy, always-available database on Kubernetes, run by a maintained operator after the old one was retired, spread across racks so no single failure removes a replica set, repaired and backed up automatically, joined back into the warehouse through Trino, on storage you own. Cassandra for the Cassandra-shaped workloads, Scylla when you need more per node, and Postgres still the default for everything else. The platform now has a database for the firehose alongside its system of record.

Next I add the last piece of the compute layer, self-hosted serverless functions with Knative, for the event-driven transforms that do not warrant a full service.