Professional notes by Craig Johnston
long-form, short-form, working drafts · since 2008
long-form, short-form, working drafts · since 2008
VOL. XIX · MMXXVI
81 NOTES IN PRINT
81 NOTES IN PRINT
Plates
Every note's diagram, all on one page. Atlas-style contact sheet of the imti.co plate library. Click any plate to open the note.
81 plates
flowchart LR
A1[Familiar tool bias] -.- M1
A2[SQL is the default hammer] -.- M1
A3[Routing hints feel optional] -.- M1
M1[Mode A: Path of Least Resistance] --> P
B1[Reads the tool description] -.- M2
B2[Repeats it back accurately] -.- M2
B3[Acts on the training-data prior anyway] -.- M2
M2[Mode B: Comprehension is not Compliance] --> P
P((BAD AGENT<br/>OUTPUT))
The Two Failure Modes That Break Your AI Data Agent
flowchart TB
CRON{{"CronJob · every 15 min"}}
CRON --> WIN
PG[("PostgreSQL 16<br/>240 GB · 300M rows")]
PG -.->|"JDBC date window"| WIN
subgraph PIPE[" PySpark · local mode "]
direction TB
WIN["Date-windowed read<br/><i>WHERE event_at BETWEEN ?</i>"]
KEY["SHA-2 deterministic _id<br/><i>idempotent on retry</i>"]
BULK["OpenSearch bulk action<br/><i>refresh=False</i>"]
WIN --> KEY --> BULK
end
BULK -->|"_id-keyed upsert"| OS[("OpenSearch 2.13<br/>indexed docs")]
PostgreSQL to OpenSearch with PySpark on Kubernetes
gitGraph
commit id: "init"
commit id: "core code"
branch release/sign
commit id: "goreleaser.yml"
commit id: "cosign keyless"
commit id: "syft sbom"
checkout main
merge release/sign
commit id: "v1.0.0" tag: "v1.0.0"
commit id: "fix darwin arm64"
commit id: "v1.0.1" tag: "v1.0.1"
GoReleaser with Cosign Signing and Syft SBOM
mindmap
root((AI on a Leash<br/>for Go))
Linting
golangci-lint
43 enabled linters
revive overrides
Testing
table-driven
race detector
mutation: gremlins
Coverage
codecov.yml
per-package thresholds
Security
gosec
govulncheck
Release
GoReleaser
cosign + SBOM
Convention
GNUmakefile
CLAUDE.md
go-leash skill
AI on a Leash: Complete Go Project Configuration
quadrantChart
title Where AI Coding Approaches Sit
x-axis Low AI Autonomy --> High AI Autonomy
y-axis Low Verification Rigor --> High Verification Rigor
quadrant-1 AI on a Leash
quadrant-2 Hand-Crafted Rigor
quadrant-3 Cowboy Codes
quadrant-4 Vibe Coding
Karpathy Vibe Coding: [0.85, 0.10]
Ralph Bash Loop: [0.78, 0.22]
Senior Solo Dev: [0.12, 0.55]
Pair Programming + AI: [0.40, 0.65]
AI on a Leash: [0.82, 0.88]
Ralph's Uncle
flowchart TB
AI(["AI proposes Go code"])
AI --> COMP{{"go build"}}
COMP -->|type error| F1[/"specific error message<br/><i>cannot use x (type T) as Y</i>"/]
F1 -.->|feedback| AI
COMP -->|pass| LINT{{"golangci-lint · 43 linters"}}
LINT -->|reject| F2[/"lint rule violated<br/><i>ineffassign, errcheck, gocritic...</i>"/]
F2 -.->|feedback| AI
LINT -->|pass| TEST{{"go test -race"}}
TEST -->|fail| F3[/"failure trace<br/><i>plus race conditions</i>"/]
F3 -.->|feedback| AI
TEST -->|pass| OUT(["idiomatic, type-safe, race-free code"])
Go's Constraints and Idioms Make AI Coding Better
stateDiagram-v2
[*] --> Skeptical: hears about CVEs
Skeptical --> Curious: context IS the hard problem
Curious --> Building: let me try one server
Building --> Stuck: hits a known flaw
Stuck --> Building: workaround / pin / fork
Building --> Production: metadata investment pays off
Production --> Hardened: real-world feedback loop
Hardened --> [*]: metadata survives the protocol
note right of Stuck
OAuth complexity
Scaling limits
Spec ambiguity
end note
note right of Hardened
Even if MCP dies,
your documented
data remains.
end note
MCP Is Flawed. Build With It Anyway.
erDiagram
BUCKET ||--o{ OBJECT : contains
OBJECT ||--o{ VERSION : "has versions"
OBJECT ||--o{ TAG : "tagged with"
OBJECT ||--o| ACL : "guarded by"
OBJECT ||--|| METADATA : describes
MCP_TOOL }o--|| OBJECT : reads
MCP_TOOL ||--o{ PRESIGNED_URL : issues
AI_AGENT ||--o{ MCP_TOOL : invokes
BUCKET {
string name PK
string region
timestamp created_at
}
OBJECT {
string key PK
bigint size_bytes
string content_type
string etag
}
VERSION {
string version_id PK
timestamp created_at
boolean is_latest
}
TAG {
string key PK
string value
}
ACL {
string canned_policy
string owner_id
}
METADATA {
string user_meta
string content_disposition
}
PRESIGNED_URL {
string url
timestamp expires_at
string scope
}
MCP_TOOL {
string name
string permission
}
AI_AGENT {
string client_id
string transport
}
AI Data Lake Access with MCP and S3
sequenceDiagram
autonumber
actor U as Analyst
participant A as Claude / AI
participant M as mcp-trino
participant T as Trino
participant PG as PostgreSQL
participant S3 as S3 / Iceberg
participant K as Kafka
U->>A: Last quarter orders by region?
A->>M: list_schemas()
M->>T: SHOW SCHEMAS
T-->>M: catalogs and tables
M-->>A: schema graph
A->>M: execute_query(SQL)
M->>T: parse + federated plan
T->>PG: SELECT customers
T->>S3: scan parquet partitions
T->>K: scan recent stream
PG-->>T: rows
S3-->>T: rows
K-->>T: rows
T-->>M: federated result set
M-->>A: rows + metadata
A-->>U: answer with chart
AI Data Warehouse Access with MCP and Trino
flowchart TB
APP(["Your App"])
APP -->|"postgres:5432"| HOSTS
HOSTS["/etc/hosts entry"]
HOSTS -->|"127.1.27.1:5432"| KFWD
KFWD{{"kubefwd listener"}}
KFWD -->|"port-forward channel"| KAPI
KAPI["kubectl port-forward"]
KAPI -->|"TCP to pod IP"| POD
POD[("postgres pod in cluster")]
kubefwd: Forward Kubernetes Services to Localhost by Name
stateDiagram-v2
[*] --> Idle: kubefwd --tui
Idle --> Forwarding: add namespace<br/>or API call
Forwarding --> Connected: hosts updated<br/>listener bound
Connected --> Dropped: pod restart<br/>or VPN drop
Dropped --> Reconnecting: auto-reconnect
Reconnecting --> Connected: success
Reconnecting --> Failed: max retries
Failed --> Reconnecting: manual retry
Connected --> Idle: remove forward
Idle --> [*]: exit
note right of Reconnecting
exponential backoff
with jitter
end note
kubefwd in 2026: Interactive TUI and Auto-Reconnect
flowchart TB
GOAL(["Developer needs cluster services locally"])
GOAL --> OLD
GOAL --> NEW
subgraph OLD[" The old way "]
direction TB
O1["open the right cluster context"]
O2["kubectl port-forward x N services"]
O3["track local ports manually"]
O4["restart everything when pods cycle"]
O1 --> O2 --> O3 --> O4
end
subgraph NEW[" With the kubefwd MCP "]
direction TB
N1["ask AI: wire up namespace foo"]
N2["AI lists pods and services"]
N3["AI forwards everything by name"]
N4["AI auto-reconnects on pod cycle"]
N1 --> N2 --> N3 --> N4
end
OLD -.->|"painful"| FAIL[("time wasted")]
NEW -.->|"seamless"| WIN[("code against<br/>redis:6379 directly")]
AI-Assisted Kubernetes Development with kubefwd
sequenceDiagram
autonumber
actor C as External Service
participant K as Ingress / LB
participant P as portpxy
participant L as HandleHttpRequest<br/>(NiFi :10001)
participant F as NiFi Flow
C->>K: POST /webhook/orders
K->>P: forwarded
P->>P: route lookup<br/>by path or host
P->>L: proxy to :10001
L->>F: FlowFile created
F->>F: route, transform, publish
F->>L: HandleHttpResponse
L-->>P: 200 OK
P-->>K: 200 OK
K-->>C: 200 OK
Note over P: dynamic listeners,<br/>no manifest changes
Apache NiFi: Dynamic HTTP Listeners with portpxy
mindmap
root((Advanced JOLT))
Cardinality
ONE - force single
MANY - force array
MANY for nested values
Modify
=concat
=toLower / =toUpper
=sum / =divide
=literalString
Wildcards
"*" any field
"&" reference back
"$" copy a key as value
Chained Specs
shift then default
cardinality then modify
multi-pass cleanup
Apache NiFi: JOLT Transformations Part 2
flowchart TB
IN[/"Input JSON"/]
IN -->|"1"| S["shift · move and rename"]
S -->|"2"| D["default · fill missing"]
D -->|"3"| R["remove · drop fields"]
R -->|"4"| SO["sort · alphabetize"]
SO -->|"5"| C["cardinality · array vs scalar"]
C -->|"6"| M["modify · transform values"]
M --> OUT[/"Output JSON"/]
Apache NiFi: JOLT Transformations Part 1
erDiagram
AUTH_PROVIDER ||--o{ USER : authenticates
USER ||--o{ MEMBERSHIP : "belongs to"
GROUP ||--o{ MEMBERSHIP : contains
GROUP ||--o{ POLICY : granted
USER ||--o{ POLICY : "directly granted"
POLICY }o--|| RESOURCE : "applies to"
POLICY ||--|| ACTION : permits
CERTIFICATE ||--|| USER : "identifies via DN"
AUTH_PROVIDER {
string type
string config
}
USER {
string identity PK
string distinguished_name
timestamp last_login
}
GROUP {
string name PK
string identity_provider
}
MEMBERSHIP {
string user_id FK
string group_id FK
}
POLICY {
string id PK
string action
string resource_path
}
RESOURCE {
string path PK
string type
}
ACTION {
string verb PK
}
CERTIFICATE {
string subject_dn PK
string issuer
timestamp not_after
}
Apache NiFi: Securing Your Data Flows
flowchart TB
USER([Internet User])
USER --> ING
subgraph EDGE[ Edge ]
ING["Ingress Controller<br/>+ TLS"]
end
ING --> SVC
subgraph NS[ NiFi Namespace ]
SVC["nifi-service<br/>headless"]
subgraph SS[ NiFi StatefulSet ]
direction LR
N0["nifi-0"]
N1["nifi-1"]
N2["nifi-2"]
end
subgraph ZK[ ZooKeeper Ensemble ]
direction LR
Z0["zk-0"]
Z1["zk-1"]
Z2["zk-2"]
end
SVC --> N0
SVC --> N1
SVC --> N2
N0 -.->|coordination| Z0
N1 -.->|coordination| Z1
N2 -.->|coordination| Z2
end
subgraph STORAGE[ Persistent Volumes ]
direction LR
PV1[("flowfile_repo")]
PV2[("content_repo")]
PV3[("provenance_repo")]
end
N0 --> PV1
N0 --> PV2
N0 --> PV3
Apache NiFi: Production Kubernetes Deployment
quadrantChart
title Linear Algebra in Go HPC Approaches
x-axis Pure Go --> Native acceleration
y-axis Low throughput --> High throughput
quadrant-1 Optimized and parallel
quadrant-2 Convenient and slow
quadrant-3 Naive loops
quadrant-4 Single-threaded native
Plain nested for-loop: [0.10, 0.10]
goroutines over slices: [0.22, 0.42]
gonum default pure Go: [0.32, 0.48]
gonum cgo OpenBLAS: [0.78, 0.82]
SIMD via x sys cpu: [0.62, 0.68]
LAPACK direct: [0.86, 0.88]
GPU offload via cgo: [0.95, 0.95]
Linear Algebra in Go: High-Performance Computing
flowchart TB
X[/"input x"/] --> M1
W1[("W1")] --> M1
M1["matmul x · W1"] --> A1{{"ReLU"}}
A1 --> H[/"hidden h"/]
H --> M2
W2[("W2")] --> M2
M2["matmul h · W2"] --> A2{{"softmax"}}
A2 --> O[/"output y_hat"/]
O --> L["loss L"]
L -.->|"∂L/∂W2"| W2
L -.->|"∂L/∂W1 chain rule"| W1
Linear Algebra in Go: Neural Network Foundations
sankey-beta
Feature 1,Total Variance,12
Feature 2,Total Variance,18
Feature 3,Total Variance,9
Feature 4,Total Variance,15
Feature 5,Total Variance,11
Feature 6,Total Variance,8
Feature 7,Total Variance,14
Feature 8,Total Variance,13
Total Variance,PC1 - 52%,52
Total Variance,PC2 - 24%,24
Total Variance,PC3 - 12%,12
Total Variance,PC4 to PC8 - 12%,12
Linear Algebra in Go: PCA Implementation
flowchart TB
D[("Training data X, y")]
D --> AUG["augment X with bias column"]
AUG --> N["normal equations: X^T X β = X^T y"]
N --> R["ridge: add λ I to X^T X"]
R --> SOLVE["solve via mat.Solve / Cholesky"]
SOLVE --> CV["k-fold cross-validation"]
CV --> COEF[/"trained coefficients β"/]
Linear Algebra in Go: Building a Regression Library
mindmap
root((Statistics in Go))
Descriptive
mean
median
variance
stddev
Distributions
Normal
Uniform
Exponential
Poisson
Multivariate
covariance matrix
correlation matrix
Mahalanobis distance
Hypothesis tests
t-test
chi-squared
Kolmogorov-Smirnov
Linear Algebra in Go: Statistics and Data Analysis
flowchart TB
A[("Matrix A (m by n)")]
A --> ATA["form A^T A"]
ATA --> EIG["eigendecompose"]
EIG --> V["V = right singular vectors"]
EIG --> SIG["Σ = sqrt(eigenvalues)"]
V --> COMBINE["U_i = (A · v_i) / σ_i"]
SIG --> COMBINE
COMBINE --> RESULT[/"A = U · Σ · V^T"/]
RESULT --> APPS["applications"]
APPS --> P["pseudoinverse"]
APPS --> L["low-rank approximation"]
APPS --> R["dimensionality reduction"]
Linear Algebra in Go: SVD and Decompositions
stateDiagram-v2
[*] --> Init: random unit vector v_0
Init --> Multiply: w = A · v_k
Multiply --> Normalize: v_{k+1} = w / ||w||
Normalize --> CheckConverge: compare to v_k
CheckConverge --> Multiply: not converged
CheckConverge --> Compute: converged
Compute --> Output: λ = v^T A v
Output --> [*]
note right of Multiply
power iteration
converges to the
dominant eigenvalue
end note
Linear Algebra in Go: Eigenvalue Problems
quadrantChart
title Linear System Solvers
x-axis Iterative --> Direct
y-axis Sparse matrices --> Dense matrices
quadrant-1 Direct dense
quadrant-2 Iterative dense
quadrant-3 Iterative sparse
quadrant-4 Direct sparse
Gaussian elimination: [0.85, 0.85]
LU decomposition: [0.80, 0.80]
Cholesky pos-def: [0.75, 0.88]
Conjugate Gradient: [0.20, 0.30]
GMRES: [0.15, 0.25]
Jacobi: [0.10, 0.50]
Gauss-Seidel: [0.18, 0.55]
Sparse LU: [0.85, 0.20]
Linear Algebra in Go: Solving Linear Systems
erDiagram
MAT_INTERFACE ||--o{ DENSE : implements
MAT_INTERFACE ||--o{ VECDENSE : implements
MAT_INTERFACE ||--o{ SYMDENSE : implements
MAT_INTERFACE ||--o{ TRIDENSE : implements
MAT_INTERFACE ||--o{ DIAGDENSE : implements
DENSE ||--|{ MATRIX_OP : supports
VECDENSE ||--|{ VECTOR_OP : supports
DENSE ||--|| TRANSPOSE : has
DENSE ||--o| INVERSE : has
MAT_INTERFACE {
method At
method Dims
method T
}
DENSE {
int rows
int cols
slice data
}
VECDENSE {
int n
slice data
}
SYMDENSE {
int n
bool symmetric
}
TRIDENSE {
bool upper_or_lower
}
DIAGDENSE {
slice diagonal
}
MATRIX_OP {
method Mul
method Add
method T
}
VECTOR_OP {
method Dot
method Norm
}
TRANSPOSE {
Dense matrix
}
INVERSE {
bool exists
}
Linear Algebra in Go: Matrix Fundamentals
mindmap
root((Vectors in Go))
Construction
mat.NewVecDense
from float64 slice
zero vector
unit vector
Arithmetic
add
subtract
scalar multiply
Inner products
dot product
magnitude
angle between
Spatial
cross product
projection
orthogonal basis
Linear Algebra
span
linear combination
basis vectors
Linear Algebra in Go: Vectors and Basic Operations
gitGraph
commit id: "Why Kubernetes"
branch concepts
commit id: "containers"
commit id: "k8s primitives"
commit id: "operators"
checkout main
merge concepts
branch data
commit id: "Cassandra"
commit id: "Elasticsearch"
commit id: "Kafka"
checkout main
merge data
branch iot
commit id: "MQTT"
commit id: "edge nodes"
checkout main
merge iot
branch ml
commit id: "TensorFlow"
commit id: "model serving"
checkout main
merge ml
branch chain
commit id: "Ethereum"
commit id: "smart contracts"
checkout main
merge chain
commit id: "Platform" tag: "v1.0"
Advanced Platform Development with Kubernetes
sankey-beta
Raw Data,Cleaning,100
Cleaning,Feature Engineering,90
Cleaning,Discarded outliers,10
Feature Engineering,Linear Models,28
Feature Engineering,Tree Models,30
Feature Engineering,Neural Nets,22
Feature Engineering,Clustering,10
Linear Models,Predictions,28
Tree Models,Predictions,30
Neural Nets,Predictions,22
Clustering,Insights,10
Linear Algebra: Practical Applications in ML
flowchart TB
D[("Centered data X")]
D --> PATH{"choose path"}
PATH -->|"covariance"| COV["C = X^T X / (n-1)"]
COV --> EIG["eigendecompose C"]
EIG --> SORT
PATH -->|"direct"| SVD["SVD of X"]
SVD --> SORT
SORT["sort by σ^2 / λ descending"]
SORT --> SELECT["select top-k components"]
SELECT --> PROJECT["project Z = X V_k"]
PROJECT --> OUT[("Reduced data Z")]
Linear Algebra: Principal Component Analysis
mindmap
root((SVD applications))
Decomposition
A = U Sigma V^T
orthonormal U and V
diagonal Sigma
Pseudoinverse
A^+ = V Sigma^+ U^T
Moore-Penrose
least squares solver
Low rank
truncate to top k
data compression
noise reduction
Dimensionality reduction
PCA via SVD
latent factors
Image processing
lossy compression
denoising
Recommender systems
matrix factorization
collaborative filtering
Linear Algebra: Singular Value Decomposition
flowchart TB
P[/"Overdetermined: Ax = b, m > n"/]
P --> CHOICE{"path"}
CHOICE -->|"normal equations"| N["A^T A x = A^T b"]
CHOICE -->|"QR factorization"| QR["A = Q R"]
N --> SOLVE
QR --> Q["form Q^T b"]
Q --> R["R x = Q^T b<br>back-substitute"]
R --> SOLVE
SOLVE["solve for x"]
SOLVE --> X[/"x_LS minimizes ||Ax - b||"/]
X --> RES[/"residual r = b - A x_LS"/]
Linear Algebra: Least Squares and Regression
stateDiagram-v2
[*] --> Start: vectors v_1 ... v_k
Start --> First: u_1 = v_1 / norm
First --> Next: i = 2
Next --> Subtract: u_i = v_i - sum proj of v_i onto u_1...u_{i-1}
Subtract --> Normalize: u_i = u_i / norm
Normalize --> Check: i < k?
Check --> Next: yes, i = i+1
Check --> Done: i == k
Done --> [*]: orthonormal basis u_1 ... u_k
note right of Subtract
remove components
already spanned
by u_1 ... u_{i-1}
end note
Linear Algebra: Orthogonality and Projections
mindmap
root((Eigenvalues part 2))
Diagonalization
A factored as P D P inverse
D holds the eigenvalues
P holds the eigenvectors
Matrix powers
compute A k via diagonalization
far cheaper than direct multiplication
Complex eigenvalues
rotation matrices
conjugate pairs
Symmetric matrices
real eigenvalues
orthogonal eigenvectors
Applications
Markov chains
stability analysis
vibration modes
Google PageRank
Linear Algebra: Eigenvalues and Eigenvectors Part 2
flowchart TB
A[("Square matrix A (n by n)")]
A --> CHAR["characteristic polynomial<br>det(A - lambda I) = 0"]
CHAR --> ROOTS["solve for roots"]
ROOTS --> EVALS["eigenvalues lambda_1 ... lambda_n"]
EVALS --> EACH{"for each lambda_i"}
EACH --> NULL["solve (A - lambda_i I) v = 0"]
NULL --> NSPACE["null space basis"]
NSPACE --> EVECS[/"eigenvectors v_i"/]
EVALS --> PAIRS[/"eigen pairs (lambda_i, v_i)"/]
EVECS --> PAIRS
Linear Algebra: Eigenvalues and Eigenvectors Part 1
erDiagram
VECTOR_SPACE ||--o{ SUBSPACE : contains
VECTOR_SPACE ||--|| BASIS : "spanned by"
BASIS ||--|{ VECTOR : "list of"
BASIS ||--|| DIMENSION : determines
SUBSPACE ||--|| SPAN : "equals span of vectors"
SUBSPACE ||--o| ORTHOGONAL_COMPLEMENT : has
LINEAR_COMBINATION }o--o{ VECTOR : "weights"
SUBSPACE ||--|| BASIS : "has its own"
VECTOR_SPACE {
field scalars
op addition
op scalar_mult
}
SUBSPACE {
bool closed_under_add
bool closed_under_scalar
bool contains_zero
}
BASIS {
bool linearly_independent
bool spans
}
VECTOR {
slice components
}
DIMENSION {
int n
}
SPAN {
bool generates_subspace
}
ORTHOGONAL_COMPLEMENT {
int dim
}
LINEAR_COMBINATION {
slice coefficients
}
Linear Algebra: Vector Spaces and Subspaces
flowchart TB
A[("Matrix A (n by n)")]
A --> DET["compute det(A)"]
DET --> CHECK{"det = 0?"}
CHECK -->|"yes"| SING[/"Singular: no inverse"/]
CHECK -->|"no"| METHOD{"choose method"}
METHOD -->|"adjugate"| ADJ["A^-1 = adj(A) / det(A)"]
METHOD -->|"Gauss-Jordan"| GJ["row reduce [A | I] to [I | A^-1]"]
METHOD -->|"LU"| LU["solve LU x_i = e_i for each column"]
ADJ --> INV[/"inverse A^-1"/]
GJ --> INV
LU --> INV
Linear Algebra: Matrix Inverses and Determinants
sequenceDiagram
autonumber
participant Aug as Augmented [A | b]
participant Up as Upper triangular form
participant X as Solution x
Note over Aug: forward elimination
Aug->>Aug: pivot at (1,1) clear column 1 below
Aug->>Aug: pivot at (2,2) clear column 2 below
Aug->>Up: continue to (n,n)
Note over Up: now upper triangular
Up->>X: back-substitute from bottom row
X-->>Aug: solution vector x
Linear Algebra: Systems of Linear Equations
erDiagram
NDARRAY ||--o{ ELEMENT : contains
NDARRAY ||--|| SHAPE : has
NDARRAY ||--|| DTYPE : has
NDARRAY ||--|{ MATRIX_OP : supports
NDARRAY ||--|| TRANSPOSE : has_T
NDARRAY ||--o| INVERSE : has
MATRIX_OP }o--o{ NDARRAY : produces
NDARRAY {
slice data
int ndim
int size
}
ELEMENT {
float64 value
int row
int col
}
SHAPE {
tuple dims
}
DTYPE {
string name
int bytes
}
MATRIX_OP {
method matmul
method add
method elementwise
}
TRANSPOSE {
view of_array
}
INVERSE {
bool exists
}
Linear Algebra: Matrices
flowchart TB
V[/"Vector v as numpy array"/]
V --> ARITH{"operation"}
ARITH -->|"add"| ADD["v + w"]
ARITH -->|"scale"| SCALE["c * v"]
ARITH -->|"dot"| DOT["v · w = sum v_i w_i"]
ARITH -->|"cross"| CROSS["v x w in R^3"]
ARITH -->|"magnitude"| MAG["||v|| = sqrt(v · v)"]
ARITH -->|"projection"| PROJ["proj_w v = (v · w / w · w) w"]
DOT --> ANGLE["cos(theta) = v · w / (||v|| ||w||)"]
MAG --> UNIT["unit vector v / ||v||"]
Linear Algebra: Vectors
flowchart TB
EXT([Producers and Consumers])
EXT --> SVC
subgraph CLUSTER[ Kafka Namespace ]
SVC[broker Service - headless]
subgraph SS[ Kafka StatefulSet - 3 brokers ]
direction LR
B0[broker-0]
B1[broker-1]
B2[broker-2]
end
subgraph ZK[ ZooKeeper Ensemble ]
direction LR
Z0[zk-0]
Z1[zk-1]
Z2[zk-2]
end
SVC --> B0
SVC --> B1
SVC --> B2
B0 -.->|coordination| Z0
B1 -.->|coordination| Z1
B2 -.->|coordination| Z2
end
PV0[(broker-0 PVC)]
PV1[(broker-1 PVC)]
PV2[(broker-2 PVC)]
B0 --> PV0
B1 --> PV1
B2 --> PV2
Kafka on Kubernetes
sequenceDiagram
autonumber
participant Geth as Geth node (private chain)
participant Stats as ethstats backend
participant DB as in-memory store
participant UI as ethstats dashboard
actor Op as Operator
Geth->>Stats: register identity (signed)
Stats->>DB: add node entry
loop every block or interval
Geth->>Stats: push metrics (hashrate, peers, gas, latency)
Stats->>DB: update metrics
end
Op->>UI: open dashboard
UI->>Stats: subscribe (websocket)
Stats-->>UI: stream node states
UI-->>Op: hashrate, peers, gas, latency
Ethereum Ethstats
erDiagram
NODE ||--o{ PEER : connects_to
NODE ||--|| KEYSTORE : has
NODE ||--o{ ACCOUNT : owns
ACCOUNT ||--o{ TRANSACTION : signs
BLOCK ||--o{ TRANSACTION : includes
GENESIS ||--|| CHAIN_CONFIG : defines
NODE ||--|| GENESIS : initialized_with
NODE {
string nodekey
string enode_url
string role
}
PEER {
string remote_enode
}
KEYSTORE {
string path
}
ACCOUNT {
string address PK
bigint balance
}
BLOCK {
bytes hash PK
bytes parent_hash
int number
timestamp time
}
TRANSACTION {
bytes hash PK
string from_addr
string to_addr
bigint value
}
GENESIS {
json config
int chain_id
}
CHAIN_CONFIG {
string consensus
bigint difficulty
}
Ethereum Blockchain on Kubernetes
gitGraph
commit id: "Genesis"
commit id: "Block 1"
commit id: "Block 2"
branch fork
commit id: "Block 3a"
checkout main
commit id: "Block 3"
commit id: "Block 4"
commit id: "Block 5" tag: "longest chain wins"
commit id: "Block 6"
commit id: "Block 7"
commit id: "Block 8" tag: "finalized"
Blockchain
sequenceDiagram
autonumber
actor Dev
participant K as kubefwd
participant API as Kubernetes API
participant Hosts as /etc/hosts
Dev->>K: sudo kubefwd svc -n myns
K->>API: list services in namespace
API-->>K: services es, db, redis, ...
loop for each service
K->>Hosts: add 127.1.27.x entry for service name
K->>API: open port-forward to pod
end
K-->>Dev: ready
Note over Dev: app uses elasticsearch:9200 normally
Dev->>Hosts: resolve elasticsearch
Hosts-->>Dev: 127.1.27.5
Dev->>K: TCP to 127.1.27.5:9200
K->>API: tunnel to ES pod
Kubernetes Port Forwarding for Local Development
flowchart LR
C(("FOLIO XXXVI"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Elasticsearch"]
C --- T0
T1["Python"]
C --- T1
T2["Kubernetes"]
C --- T2
T3["Serverless"]
C --- T3
classDef ser fill:transparent,stroke-dasharray:3 3
FaaS on Kubernetes
flowchart LR
C(("FOLIO XXXV"))
S["SERIES · Elasticsearch"]:::ser
C --- S
T0["Elasticsearch"]
C --- T0
T1["Data"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Elasticsearch Essential Queries
flowchart LR
C(("FOLIO XXXIV"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Elasticsearch"]
C --- T1
T2["Data"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Remote Query Elasticsearch on Kubernetes
flowchart LR
C(("FOLIO XXXIII"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Elasticsearch"]
C --- T1
T2["Data"]
C --- T2
T3["JSON"]
C --- T3
classDef ser fill:transparent,stroke-dasharray:3 3
High Traffic JSON Data into Elasticsearch on Kubernetes
flowchart LR
C(("FOLIO XXXII"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Elasticsearch"]
C --- T1
T2["Data"]
C --- T2
T3["Kibana"]
C --- T3
classDef ser fill:transparent,stroke-dasharray:3 3
Kibana on Kubernetes
flowchart LR
C(("FOLIO XXXI"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Elasticsearch"]
C --- T1
T2["Data"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Production Grade Elasticsearch on Kubernetes
flowchart LR
C(("FOLIO XXX"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["DevOps"]
C --- T1
T2["Security"]
C --- T2
T3["kubectl"]
C --- T3
classDef ser fill:transparent,stroke-dasharray:3 3
Kubernetes Team Access - RBAC for developers and QA
flowchart LR
C(("FOLIO XXIX"))
S["SERIES · Data Science"]:::ser
C --- S
T0["Python"]
C --- T0
T1["Data"]
C --- T1
T2["Data Science"]
C --- T2
T3["Data Visualization"]
C --- T3
classDef ser fill:transparent,stroke-dasharray:3 3
Python Data Essentials - Matplotlib and Seaborn
flowchart LR
C(("FOLIO XXVIII"))
T0["Microservice"]
C --- T0
T1["PDF"]
C --- T1
T2["Data"]
C --- T2
T3["Docker"]
C --- T3
T4["Kubernetes"]
C --- T4
classDef ser fill:transparent,stroke-dasharray:3 3
Webpage to PDF Microservice
flowchart LR
C(("FOLIO XXVII"))
T0["Microservices"]
C --- T0
T1["DevOps"]
C --- T1
T2["Golang"]
C --- T2
T3["Gitlab"]
C --- T3
classDef ser fill:transparent,stroke-dasharray:3 3
A Microservices Workflow with Golang and Gitlab CI
flowchart LR
C(("FOLIO XXVI"))
S["SERIES · Data Science"]:::ser
C --- S
T0["Python"]
C --- T0
T1["Data"]
C --- T1
T2["Data Science"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Python Data Essentials - Pandas
flowchart LR
C(("FOLIO XXV"))
S["SERIES · Data Science"]:::ser
C --- S
T0["Python"]
C --- T0
T1["Data"]
C --- T1
T2["Data Science"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Python Data Essentials - Numpy
flowchart LR
C(("FOLIO XXIV"))
T0["DevOps"]
C --- T0
T1["Security"]
C --- T1
T2["Golang"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Reverse Proxy in Golang
flowchart LR
C(("FOLIO XXIII"))
S["SERIES · Golang"]:::ser
C --- S
T0["Golang"]
C --- T0
T1["Jupyter Notebooks"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Golang to Jupyter
flowchart LR
C(("FOLIO XXII"))
S["SERIES · Python"]:::ser
C --- S
T0["Python"]
C --- T0
classDef ser fill:transparent,stroke-dasharray:3 3
Essential Python 3
flowchart LR
C(("FOLIO XXI"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Security"]
C --- T1
T2["Ingress"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
CORS on Kubernetes Ingress Nginx
flowchart LR
C(("FOLIO XX"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Security"]
C --- T1
T2["Ingress"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Basic Auth on Kubernetes Ingress
flowchart LR
C(("FOLIO XVIII"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Security"]
C --- T1
T2["Ingress"]
C --- T2
T3["Docker"]
C --- T3
T4["Golang"]
C --- T4
classDef ser fill:transparent,stroke-dasharray:3 3
JWT Microservice, Tokenize Remote APIs
flowchart LR
C(("FOLIO XIX"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Security"]
C --- T1
T2["Ingress"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Let's Encrypt, Kubernetes
flowchart LR
C(("FOLIO XVI"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["helm"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Helm on Custom Kubernetes
flowchart LR
C(("FOLIO XVII"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["DNS"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Kubernetes Custom Upstream DNS
flowchart LR
C(("FOLIO XIV"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["ingress"]
C --- T1
T2["nginx"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Ingress on Custom Kubernetes
flowchart LR
C(("FOLIO XV"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["kubectl"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
kubectl Context Multiple Clusters
flowchart LR
C(("FOLIO XIII"))
T0["MacOs"]
C --- T0
T1["Utils"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Formatting Drives on MacOS
flowchart LR
C(("FOLIO XII"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["kubectl"]
C --- T1
T2["security"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Kubectl x509 Unable to Connect
flowchart LR
C(("FOLIO XI"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
classDef ser fill:transparent,stroke-dasharray:3 3
Production Hobby Cluster
flowchart LR
C(("FOLIO IX"))
T0["Raspberry Pi"]
C --- T0
T1["Utils"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
rSync Files on Interval
flowchart LR
C(("FOLIO X"))
T0["SQL"]
C --- T0
T1["Data"]
C --- T1
T2["Database"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
SQL Foundations
flowchart LR
C(("FOLIO VIII"))
T0["MacOS"]
C --- T0
T1["Utils"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Burn SD Images on MacOs
flowchart LR
C(("FOLIO V"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["ingress"]
C --- T1
T2["nginx"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Kubernetes - 413 Request Entity Too Large
flowchart LR
C(("FOLIO VI"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["kubectl"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Kubernetes Remote Control
flowchart LR
C(("FOLIO VII"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Microservices"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Microservices & Kubernetes
flowchart LR
C(("FOLIO II"))
S["SERIES · Cassandra"]:::ser
C --- S
T0["Cassandra"]
C --- T0
T1["Docker"]
C --- T1
T2["Data"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Don't Install cqlsh
flowchart LR
C(("FOLIO III"))
T0["Emacs"]
C --- T0
T1["Docker"]
C --- T1
classDef ser fill:transparent,stroke-dasharray:3 3
Don't Install Emacs
flowchart LR
C(("FOLIO IV"))
S["SERIES · Kubernetes"]:::ser
C --- S
T0["Kubernetes"]
C --- T0
T1["Microservices"]
C --- T1
T2["Docker"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Kubernetes Overview
flowchart LR
C(("FOLIO I"))
S["SERIES · Raspberry Pi"]:::ser
C --- S
T0["Raspberry Pi"]
C --- T0
T1["Development"]
C --- T1
T2["IOT"]
C --- T2
classDef ser fill:transparent,stroke-dasharray:3 3
Raspberry Pi - Serial Number