IMTI

(I Make Things Interesting) Architecting, Developing, nixCraft, DevOps, AI/ML, Blockchain

kubectl Context Multiple Clusters

Managing multiple clusters with kubectl.

I use a few Kubernetes clusters on a daily basis, and I use kubectl to access and configure them from my workstation. There are dozens of ways to configure kubectl however I find the following method the easiest for me to manage and not make a mess. I also set up test clusters from time-to-time, and so keeping my configs organized is, so I don’t confuse myself or make a mess.

Posted by Craig Johnston Tuesday, May 15, 2018

Formatting Drives on MacOS

A stack of old drives, a terminal and diskutil.

I have had this collection of old external drives hanging around for years. There was a time I was having terrible luck with hard drives. It turned out I managed to amass a collection of about eight drives from one terabyte to four terabytes. Fortunately, I keep copies of nearly all my files on cloud drives, split between Amazon, Google and DropBox. I had a sneaking suspicion that the discs themselves were ok and somehow my Mac was communicating with the RAID controllers in a way that caused them to fail.

Posted by Craig Johnston Saturday, May 12, 2018

Kubectl x509 Unable to Connect

Kubernetes remote access and TLS certs.

Just set up a brand new cluster? Changed the domain or IP of your admin node? Then you may have encountered the error Unable to connect to the server: x509: certificate is valid for …. The following is a fix for this common issue. However, there are often other reasons to rebuild your cluster cert, and it’s relatively easy. TL;DR: “I don’t care about the fix I need to remote control my cluster.

Posted by Craig Johnston Thursday, May 10, 2018

Production Hobby Cluster

Production-grade cluster on a hobby budget.

Setting up a production-grade Kubernetes cluster can be done on a hobby budget, and if this is true why mess around with a lesser grade. If you are investing time to learn distributed cloud computing or microservices, is the distance between $0 and 15 dollars a month worth the time in translating best practices? Kubernetes is designed to host production applications. My personal web applications may only be hobbies, but they might as well be production grade hobbies.

Posted by Craig Johnston Wednesday, May 9, 2018

rSync Files on Interval

Sync media to Raspberry Pi or any ARM SoC.

A recurring requirement for my IOT projects involves keeping a set of files synced with a central server. Many of these projects include media players, kiosk systems, or applications that need frequently updated configuration files, all while entirely unattended, and in most cases unreachable through firewalls. I have one project that alone has 2000+ devices pulling media continuously from an rsync server. Many of these devices are on doggy wifi networks.

Posted by Craig Johnston Monday, April 2, 2018

SQL Foundations

Selects, joins and aliases.

The following is an attempt at explaining the basics of an SQL query, and more importantly how I believe you can best think through them. All queries can be broken down into the basics of this declarative language. I recently helped a co-worker read through a large SQL query with a few dozen joins and left joins, alias, and recursions. He is mostly a front-end integrator and although he has been tinkering with SQL for years, he never really understood the basics.

Posted by Craig Johnston Monday, April 2, 2018

Burn SD Images on MacOs

Use the command line to burn SD cards, easy and fast.

Use your terminal to burn images fast and easy with dd. I do a lot of professional and hobby development for projects using devices such as Raspberry Pi, Orange Pi, Libre Computer, Tinker Board, etc. I run across a lot of tutorials with people downloading and using big GUI apps with clunky drag and drop interfaces to burn images. It’s one command in your terminal. Technically, it’s three, but I don’t count listing and unmounting as the final act of burning.

Posted by Craig Johnston Sunday, April 1, 2018

Kubernetes - 413 Request Entity Too Large

Configuring the NGINX Ingress Controller

When setting up nginx ingress on Kubernetes for a private Docker Registry, I ran into an error when trying to push an image to it. Error parsing HTTP response: invalid character '<' looking for beginning of value: "<html>\r\n<head><title>413 Request Entity Too Large</title></head>\r\n<body bgcolor=\"white\">\r\n<center><h1>413 Request Entity Too Large</h1></center>\r\n<hr><center>nginx/1.9.14</center>\r\n</body>\r\n</html>\r\n" The “413 Request Entity Too Large” error is something many accustomed to running nginx as a standard web server/proxy. nginx is configured to restrict the size of files it will allow over a post.

Posted by Craig Johnston Saturday, March 24, 2018

Kubernetes Remote Control

Using kubectl to Control a Remote Kubernetes Cluster

I use Minikube to run a local Kubernetes single node cluster (cluster?). However, I also work with a custom production cluster for work. This cluster consists of development and production nodes. I often need to switch between working on my local Minikube and the online Kubernetes cluster. TIP: Visit the kubectl Cheat Sheet often. Support this blog! Buy my new book: Advanced Platform Development with Kubernetes What You'll Learn Build data pipelines with MQTT, NiFi, Logstash, MinIO, Hive, Presto, Kafka and Elasticsearch Leverage Serverless ETL with OpenFaaS Explore Blockchain networking with Ethereum Support a multi-tenant Data Science platform with JupyterHub, MLflow and Seldon Core Build a Multi-cloud, Hybrid cluster, securely bridging on-premise and cloud-based Kubernetes nodes The default configuration kubectl is stored in ~/.

Posted by Craig Johnston Saturday, March 24, 2018

Microservices & Kubernetes

Overview

The following is a collection of articles, videos, and notes on Microservices. The Microservices architecture is a variant of the service-oriented architecture (SOA), a collection of loosely coupled services. Support this blog! Buy my new book: Advanced Platform Development with Kubernetes What You'll Learn Build data pipelines with MQTT, NiFi, Logstash, MinIO, Hive, Presto, Kafka and Elasticsearch Leverage Serverless ETL with OpenFaaS Explore Blockchain networking with Ethereum Support a multi-tenant Data Science platform with JupyterHub, MLflow and Seldon Core Build a Multi-cloud, Hybrid cluster, securely bridging on-premise and cloud-based Kubernetes nodes Articles Background concept - “Open Data: Small Pieces Loosely Joined”, Tim O’Reilly Modern software design problems and solutions - “12-Fractured Apps”, Kelsey Hightower (SysAdmin @ Google) 12-Factor Defined - “The 12-Factor App”, Adam Wiggins Pros and Cons of Microservices - “Microservices” and Microservice Trade-Offs, Martin Fowler “What are containers and why do you need them?

Posted by Craig Johnston Saturday, March 24, 2018

Don't Install cqlsh

Containers as utility applications

We live in a world of process isolation and tools that make utilizing it extremely simple, with apps like Docker we can perform dependency management with dependency isolation. As I am slowly becoming a fanboy of containerization, I look forward to the day when typing ps on my local workstation or remote server is nearly synonymous with commands like docker ps or kubectl get services. Case: Cassandra development and your local workstation.

Posted by Craig Johnston Thursday, March 1, 2018

Don't Install Emacs

Containers as utility applications

I grew up on emacs. One of my first jobs I sat down at a terminal and was editing some files with pico, it’s what I knew since I used that fantastic email client pine. I was quickly told by my the lead developer that I need to use a real text editor if I’m going to progress in my career. He told me I need to try emacs, and after suffering through a few weeks of memorizing multi command-char sequences and training the muscle memory in my pinky to perform bizarre contortions of my left hand just to save my file, I became a convert.

Posted by Craig Johnston Thursday, March 1, 2018

Kubernetes Overview

Container Orchestration & Microservices

Getting started with Kubernetes for local development. I develop on a Mac however much of this is easily translated to windows. The following is primarily a getting started guide wrapped around my personal development notes. This set of notes are specifically for my co-workers in helping them get up to speed quickly. If you see an error feel free to make a pull request or just add an issue. Contents Deeper Reading and Resources Free Courses Paid Courses Prerequisites Test Installation Get some status Architecture Create a Deployment Useful Commands Support this blog!

Posted by Craig Johnston Thursday, March 1, 2018

Raspberry Pi - Serial Number

Getting the unique serial number from a Raspberry Pi.

Getting the unique serial number from a Raspberry Pi. cat /proc/cpuinfo | grep ^Serial | cut -d":" -f2 Example output: 00000000e215b4a2 An interesting use for this is “binding” software, encryption or other servcies to a specific Pi. Found this in a suggestion on the Stack Overflow question “Securing data on SD card Raspberry Pi” Support this blog! Buy my new book: Advanced Platform Development with Kubernetes What You'll Learn Build data pipelines with MQTT, NiFi, Logstash, MinIO, Hive, Presto, Kafka and Elasticsearch Leverage Serverless ETL with OpenFaaS Explore Blockchain networking with Ethereum Support a multi-tenant Data Science platform with JupyterHub, MLflow and Seldon Core Build a Multi-cloud, Hybrid cluster, securely bridging on-premise and cloud-based Kubernetes nodes

Posted by Craig Johnston Monday, February 13, 2017

FEATURED TAGS
blockchain data data-science development devops docker elasticsearch golang ingress iot kubectl kubernetes microservices python raspberry-pi security utils
ABOUT ME

I lead a talented team of software development and creative engineers, covering many industries looking to collect, analyze, move, buffer, queue, process and present data in significant ways. My expertise and that of my team revolve around microservices, artificial intelligence, algorithms, machine learning and blockchain technologies.

BOOKMARKS
REDDIT
AWESOME
DOCKER CONTAINERS