IMTI

(I Make Things Interesting) Architecting, Developing, nixCraft, DevOps, AI/ML, Blockchain

FaaS on Kubernetes

Kubeless, Python and Elasticsearch

FaaS or Function as a Service also known as Serverless computing implementations are gaining popularity. Discussed often are the cost savings and each implementations relationship to the physical and network architecture of a specific platform or vendor. While many of the cost and infrastructure advantages of FaaS are compelling, its only one of many advantages. Below, I hope to demonstrate how easy it is to develop and deploy FaaS components into a custom Kubernetes cluster.

Posted by Craig Johnston Saturday, July 28, 2018

Elasticsearch Essential Queries

Getting started with Elasticsearch

The following is an overview for querying Elasticsearch. Over the years I have tried to assemble developer notes for myself and my team on a variety of platforms, languages and frameworks, a type of cheat-sheet but with context, not a comprehensive how-to, but a decent 15-minute overview of the features we are most likely to implement in a given iteration. Explore the Elasticsearch official documentation: Search in Depth. Contents Motivation Following Along with Elasticsearch and Kubernetes Vocabulary Basic CRUD API Delete an Index Create an Index Create or Update a Document (Upsert) Get a Document Mappings, Types and Metadata Get Mapping Create a Mapping Searching Range Filtering Aggregations Counts Averages, Minimums and Maximums Percentile Percent by Rank Percent by Rank Interval Resources Support this blog!

Posted by Craig Johnston Thursday, July 26, 2018

Remote Query Elasticsearch on Kubernetes

Local workstation-based microservices development

Developing on our local workstations has always been a conceptual challenge for my team when it comes to remote data access. Local workstation-based development of services that intend to connect to a wide range of remote services that may have no options for external connections poses a challenge. Mirroring the entire development environment is possible in many cases, just not practical. In days before Kubernetes, writing code in IDEs on our local workstation meant we had only a few options for developing server-side-API-style services that needed to connect to a database.

Posted by Craig Johnston Wednesday, July 25, 2018

High Traffic JSON Data into Elasticsearch on Kubernetes

Instant, reliable, send and forget.

IOT devices, Point-of-Sale systems, application events or any client that sends data destined for indexing in Elasticsearch often need to send and forget, however, unless that data is of low value there needs to be assurance that arrives at its final destination. Back-pressure and database outages can pose a considerable threat to data integrity. Contents Background Overview Development Environment the-project Namespace The Project: Weather (wx) Data rxtx for Store-and-Forward wx-rxtx Service wx-rxtx StatefulSet rtBeat to Collect, Buffer and Publish wx-rtbeat Service wx-rtbeat ConfigMap wx-rtbeat Deployment Client Simulation / Kubernetes Cron Performance Conclusion Port Forwarding / Local Development Reference Support this blog!

Posted by Craig Johnston Wednesday, July 18, 2018

Kibana on Kubernetes

Visualize your Elasticsearch data.

This guide walks through a process for setting up Kibana within a namespace on a Kubernetes cluster. If you followed along with Production Grade Elasticsearch on Kubernetes then aside from personal or corporate preferences, little modifications are necessary for the configurations below. Contents Project Namespace Service Kibana ConfigMap Deployment Basic Auth (Optional) TLS Certificate (Optional) Ingress Conclusion Port Forwarding / Local Development Resources Support this blog!

Posted by Craig Johnston Sunday, July 15, 2018

Production Grade Elasticsearch on Kubernetes

Setup a fast, custom production grade Elasticsearch cluster.

Installing production ready, Elasticsearch 6.2 on Kubernetes requires a hand full of simple configurations. The following guide is a high-level overview of an installation process using Elastic’s recommendations for best practices. The Github project kubernetes-elasticsearch-cluster is used for the Elastic Docker container and built to operate Elasticsearch with nodes dedicated as Master, Data, and Client/Ingest. The Docker container docker-elasticsearch, a “Ready to use, lean and highly configurable Elasticsearch container image.” by pires is sufficient for use in this guide.

Posted by Craig Johnston Saturday, July 14, 2018

FEATURED TAGS
blockchain data data-science development devops docker elasticsearch golang ingress iot kubectl kubernetes microservices python raspberry-pi security utils
ABOUT ME

I lead a talented team of software development and creative engineers, covering many industries looking to collect, analyze, move, buffer, queue, process and present data in significant ways. My expertise and that of my team revolve around microservices, artificial intelligence, algorithms, machine learning and blockchain technologies.

BOOKMARKS
REDDIT
AWESOME
DOCKER CONTAINERS