# From Mesos to KUDO

When I started on KUDO, the first thing I needed to wrap my head around was the difference between Mesos (opens new window) and Kubernetes (opens new window). I already had operator level experience with both, but my technical understanding of Mesos was high and I needed a more intimate understanding of Kubernetes. In the process of coming up to speed I threw together a guide for other Mesonauts. I've been encouraged that this guide could be useful for a wider audience. This guide assumes that you know or understand Mesos and its eco-system, and that you are new to Kubernetes.

# Kubernetes Operators and KUDO

The purpose of KUDO is to be able to make the creation and management of Kubernetes operators (opens new window) simple and reusable. A Kubernetes operator is similar to a Mesos Framework. Operators are a piece of software that connects to the Kubernetes cluster and controls aspects of provisioning and maintenance strategies to the cluster, usually to achieve the availability of a specific service. A simple provisioning example is HDFS. For HA of HDFS, it is necessary to install 3 JournalNodes (JN) on 3 separate fault domains (commonly across 3 different worker nodes), then provision and format a NameNode (NN) on 1 of the JN nodes, then bootstrap a 2nd NN on one of the remaining JN nodes and so on. This recipe is codified into a controller which in Kubernetes parlance is called an operator.

With years of experience writing Mesos Frameworks, Mesosphere created an SDK to make the creation of Frameworks simple. A lot of these are as simple as a YAML configuration and we believe that the same opportunity exists for Kubernetes operators which is the purpose of KUDO: KUDO's goal is to simplify creation of an operator.

This includes:

standard conventions of defining provision plans
standard expectations for install, uninstall, upgrade, backup/restore
CLI integration for control of the operator

# From Frameworks to Operators

The concept of operators is relatively new: it was introduced near the end of 2016 (opens new window). Yet, they have a lot in common with Mesos Frameworks (opens new window). The following is provided to help you jump-start the Kubernetes learning curve assuming you already have a background in the Mesos community, and have limited experience with Kubernetes. This is all the information needed to get involved with KUDO (we’d love if you did).

The following is a checklist with resources to accelerate that learning curve. This blog post goes into each of these topics:

Learn Kubernetes Basics and Internals
Learn kubectl
Learn Kubernetes API / Resources
Learn Custom Resource Definitions
Learn Kubernetes Operators
Dynamic CRD / KUDO

# Learn Kubernetes Basics

The Kubernetes Docs (opens new window) are fantastic. I suggest you get started with Minikube and become familiar with the basic concepts. At a minimum, go through the Hello Minikube (opens new window) tutorial.

Important in the learning at this phase is understanding the concept of "Kind" which is one of the top level API Objects (opens new window). Kinds (opens new window). Kinds for a default cluster includes Pod, ReplicationController, Service, Namespace, Node. Each of these you can query or manipulate from the CLI using kubectl, for example: kubectl get pod.

Now that you have some basics, it is important to understand the communication and event based system that is behind Kubernetes with this video: Events, the DNA of Kubernetes (opens new window). The event system details provided by this blog post are crucial for understanding the core of Kubernetes.

Resources:

Kubernetes by Example (opens new window)

# Learn kubectl

This will just take time but it is incredibly useful to start with kubectl Cheat Sheet (opens new window).

It is useful to understand creating objects from the CLI and from files. It is important to understand that Kubernetes works with YAML, JSON and protobuf. It uses and stores objects in protobuf internally and can output in a number of formats which include YAML or JSON. Try:

kubectl create deployment nginx --image=nginx
## Followed by
kubectl get deployment -o yaml

YAML is commonly the file format of choice. JSON is useful when working with jq for a specific element.

For deeper learning into Kubernetes there are 2 options which are superb: -w for watch and -v=7 for verbose level. If you are trying to understand deployments try the following:

In terminal 1: kubectl get deployment -w -v=7
In terminal 2: kubectl create deployment nginx --image=nginx

Resources:

kubectl plugins (opens new window)

# Learn Kubernetes API / Resources

It is important to understand that all interactions with Kubernetes is through the API Server. Start with the Kubernetes API Concepts (opens new window). Focus on Resources and Resource Types. They are defined as part of the API. Understand the basic format of /apis/GROUP/VERSION/RESOURCETYPE or /apis/GROUP/VERSION/RESOURCETYPE/NAME for working with a resource.

A great way to learn the base API is to kubectl proxy. Then browse the API with curl localhost:8001 or curl localhost:8001/apis/batch/v1

Once you have the basics read Michael Hausenblas's post on Kubernetes deep dive: API Server - part 1 (opens new window). Followed by Part 2 (opens new window) and Part 3 (opens new window).

# Learn Custom Resource Definitions

Now that you understand the API / Resources that come with Kubernetes, you might be wondering am I limited to that? or can I extend that? That is where Custom Resource Definitions (CRD) come in. You can extend Kubernetes by defining extensions to its API. You might start with the official CRD docs (opens new window). It is worth going through the exercises presented there. If you want to watch and learn, start with Stefan Schimanski's CRD talk (opens new window) on the topic.

Additional resources:

# Learn Kubernetes Operators

In order to understand operators, it is useful to understand Kubernetes controllers (opens new window) first, which may raise the question what the difference is. It is best answered on stackoverflow (opens new window). The Kubernetes community has a great read on Writing Controllers (opens new window) which includes a sample controller (opens new window).

Resources:

# KUDO

Now we can talk KUDO. KUDO is about creating CRDs on the fly based on information passed to it or Dynamic CRD. It is about being 1 operator for a number of operator configurations.

From the KUDO Website (opens new window) it is worth looking at the comparison guide (opens new window).

The KUDO project (opens new window) is written in Go. It was initially created using kubebuilder. It also contains the CLI code. CI service is by circleci. All artifacts can be built (if all prerequisites are met) with the Makefile. Releases as outlined (opens new window) are released with goreleaser (opens new window).

Example operators on in the operator GH project (opens new window).

Follow the Readme.md (opens new window) and guides to get started.

Features are proposed and eventually added to KUDO using the KUDO Enhancement Process (opens new window) (KEP). This is a simplified variant of the Kubernetes Enhancement Process.

Design Work:

About the author
Ken Sipe is a Distributed Application Engineer at D2iQ working on the Orchestration team. In addition to being a committer on the KUDO project, Ken is an author and award winning international speaker on the practices of software architecture and engineering and has been honored with the JavaOne Rockstar Award. Ken is also a contributor to Apache Mesos and a committer on a number of OSS projects including Marathon and Metronome. When not coding or talking about code, Ken is an IFR private pilot, a SCUBA dive master and has recently taken up the art of glass blowing. He is based in St. Charles, MO, USA. Find Ken on GitHub Twitter

← KUDO Terraform Provider KUDO, Mesos, and Kubernetes →