# From Mesos to KUDO
When I started on KUDO, the first thing I needed to wrap my head around was the difference between Mesos (opens new window) and Kubernetes (opens new window). I already had operator level experience with both, but my technical understanding of Mesos was high and I needed a more intimate understanding of Kubernetes. In the process of coming up to speed I threw together a guide for other Mesonauts. I've been encouraged that this guide could be useful for a wider audience. This guide assumes that you know or understand Mesos and its eco-system, and that you are new to Kubernetes.
# Kubernetes Operators and KUDO
The purpose of KUDO is to be able to make the creation and management of Kubernetes operators (opens new window) simple and reusable. A Kubernetes operator is similar to a Mesos Framework. Operators are a piece of software that connects to the Kubernetes cluster and controls aspects of provisioning and maintenance strategies to the cluster, usually to achieve the availability of a specific service. A simple provisioning example is HDFS. For HA of HDFS, it is necessary to install 3 JournalNodes (JN) on 3 separate fault domains (commonly across 3 different worker nodes), then provision and format a NameNode (NN) on 1 of the JN nodes, then bootstrap a 2nd NN on one of the remaining JN nodes and so on. This recipe is codified into a controller which in Kubernetes parlance is called an operator.
With years of experience writing Mesos Frameworks, Mesosphere created an SDK to make the creation of Frameworks simple. A lot of these are as simple as a YAML configuration and we believe that the same opportunity exists for Kubernetes operators which is the purpose of KUDO: KUDO's goal is to simplify creation of an operator.
- standard conventions of defining provision plans
- standard expectations for install, uninstall, upgrade, backup/restore
- CLI integration for control of the operator
# From Frameworks to Operators
The concept of operators is relatively new: it was introduced near the end of 2016 (opens new window). Yet, they have a lot in common with Mesos Frameworks (opens new window). The following is provided to help you jump-start the Kubernetes learning curve assuming you already have a background in the Mesos community, and have limited experience with Kubernetes. This is all the information needed to get involved with KUDO (we’d love if you did).
The following is a checklist with resources to accelerate that learning curve. This blog post goes into each of these topics:
- Learn Kubernetes Basics and Internals
- Learn kubectl
- Learn Kubernetes API / Resources
- Learn Custom Resource Definitions
- Learn Kubernetes Operators
- Dynamic CRD / KUDO
# Learn Kubernetes Basics
The Kubernetes Docs (opens new window) are fantastic. I suggest you get started with Minikube and become familiar with the basic concepts. At a minimum, go through the Hello Minikube (opens new window) tutorial.
Important in the learning at this phase is understanding the concept of "Kind" which is one of the top level API Objects (opens new window). Kinds (opens new window). Kinds for a default cluster includes
Node. Each of these you can query or manipulate from the CLI using
kubectl, for example:
kubectl get pod.
Now that you have some basics, it is important to understand the communication and event based system that is behind Kubernetes with this video: Events, the DNA of Kubernetes (opens new window). The event system details provided by this blog post are crucial for understanding the core of Kubernetes.
# Learn kubectl
This will just take time but it is incredibly useful to start with kubectl Cheat Sheet (opens new window).
It is useful to understand creating objects from the CLI and from files. It is important to understand that Kubernetes works with YAML, JSON and protobuf. It uses and stores objects in protobuf internally and can output in a number of formats which include YAML or JSON. Try:
kubectl create deployment nginx --image=nginx ## Followed by kubectl get deployment -o yaml
YAML is commonly the file format of choice. JSON is useful when working with
jq for a specific element.
For deeper learning into Kubernetes there are 2 options which are superb:
-w for watch and
-v=7 for verbose level.
If you are trying to understand deployments try the following:
- In terminal 1:
kubectl get deployment -w -v=7
- In terminal 2:
kubectl create deployment nginx --image=nginx
# Learn Kubernetes API / Resources
It is important to understand that all interactions with Kubernetes is through the API Server. Start with the Kubernetes API Concepts (opens new window). Focus on Resources and Resource Types. They are defined as part of the API. Understand the basic format of
/apis/GROUP/VERSION/RESOURCETYPE/NAME for working with a resource.
A great way to learn the base API is to
kubectl proxy. Then browse the API with
curl localhost:8001 or
Once you have the basics read Michael Hausenblas's post on Kubernetes deep dive: API Server - part 1 (opens new window). Followed by Part 2 (opens new window) and Part 3 (opens new window).
# Learn Custom Resource Definitions
Now that you understand the API / Resources that come with Kubernetes, you might be wondering am I limited to that? or can I extend that? That is where Custom Resource Definitions (CRD) come in. You can extend Kubernetes by defining extensions to its API. You might start with the official CRD docs (opens new window). It is worth going through the exercises presented there. If you want to watch and learn, start with Stefan Schimanski's CRD talk (opens new window) on the topic.
- Custom Resources (opens new window)
- Extending Kubernetes with Custom Resources (opens new window)
- Kubernetes CRD Tutorial (opens new window)
# Learn Kubernetes Operators
In order to understand operators, it is useful to understand Kubernetes controllers (opens new window) first, which may raise the question what the difference is. It is best answered on stackoverflow (opens new window). The Kubernetes community has a great read on Writing Controllers (opens new window) which includes a sample controller (opens new window).
- What is an Operator (opens new window)
- CoreOs Blog on Operators (opens new window)
- Operator SDK (opens new window)
- Article on Writing first operator (opens new window)
- Operator from CRD Tutorial (opens new window)
Now we can talk KUDO. KUDO is about creating CRDs on the fly based on information passed to it or Dynamic CRD. It is about being 1 operator for a number of operator configurations.
The KUDO project (opens new window) is written in Go. It was initially created using kubebuilder. It also contains the CLI code. CI service is by circleci. All artifacts can be built (if all prerequisites are met) with the Makefile. Releases as outlined (opens new window) are released with goreleaser (opens new window).
Example operators on in the operator GH project (opens new window).
Follow the Readme.md (opens new window) and guides to get started.
Features are proposed and eventually added to KUDO using the KUDO Enhancement Process (opens new window) (KEP). This is a simplified variant of the Kubernetes Enhancement Process.