# From Mesos to KUDO
When I started on KUDO, the first thing I needed to wrap my head around was the difference between Mesos and Kubernetes. I already had operator level experience with both, but my technical understanding of Mesos was high and I needed a more intimate understanding of Kubernetes. In the process of coming up to speed I threw together a guide for other Mesonauts. I've been encouraged that this guide could be useful for a wider audience. This guide assumes that you know or understand Mesos and its eco-system, and that you are new to Kubernetes.
# Kubernetes Operators and KUDO
The purpose of KUDO is to be able to make the creation and management of Kubernetes operators simple and reusable. A Kubernetes operator is similar to a Mesos Framework. Operators are a piece of software that connects to the Kubernetes cluster and controls aspects of provisioning and maintenance strategies to the cluster, usually to achieve the availability of a specific service. A simple provisioning example is HDFS. For HA of HDFS, it is necessary to install 3 JournalNodes (JN) on 3 separate fault domains (commonly across 3 different worker nodes), then provision and format a NameNode (NN) on 1 of the JN nodes, then bootstrap a 2nd NN on one of the remaining JN nodes and so on. This recipe is codified into a controller which in Kubernetes parlance is called an operator.
With years of experience writing Mesos Frameworks, Mesosphere created an SDK to make the creation of Frameworks simple. A lot of these are as simple as a YAML configuration and we believe that the same opportunity exists for Kubernetes operators which is the purpose of KUDO: KUDO's goal is to simplify creation of an operator.
- standard conventions of defining provision plans
- standard expectations for install, uninstall, upgrade, backup/restore
- CLI integration for control of the operator
# From Frameworks to Operators
The concept of operators is relatively new: it was introduced near the end of 2016. Yet, they have a lot in common with Mesos Frameworks. The following is provided to help you jump-start the Kubernetes learning curve assuming you already have a background in the Mesos community, and have limited experience with Kubernetes. This is all the information needed to get involved with KUDO (we’d love if you did).
The following is a checklist with resources to accelerate that learning curve. This blog post goes into each of these topics:
- Learn Kubernetes Basics and Internals
- Learn kubectl
- Learn Kubernetes API / Resources
- Learn Custom Resource Definitions
- Learn Kubernetes Operators
- Dynamic CRD / KUDO
# Learn Kubernetes Basics
Important in the learning at this phase is understanding the concept of "Kind" which is one of the top level API Objects. Kinds. Kinds for a default cluster includes
Node. Each of these you can query or manipulate from the CLI using
kubectl, for example:
kubectl get pod.
Now that you have some basics, it is important to understand the communication and event based system that is behind Kubernetes with this video: Events, the DNA of Kubernetes. The event system details provided by this blog post are crucial for understanding the core of Kubernetes.
# Learn kubectl
This will just take time but it is incredibly useful to start with kubectl Cheat Sheet.
It is useful to understand creating objects from the CLI and from files. It is important to understand that Kubernetes works with YAML, JSON and protobuf. It uses and stores objects in protobuf internally and can output in a number of formats which include YAML or JSON. Try:
kubectl create deployment nginx --image=nginx ## Followed by kubectl get deployment -o yaml
YAML is commonly the file format of choice. JSON is useful when working with
jq for a specific element.
For deeper learning into Kubernetes there are 2 options which are superb:
-w for watch and
-v=7 for verbose level.
If you are trying to understand deployments try the following:
- In terminal 1:
kubectl get deployment -w -v=7
- In terminal 2:
kubectl create deployment nginx --image=nginx
# Learn Kubernetes API / Resources
It is important to understand that all interactions with Kubernetes is through the API Server. Start with the Kubernetes API Concepts. Focus on Resources and Resource Types. They are defined as part of the API. Understand the basic format of
/apis/GROUP/VERSION/RESOURCETYPE/NAME for working with a resource.
A great way to learn the base API is to
kubectl proxy. Then browse the API with
curl localhost:8001 or
# Learn Custom Resource Definitions
Now that you understand the API / Resources that come with Kubernetes, you might be wondering am I limited to that? or can I extend that? That is where Custom Resource Definitions (CRD) come in. You can extend Kubernetes by defining extensions to its API. You might start with the official CRD docs. It is worth going through the exercises presented there. If you want to watch and learn, start with Stefan Schimanski's CRD talk on the topic.
# Learn Kubernetes Operators
In order to understand operators, it is useful to understand Kubernetes controllers first, which may raise the question what the difference is. It is best answered on stackoverflow. The Kubernetes community has a great read on Writing Controllers which includes a sample controller.
- What is an Operator
- CoreOs Blog on Operators
- Operator SDK
- Article on Writing first operator
- Operator from CRD Tutorial
Now we can talk KUDO. KUDO is about creating CRDs on the fly based on information passed to it or Dynamic CRD. It is about being 1 operator for a number of operator configurations.
The KUDO project is written in Go. It was initially created using kubebuilder. It also contains the CLI code. CI service is by circleci. All artifacts can be built (if all prerequisites are met) with the Makefile. Releases as outlined are released with goreleaser.
Example operators on in the operator GH project.
Follow the Readme.md and guides to get started.
Features are proposed and eventually added to KUDO using the KUDO Enhancement Process (KEP). This is a simplified variant of the Kubernetes Enhancement Process.