Navigate back to the homepage
📚 Books

Understanding the Kubernetes architecture

Mark Pollmann
January 15th, 2021 · 3 min read

Introduction

One cool thing about Kubernetes is that the infrastructure is transparently handled by Kubernetes objects themselves. If you want to see how everything works together, you can just inspect the pods, services and other objects themselves.

Disclaimer: This setup is valid for Kubernetes in Docker Desktop on Mac. In a managed environment, or in your setup, the setup might be different and components are not necessarily deployed as simple pods.

Let’s have a first look. I’m using the built-in Kubernetes for Docker Desktop on a Mac here (and use k as an alias to kubectl)

1╰$ k get all
2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
3service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 39d

Well, not a lot going on here. Is that all there is?

Of course not, the real meat is in other namespaces. Anyway, let’s first inspect this service we found:

1╰$ k describe service/kubernetes
2Name: kubernetes
3Namespace: default
4Labels: component=apiserver
5 provider=kubernetes
6Annotations: <none>
7Selector: <none>
8Type: ClusterIP
9IP: 10.96.0.1
10Port: https 443/TCP
11TargetPort: 6443/TCP
12Endpoints: 192.168.65.3:6443
13Session Affinity: None
14Events: <none>

What is it doing? It’s actually just forwarding requests to the Kubernetes API server (which we will meet soon).

Okay, but where is the rest?

Let’s check out the other namespaces.

1╰$ k get namespaces
2NAME STATUS AGE
3default Active 39d
4kube-node-lease Active 39d
5kube-public Active 39d
6kube-system Active 39d

The default namespace we already know.

kube-node-lease is relatively new (starting from version 1.14) and helps in determining the availability of a node via heartbeats. We’ll ignore it for now.

kube-public is a namespace that exists if the cluster was created with kudeadm. It contains a single, lonely configmap called config-info and aids in discovery for other clients. Read more about the discovery API if you are interested.

kube-system is the most interesting namespace for us. Let’s dive in:

The kube-system namespace

Let’s do a quick lookup:

1╰$ k get all -n kube-system
2NAME READY STATUS RESTARTS AGE
3pod/coredns-f9fd979d6-dmkxf 1/1 Running 0 4m57s
4pod/coredns-f9fd979d6-vrd8t 1/1 Running 0 4m57s
5pod/etcd-docker-desktop 1/1 Running 0 3m43s
6pod/kube-apiserver-docker-desktop 1/1 Running 0 4m7s
7pod/kube-controller-manager-docker-desktop 1/1 Running 0 3m45s
8pod/kube-proxy-s4jfp 1/1 Running 0 4m57s
9pod/kube-scheduler-docker-desktop 1/1 Running 0 3m51s
10pod/storage-provisioner 1/1 Running 0 3m43s
11pod/vpnkit-controller 1/1 Running 0 3m42s
12
13NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
14service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 5m4s
15
16NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
17daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 5m4s
18
19NAME READY UP-TO-DATE AVAILABLE AGE
20deployment.apps/coredns 2/2 2 2 5m4s
21
22NAME DESIRED CURRENT READY AGE
23replicaset.apps/coredns-f9fd979d6 2 2 2 4m57s

By the way: kubectl get all does not really get all resource types, only the most important. Check out ketall if you want to get everything.

A lot more going on here! We can see see a deployment (plus replicaset and its pods) of coredns and a service kube-dns, which, unsurprisingly, handle DNS in the cluster. I will skip this for now and write a separate post about DNS in Kubernetes.

So, what is left? I count 7 pods, let’s go through them one by one.

VPN Controller

pod/vpnkit-controller is there to interact with my host’s VPN configuration. Not very interesting for us right now.

etcd

etcd is a key-value store that stores information concerning the cluster, like pods, nodes, roles, configs, secrets. Think of it as its database. Every piece of information you get via kubectl get comes from etcd and every update you make goes into etcd. As Kubernetes is a distributed system the database needs to be distributed, too, and etcd is the perfect solution for it.

API server

kube-apiserver is quite self-explanatory. This is the endpoint with which other components interact with REST operations if they want to lookup or change state. If you run kubectl you are interacting with the apiserver.

It’s also the only component that interacts with etcd.

Controller Manager

kube-controller-manager. Controllers (e.g. Node controllers, deployment controllers, namespace controllers) supervise the state of the components they are in charge of. Node controllers, for example, check the health of nodes every five seconds to see if they are still reachable. If they don’t respond after 40 seconds it is marked unreachable. If it doesn’t come up again after 5 minutes it evicts the pods scheduled on it to other nodes.

All these controllers run in the process of the controller-manager.

Kube Proxy

kube-proxy runs on every node and handles service networking. As a service is not a real object but just lives in memory, somebody needs to handle the routing of services and kube-proxy does just this. One way it does this is with iptable rules. On every node it creates entries for every service and forwards traffic to the IP of the actual pod selecting the service.

Kube Scheduler

kube-scheduler decides which pod gets scheduled on which node. Any pod might have specific resource requirements to run.

In the first phase the scheduler filters out the nodes which can’t satisfy the resource requirements or are tainted or don’t tolerate this specific pod.

Then it ranks the remaining nodes by a priority function. If the pod is CPU-intensive the node with more free CPU capacity gets a higher rank and so one. The one with the best ranking wins the pod.

Storage Provisioner

storage-provisioner handles storage specific to the platform being used. In my case this pod is using the docker/desktop-storage-provisioner image specific for Mac’s Docker Desktop.

By the way, you can see more of the configuration for some of these by looking at the config maps in the kube-system namespace

1╰$ k get cm -n kube-system
2NAME DATA AGE
3coredns 1 6m5s
4extension-apiserver-authentication 6 6m8s
5kube-proxy 2 6m5s
6kubeadm-config 2 6m6s
7kubelet-config-1.19 1 6m6s

Conclusion

That’s it, a quick primer on what is going on under the hood of a Kubernetes setup. While Kubernetes does a ton of things and in the end is quite complex, the infrastructure is very modular and understandable. It’s possible to do a deep dive into any component you are interested it, if you so desire.

More articles from Mark Pollmann

GraphQL - From Beginner To Expert in 2020

GraphQL is growing fast. Created by Facebook in 2012 and released to the public in 2015 it has taken the world by storm. Companies using it…

July 6th, 2020 · 10 min read

Using NodeJS with TypeScript in 2020

Note: This article is aimed at Node developers looking into TypeScript and is planned as a living document. When I find something noteworthy…

May 15th, 2020 · 4 min read
© 2017–2021 Mark Pollmann
Link to $https://twitter.com/MarkPollmannLink to $https://github.com/MarkPollmannLink to $https://www.linkedin.com/in/mark-pollmann-961446132/