One cool thing about Kubernetes is that the infrastructure is transparently handled by Kubernetes objects themselves. If you want to see how everything works together, you can just inspect the pods, services and other objects themselves.
Disclaimer: This setup is valid for Kubernetes in Docker Desktop on Mac. In a managed environment, or in your setup, the setup might be different and components are not necessarily deployed as simple pods.
Let’s have a first look. I’m using the built-in Kubernetes for Docker Desktop on a Mac here (and use
k as an alias to
1╰$ k get all2NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE3service/kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 39d
Well, not a lot going on here. Is that all there is?
Of course not, the real meat is in other namespaces. Anyway, let’s first inspect this service we found:
1╰$ k describe service/kubernetes2Name: kubernetes3Namespace: default4Labels: component=apiserver5 provider=kubernetes6Annotations: <none>7Selector: <none>8Type: ClusterIP9IP: 10.96.0.110Port: https 443/TCP11TargetPort: 6443/TCP12Endpoints: 192.168.65.3:644313Session Affinity: None14Events: <none>
What is it doing? It’s actually just forwarding requests to the Kubernetes API server (which we will meet soon).
Okay, but where is the rest?
Let’s check out the other namespaces.
1╰$ k get namespaces2NAME STATUS AGE3default Active 39d4kube-node-lease Active 39d5kube-public Active 39d6kube-system Active 39d
default namespace we already know.
kube-node-lease is relatively new (starting from version 1.14) and helps in determining the availability of a node via heartbeats. We’ll ignore it for now.
kube-public is a namespace that exists if the cluster was created with kudeadm. It contains a single, lonely configmap called
config-info and aids in discovery for other clients. Read more about the discovery API if you are interested.
kube-system is the most interesting namespace for us. Let’s dive in:
The kube-system namespace
Let’s do a quick lookup:
1╰$ k get all -n kube-system2NAME READY STATUS RESTARTS AGE3pod/coredns-f9fd979d6-dmkxf 1/1 Running 0 4m57s4pod/coredns-f9fd979d6-vrd8t 1/1 Running 0 4m57s5pod/etcd-docker-desktop 1/1 Running 0 3m43s6pod/kube-apiserver-docker-desktop 1/1 Running 0 4m7s7pod/kube-controller-manager-docker-desktop 1/1 Running 0 3m45s8pod/kube-proxy-s4jfp 1/1 Running 0 4m57s9pod/kube-scheduler-docker-desktop 1/1 Running 0 3m51s10pod/storage-provisioner 1/1 Running 0 3m43s11pod/vpnkit-controller 1/1 Running 0 3m42s1213NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE14service/kube-dns ClusterIP 10.96.0.10 <none> 53/UDP,53/TCP,9153/TCP 5m4s1516NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE17daemonset.apps/kube-proxy 1 1 1 1 1 kubernetes.io/os=linux 5m4s1819NAME READY UP-TO-DATE AVAILABLE AGE20deployment.apps/coredns 2/2 2 2 5m4s2122NAME DESIRED CURRENT READY AGE23replicaset.apps/coredns-f9fd979d6 2 2 2 4m57s
By the way:
kubectl get all does not really get all resource types, only the most important. Check out ketall if you want to get everything.
A lot more going on here! We can see see a deployment (plus replicaset and its pods) of coredns and a service
kube-dns, which, unsurprisingly, handle DNS in the cluster. I will skip this for now and write a separate post about DNS in Kubernetes.
So, what is left? I count 7 pods, let’s go through them one by one.
pod/vpnkit-controller is there to interact with my host’s VPN configuration. Not very interesting for us right now.
etcd is a key-value store that stores information concerning the cluster, like pods, nodes, roles, configs, secrets. Think of it as its database. Every piece of information you get via
kubectl get comes from etcd and every update you make goes into etcd. As Kubernetes is a distributed system the database needs to be distributed, too, and etcd is the perfect solution for it.
kube-apiserver is quite self-explanatory. This is the endpoint with which other components interact with REST operations if they want to lookup or change state. If you run
kubectl you are interacting with the apiserver.
It’s also the only component that interacts with etcd.
kube-controller-manager. Controllers (e.g. Node controllers, deployment controllers, namespace controllers) supervise the state of the components they are in charge of. Node controllers, for example, check the health of nodes every five seconds to see if they are still reachable. If they don’t respond after 40 seconds it is marked unreachable. If it doesn’t come up again after 5 minutes it evicts the pods scheduled on it to other nodes.
All these controllers run in the process of the controller-manager.
kube-proxy runs on every node and handles service networking. As a service is not a real object but just lives in memory, somebody needs to handle the routing of services and kube-proxy does just this. One way it does this is with iptable rules. On every node it creates entries for every service and forwards traffic to the IP of the actual pod selecting the service.
kube-scheduler decides which pod gets scheduled on which node. Any pod might have specific resource requirements to run.
In the first phase the scheduler filters out the nodes which can’t satisfy the resource requirements or are tainted or don’t tolerate this specific pod.
Then it ranks the remaining nodes by a priority function. If the pod is CPU-intensive the node with more free CPU capacity gets a higher rank and so one. The one with the best ranking wins the pod.
storage-provisioner handles storage specific to the platform being used. In my case this pod is using the
docker/desktop-storage-provisioner image specific for Mac’s Docker Desktop.
By the way, you can see more of the configuration for some of these by looking at the config maps in the kube-system namespace
1╰$ k get cm -n kube-system2NAME DATA AGE3coredns 1 6m5s4extension-apiserver-authentication 6 6m8s5kube-proxy 2 6m5s6kubeadm-config 2 6m6s7kubelet-config-1.19 1 6m6s
That’s it, a quick primer on what is going on under the hood of a Kubernetes setup. While Kubernetes does a ton of things and in the end is quite complex, the infrastructure is very modular and understandable. It’s possible to do a deep dive into any component you are interested it, if you so desire.