In this walkthrough we will setup Kubernetes on three minimalized virtualized Ubuntu 24.04 LTS Servers. It is up to you which virtualization application you use and is not part of this blog. It is important that these VM images are not running a container runtime like docker. All the described steps here are based on the documentation which can be find under Kubeadm production environment.

1. Container Runtime Setup

This first section is a preparation of container runtime for the control-plane node and also for the worker nodes. The second section is about to install kubeadm, kubelet, kubectl and is just necessary to install on the control-plane node and not on the worker nodes.

1.1 Repo preparation

Run the following command with root privileges.

sudo apt update && sudo apt upgrade -y

After finished this process install the following optional packages as well.

sudo apt install git vim jq inetutils-ping net-tools -y

1.2 Prepare containerd.conf

Now we have to setup a containerd.conf file with root privileges. Containerd is an open-source container runtime which is interacting directly to the operating system. For example docker engine sits on top of containerd and provides additional functionality.

sudo vi /etc/modules-load.d/containerd.conf

Now enter the following modules and save

overlay

br_netfilter

After containerd.conf file created with the modules successfully active these modules as sudo user like as follows.

sudo modprobe overlay

sudo modprobe br_netfilter

1.3 Sysctl parameters

Here we will create the necessary sysctl parameters for the Kubernetes cluster. These parameters will persist across reboots. Before we apply the sysctl params, we have to create the following file with root privileges.

sudo vi /etc/sysctl.d/99-kubernetes-cri.conf

Now enter the following sysctl params and save it

net.bridge.bridge-nf-call-iptables = 1

net.ipv4.ip_forward = 1

net.bridge.bridge-nf-call-ip6tables = 1

After you saved the file successfully you can apply the sysctl parameters without rebooting the whole like as follows

sudo sysctl --system

After running the command for applying the sysctl params you should see the following output.

* Applying /usr/lib/sysctl.d/10-apparmor.conf ...

* Applying /etc/sysctl.d/10-console-messages.conf ...

...

* Applying /etc/sysctl.d/99-kubernetes-cri.conf ...

...

* Applying /usr/lib/sysctl.d/99-protect-links.conf ...

* Applying /etc/sysctl.d/99-sysctl.conf ...

...

1.4 Install and setup containerd

Before installing and setting up kubeadm, kubelet and kubectl it is necessary to install containerd runtime binary from the github repository and configure this one.

Run now the following commands to get the latest containerd version and save into environment variable called CONTAINERD_VERSION.

CONTAINERD_VERSION=$(curl -s https://api.github.com/repos/containerd/containerd/releases/latest | jq -r '.tag_name')

~:$ echo $CONTAINERD_VERSION

~:$ v1.7.20

To download the correct containerd runtime binary find out the platform architecture first with the foolowing shell command. In this case we install and setup Kubernetes under a aarch64 platform architecture.

~:$ echo $(arch)

~:$ aarch64 // Use arm64 in the download url

Now we can create the binary download url with a recommended checksum and extract this artifact into the correct location.

BINARY_NAME=containerd-${CONTAINERD_VERSION#v}-linux-arm64.tar.gz

BINARY_URL=https://github.com/containerd/containerd/releases/download/${CONTAINERD_VERSION}/${BINARY_NAME}

curl -L -o ${BINARY_NAME} ${BINARY_URL}

curl -L -o ${BINARY_NAME}.sha256sum ${BINARY_URL}.sha256sum

sha256sum -c ${BINARY_NAME}.sha256sum

~:$ containerd-1.7.20-linux-arm64.tar.gz: OK // Should be OK

sudo tar xvf ${BINARY_NAME} -C /usr/local

Run now the following commands to the configure containerd

sudo mkdir -p /etc/containerd

sudo vi /etc/containerd/config.toml

Enter now the following plugins into config.toml file and save it.

version = 2

[plugins]

[plugins."io.containerd.grpc.v1.cri"]

[plugins."io.containerd.grpc.v1.cri".containerd]

discard_unpacked_layers = true

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes]

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]

runtime_type = "io.containerd.runc.v2"

[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc.options]

SystemdCgroup = true

1.5 Install runc and restart containerd

In this last step of the preparation of the containerd environment we will install now the runc and restart containerd for the next section to install kubeadm, kubelet and kubectl on the control-plane node. Runc is a CLI spawning and running containers on Unix according to the OCI specification.

Get the version of the opencontainer project from github api like as follows.

RUNC_VERSION=$(curl -s https://api.github.com/repos/opencontainers/runc/releases/latest | jq -r '.tag_name')

RUNC_URL=https://github.com/opencontainers/runc/releases/download/${RUNC_VERSION}/runc.arm64

curl -L -o runc.arm64 ${RUNC_URL}

sudo install -m 755 runc.arm64 /usr/local/sbin/runc

Before we can restart containerd we have to download the containerd.servcie. This service we can get also from the github repository. For this run now the following commands.

CONTAINERD_SERVICE=https://raw.githubusercontent.com/containerd/containerd/main/containerd.service

curl -L -o containerd.service ${CONTAINERD_SERVICE}

sudo mv containerd.service /usr/lib/systemd/system/

sudo systemctl daemon-reload

systemctl enable --now containerd

2. Install kubeadm, kubelet and kubectl

In this section we will install only on the control-plane node the Kubernetes tools to initialize the Kubernetes Cluster and connect to the worker nodes.

2.1 Install and Setup Kuberenetes

In this section we get the latest version of Kubernetes and extend the Ubuntu apt packages and downloading also the public signing key for the Kubernetes package repositories.

KUBE_VERSION=$(curl -s https://api.github.com/repos/kubernetes/kubernetes/releases/latest | jq -r '.tag_name')

Create now the k8s.conf file and use the following commands.

sudo vi /etc/modules-load.d/k8s.conf

br_netfilter

After you save the k8s.conf file successfully add the Kubernetes urls to the package manager list.

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

curl -fsSL https://pkgs.k8s.io/core:/stable:/${KUBE_VERSION}/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo "deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/${KUBE_VERSION}/deb/ /" | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt-get update

sudo apt-get install -y kubelet kubeadm kubectl

sudo apt-mark hold kubelet kubeadm kubectl

sudo swapoff -a

sudo sed -i 's/\/swap/#\/swap/' /etc/fstab

After installing kubeadm, kubelet and kubectl successfully we have to setup the iptabels bridging in the k8s.conf file. For this we have to create this file in the correct folder and enter the bridging setup.

sudo vi /etc/sysctl.d/k8s.conf

net.bridge.bridge-nf-call-ip6tables = 1

net.bridge.bridge-nf-call-iptables = 1

After saved this file restart the sysctl and configure the container runtime interface.

sudo sysctl --system

sudo crictl config --set runtime-endpoint=unix:///run/containerd/containerd.sock

Now the control-plane node is ready to initialize kubeadm and add the worker nodes. This will be explained in the next section.

3. Initialize kubeadm on the control-plane

After setup the container environment and installing the toolset for Kubernetes you should be able to initialize kubeadm. Run the following command to initialize kubeadm. This can take few minutes.

sudo kubeadm init

After the initialization is finished without any errors you should see the next steps to do like as follows.

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

mkdir -p $HOME/.kube

sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.

Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:

https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.211.55.12:6443 --token dek8zj.chkhm1l0y19pngie \

--discovery-token-ca-cert-hash sha256:4a28e45c9c4142f9585ad246e179cd8295848a59766ec2fd6c827f54067c9682

After you finished the steps to copy the .kube/config file you have to install in the last step a container networking. In this case we will install calico as follows.

kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml