Kubernetes Installation on Openstack VM’s with Kubeadm

This post demonstrates installing kubernetes on Openstack hosted VM’s by using kubeadm.

We assumed to create a blank Openstack project for this installation.

Debian 11 was used as VM’s OS.

Step 1: Create a private network

Click “Create Network ” button after following “Network -> Networks” menus.

Write the network name which is suits for your personal pleasure.

Naming the Network

We should create a subnet for IP allocation to instances(future Kubernetes nodes).

Determine a Subnet for the Network

Click “next” and complete the wizard.

Step 2: Create Router

For accessing instances from the internet we need to connect instances to the public network by a router.

Follow the “Network-> Routers” path and click “Create Router” button.

Create Router

Name the router and select the public network from “External Network” list then complete the wizard.

Click the router name after it was created.

Router Details

Click “Interfaces” tab and then click “Add Interface” button on opened page.

Add Interface to Router

Select a suitable private subnet that is used by our instances then complete the wizard.

After that process, you should see a router menu like below.

Router Menu

Step 3: Create Instances

We are ready to create Instances.

Open the “Instances” menu and click “Launch Instance” button.

Name the instance on the opening screen.

Launch Instance Screen

Select Flavor. You should select a flavor that has more than 2-gigabyte memory(4GB is recommended for Kubernetes master node).

Select Flavor

Select a network that is created by you for use as an internal network.

Select Network

Attach key pair for connecting instances.(you are on your own)

Do this process by two times if you are planning the separate installation of master and worker nodes.

Step 4: Allocate and Associate Floating IP’s

We need to allocate and associate floating ip addresses to our instances for accessing them from the outside of the network.

Instances Screen

Click drop-down list which is placed right side of the instance then select “Associate Floating IP”

Associate Floating IP

If you created a floating IP address before you can select an existing one and associate it to a selected instance. If you don’t have any floating IP’s wizard will help you to create a floating IP and after all, you could able to associate this floating IP to the instance.

After this process, you should see a screen like below.

Floating IP Associated Instances

Step 5: Modify Security Groups

We have to allow ssh traffic for accessing the instances from outside by using security groups.

Follow the “Network->Security Groups” path then click the “Manage Rules” button on the security group which is associated with our instances(if you didn’t make any changes before it’s probably named as “default”).

Security Group Rule List

Select SSH from list and complete the wizard.

Step 6: Connect and Prepare Instances

We need to prepare created instances for Kubernetes Installation.

1) For letting iptables for see bridged traffic we have to load“br-netfilter” module. This can be done by executingsudo modprobe br_netfilter

After successfully installing the module you have to add parameters by executing the following commands to correctly see bridged traffic.

$ cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

$ cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
$ sudo sysctl --system

2) Disable cgroup support on the system for Docker Compatibility

We have to disable cgroup support on instances because they may producing compatibility errors.

You must add following kernel command line parameters to “/etc/default/grub” file

systemd.unified_cgroup_hierarchy=0
systemd.legacy_systemd_cgroup_controller=0

Here is the sample grub file. I don’t recommend replacing it by yours. You should add these parameters by your hand.

Grub File Sample

After that step, you have to update grub by using the following command.

$ sudo update-grub

Then reboot the instance.

Important Note: You have to do this configuration for all of your instances.

3) We need to install Docker Environment as container runtime which will be used by Kubernetes.

a) Install requirements:

$ sudo apt-get update

$ sudo apt-get install \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

b) Get and add GPG key of Docker repository:

$ curl -fsSL https://download.docker.com/linux/debian/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

c) Add Docker Repository:

$ echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/debian \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

d) Install Docker Environment:

$ sudo apt-get update
$ sudo apt-get install docker-ce docker-ce-cli containerd.io

Step 7: Install Kubernetes

If you didn’t get any errors until this step you are ready to install Kubernetes.

1) Install requirements:

$ sudo apt-get update
$ sudo apt-get install -y apt-transport-https ca-certificates curl

2) Get and add GPG key of Google Cloud repository:

$ sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

3) Add the Kubernetes apt repository:

$ echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

4) Install kubelet, kubeadm and kubectl:

$ sudo apt-get update
$ sudo apt-get install -y kubelet kubeadm kubectl
$ sudo apt-mark hold kubelet kubeadm kubectl

Important Note: You have to do the same processes until here on all of your instances.

5) Create kubeadm-config.yaml:

We will use kubeadm-config.yaml for configure our Kubernetes cluster. You can create and edit the file by using the following command.

$ sudo nano kubeadm-config.yaml

Here is the content of our kubeadm-config.yaml file.

# kubeadm-config.yaml
kind: ClusterConfiguration
apiVersion: kubeadm.k8s.io/v1beta3
kubernetesVersion: v1.23.1
networking:
podSubnet: "10.244.0.0/16" # --pod-network-cidr for using with flannel
---
kind: KubeletConfiguration
apiVersion: kubelet.config.k8s.io/v1beta1
cgroupDriver: cgroupfs

Write changes to the file and close the editor. (Press F3->Enter->F2)

6) Apply kubeadm-config.yaml:

We are ready to configure Kubernetes Master node.You can start configuration by following command.

$ kubeadm init --config kubeadm-config.yaml

After successfully executing of this command the kubeadm tool will prompt you an output that looks like below:

Kubeadm Output

Here is a sample output pattern of kubeadm command:

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a Pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  /docs/concepts/cluster-administration/addons/

You can now join any number of machines by running the following on each node
as root:

  kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

You have to store last line of that output for adding worker nodes to your cluster.

kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

7) Creating Pod Network:

Kubernetes needs add-ons to ensure the connection between pods. This add-on is named CNI tools. We are going to use flannel as CNI tool. You can add this tool to your cluster by using the following command:

$ kubectl apply -f https://raw.githubusercontent.com/flannel-io/flannel/master/Documentation/kube-flannel.yml

8) Adding worker nodes to the cluster:

If you want to add worker nodes to your cluster you can use the command over worker nodes which is recommended to store on step 6.

kubeadm join <control-plane-host>:<control-plane-port> --token <token> --discovery-token-ca-cert-hash sha256:<hash>

Here is the sample output of the command:

root@kubernetes-worker:/home/debian# kubeadm join 192.168.12.9:6443 --token ######## \
        --discovery-token-ca-cert-hash sha256:##########
[preflight] Running pre-flight checks
[preflight] Reading configuration from the cluster...
[preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Starting the kubelet
[kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.
Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

Here is the sample output of “kubectl get nodes”:

Sample Output of get nodes

You can see kube-system pods by using following command:

$ kubectl get pods --all-namespaces

Here is the sample output:

Kubernetes Pods

9) Copy Kubernetes configuration for regular user:

$  mkdir -p $HOME/.kube
$  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
$  sudo chown $(id -u):$(id -g) $HOME/.kube/config

If you want to access and modify your Kubernetes cluster from a different location you can use created configuration files on your control plane computer.

Congratulations

You are installed Kubernetes on your VM’s which is hosted by Openstack. You can enjoy yourself with your cluster now :)