How to Build Highly Available Kubernetes Clusters on AWS | K8s Series By M. Sharma (On-Demand)
In this K8-Series, I’ll be making different Kubernetes #k8s topics such as
https://www.youtube.com/watch?v=Y0AsTseEMb8&t
- Getting Started with K8s
- Cluster Management
- Pods & Container Allocation
- K8s Object Management
- K8s Deployments
- K8s Networking & Services
- K8s Storage & so on*
*new topics add-ons as we go farther in this journey
This interactive article assumes you have some basic knowledge of Containerization, how to operate with AWS as Cloud Providers (or other cloud providers such as GCP, Azure etc.), how to launch your nodes/servers and have some basic understanding of Linux commands.
Support the author: Mohit Sharma
You can use Wise for sending money abroad: https://wise.prf.hn/l/KN8wJwq
So, what is Kubernetes?
Kubernetes, also known as K8s, is an open-source system for automating deployment, scaling, and management of containerized applications (according to Kubernetes documentation).
Whenever you deploy Kubernetes, you get a cluster. A K8s cluster consists of a set of worker machines, called nodes (in our case it’s ec2 instances), that run containerized applications*. Every cluster has at least one worker node.
*A container is a standard unit of software that packages up code and all its dependencies so the application runs quickly and reliably from one computing environment to another.
The worker node(s) host the Pods that are the components of the application workload. The control plane manages the worker nodes and the Pods in the cluster. In production environments, the control plane usually runs across multiple computers and a cluster usually runs multiple nodes, providing fault-tolerance and high availability.
Why do we need to build a High-Availability Cluster?
To get our services running up all the times, with minimum or no downtime, high scalability, resilient architecture with high availability for our company products/services.
The control plane’s components make global decisions about the cluster (for example, scheduling), as well as detecting and responding to cluster events (for example, starting up a new pod when a deployment’s replicas the field is unsatisfied).
Control plane components can be run on any machine in the cluster. However, for simplicity, set up scripts typically start all control plane components on the same machine, and do not run user containers on this machine.
In later series, we will see how to setting up a highly available Kubernetes cluster using kubeadm: (where we work with at-least 2–3 Control planes and 5–10 Worker Nodes)
1. With stacked control plane nodes. This approach requires less infrastructure. The etcd members and control plane nodes are co-located.
2. With an external etcd cluster. This approach requires more infrastructure. The control plane nodes and etcd members are separated.
Note — This article doesn’t come under the free-tier of any Cloud Provider such as AWS, Azure, or GCP. So, keep the cost in mind.
Prerequisite Steps:-
- Sign-up or Sign-in to your AWS account
- Launch a minimum of three EC2 Instances in any AWS region or whichever closest to you (* All instances must be under the same region, you can’t launch instances in different regions).
- Minimum Configuration requirements for Control Plane (while installing kubeadm)
One or more machines running one of:
- Ubuntu 16.04+
- Debian 9+
- CentOS 7+
- Red Hat Enterprise Linux (RHEL) 7+
- Fedora 25+
- HypriotOS v1.0.1+
- Flatcar Container Linux (tested with 2512.3.0)
- 2 GB or more of RAM per machine (any less will leave little room for your apps).
- 2 CPUs or more.
- Full network connectivity between all machines in the cluster (public or private network is fine).
- Unique host-name, MAC address, and product_uuid for every node. See here for more details.
- Certain ports are open on your machines.
- Swap disabled. You MUST disable swap in order for the kubelet to work properly
4. Create two Security Groups K8s-Master-Node-SG & K8s-Worker-Node-SG
Create Inbound Rules on your Security Groups.
On K8s-Master-Node-SG, open all these ports for Control Plane (Note- May required to open more ports to work accordingly, check your cloud providers security group details)
On K8s-Worker-Node-SG open all these ports for Worker Nodes (Note- May required to open more ports to work accordingly, check your cloud providers security group details)
† Default port range for NodePort Services.
Any port numbers marked with * are overridable, so you will need to ensure any custom ports you provide are also open.
Although etcd ports are included in control-plane nodes, you can also host your own etcd cluster externally or on custom ports.
5. Connect to your terminals via SSH
Once, you connected via SSH to your instances. Follow the below steps:-
Steps required to Build Highly Available Kubernetes Cluster
- You need to install a container runtime into each node in the cluster so that Pods can run there.
Currently, Kubernetes support three CRI (Container Runtime Interface) on Linux:
2. In this article, I will show you with Containerd as CRI runtime and all the steps required to run Containerd
On all nodes (instances), set up containerd. You will need to load some kernel modules and modify some system settings as part of this process.
(Notes- these steps are only applicable to Ubuntu 18.04/20.4 LTS)
cat <<EOF | sudo tee /etc/modules-load.d/containerd.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilterSimilarly, we create Kubernetes-cri.conf file
# Setup required sysctl params, these persist across reboots.cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables = 1
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
EOF
# Apply sysctl params without reboot
sudo sysctl --system
Let’s installed the containerd
# (Install containerd)
sudo apt-get update && sudo apt-get install -y containerdConfigure & Restart the containerd
# Configure containerd
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml# Restart containerd
sudo systemctl restart containerd
On all nodes, disable swap (necessary step, in order to work kubelet properly)
sudo swapoff -a
sudo sed -i ‘/ swap / s/^\(.*\)$/#\1/g’ /etc/fstabBy default, Kubernetes uses the Container Runtime Interface (CRI) to interface with your chosen container runtime such as Containerd, CRI-O or Docker.
If you don’t specify a runtime, kubeadm automatically tries to detect an installed container runtime by scanning through a list of well known Unix domain sockets. The following table lists container runtimes and their associated socket paths:
Runtime Path to Unix domain socket
Docker
/var/run/docker.sockContainerd
/run/containerd/containerd.sockCRI-O
/var/run/crio/crio.sock
If both Docker and containerd are detected, Docker takes precedence. This is needed because Docker 18.09 ships with containerd and both are detectable even if you only installed Docker. If any other two or more runtimes are detected, kubeadm exits with an error.
3. Installing kubeadm, kubelet and kubectl
You will install these packages on all of your ec2-instances:
kubeadm: the command to bootstrap the cluster.kubelet: the component that runs on all of the machines in your cluster and does things like starting pods and containers.kubectl: the command line util to talk to your cluster
# update and installing packages
sudo apt-get update && sudo apt-get install -y apt-transport-https curl# downloading the gpg key
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -# adding source kubernetes.list
cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF# updating the packages and installing kubeadm,kubectl,kubelet (v 1.20.1)sudo apt-get update
sudo apt-get install -y kubelet=1.20.1-00 kubeadm=1.20.1-00 kubectl=1.20.1-00 # hold the updates
sudo apt-mark hold kubelet kubeadm kubectl
4. On the control plane node only, initialize the cluster and set up kubectl access.
Kubeadm is a tool built to provide kubeadm init and kubeadm join as best-practice "fast paths" for creating Kubernetes clusters.
kubeadm performs the actions necessary to get a minimum viable cluster up and running. By design, it cares only about bootstrapping, not about provisioning machines. Likewise, installing various nice-to-have add-ons, like the Kubernetes Dashboard, monitoring solutions etc.
# using kubeadm init we initiating the pod on this network cidr # range
sudo kubeadm init — pod-network-cidr 192.168.0.0/16"""
if above command runs successfully, you will this see the below command output in your CLI, which is creating a .kube directory and copying the files and change the, permission
"""mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
5. Verify the cluster is working.
kubectl versionIf you see the version which we installed above, congrats you have installed it successfully.
6. Install the Calico network add-on.
kubectl apply -f https://docs.projectcalico.org/manifests/calico.yaml# Check the calico-related kube-system Pods to verify that # everything is working finekubectl get pods -n kube-system (it's sometimes takes 1-2 mins to show and ready the pods)"""
Using the kubeadm join command (this command is also printed during kubeadm init)
"""kubeadm token create — print-join-command"""
Copy the join command from the control plane node. Run it on your Worker nodes with Root permission (i.e. sudo), if everything set up correctly you will it's joined successfully
"""
On the control plane node, you can verify all nodes in your cluster are ready or not.
- Note, sometimes it takes up to a few minutes for all of the nodes to
enter the READY state.
kubectl get nodes -ACongratulations!! For reaching this far with me. Now, you have created your own Kubernetes Cluster. This is your first step in learning how to build HA K8s Cluster on AWS.
Now, in the next coming weeks, we will be going to do some intermediate level of work using Kubernetes. Till then, try to understand each concept. Let me know if you have any more questions.
For the latest updates connect with me on -
LinkedIn | Twitter | GitHub |Facebook | Instagram | YouTube
👋 Join FAUN today and receive similar stories each week in your inbox! ️ Get your weekly dose of the must-read tech stories, news, and tutorials.
Follow us on Twitter 🐦 and Facebook 👥 and Instagram 📷 and join our Facebook and Linkedin Groups 💬
