Lab 16.2 HA multiple master nodes cluster when using cri-o steps??

Hi @josephkwong,

Both chapters 3 and 16 provide detailed steps to set up the first control-plane node, the worker, then the additional two control-plane nodes and the haproxy. The cri-o installation steps can be found in chapter 3 as well, and referenced whenever necessary for the worker, second and third control-plane nodes. As cri-o installation is much more complex than docker installation, extra care must be taken to ensure all install and config steps are properly executed.

Regards,
-Chris

Hi @josephkwong,

What podNetwork is used by Calico and the kubead-crio.yaml config file?
What IP range is used by the hypervisor when provisioning VMs?

If the podSubnet overlaps the IP range of the VMs, cluster internal traffic will be impacted, causing communication issues between nodes, control-plane pods and client workload pods.

Regards,
-Chris

Hi @josephkwong,

As I suspected, your network subnets overlap, causing major issues with your cluster.
First I would recommend re-building your cluster with distinct networks for nodes and pods - try 10.200.0.0/16 as VM/node subnet managed by the hypervisor's DHCP server.

Second, I noticed the kubeadm join command uses an IP address, as the advertised control-plane endpoint. That makes the HA config impossible to achieve. In order to fix it, edit the kubeadm-crio.yaml file as follows:

1 - for the InitConfiguration resource update the nodeRegistration.name to be cp or the HOSTNAME of your control-plane node.
2 - for the ClusterConfiguration resource add a new property controlPlaneEndpoint: "k8scp:6443" and double-check the desired value of kubernetesVersion:. After the init completion, the suggested join command should display kubeadm join k8scp:6443 --token abcdef...

As you initialize the cluster and join the first worker node, edit the /etc/hosts files as such:
a) on the control plane node add its Private IP with k8scp alias.
Example: 10.200.0.5 k8scp
b) on the worker node add the Private IP of the control plane node with k8scp alias, same entry as (a).
Example: 10.200.0.5 k8scp

Later, for HA configuration these aliases will be replaced with the ha-proxy Private IP, as instructed in the lab guide.

apiVersion: kubeadm.k8s.io/v1beta2
...
kind: InitConfiguration
localAPIEndpoint:
  bindPort: 6443
nodeRegistration:
  criSocket: unix:///var/run/crio/crio.sock
  name: k8scp              # <-- EDIT this line per instructions above
  taints: null
---
...
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
controlPlaneEndpoint: "k8scp:6443"        # <-- ADD this line
...
kind: ClusterConfiguration
kubernetesVersion: 1.22.1           # < -- EDIT to 1.21.1 if planning a cluster UPGRADE after init + join
networking:
...
<leave unchanged the rest of the file>

Regards,
-Chris

Hi @maybel,

Yes, the three new instances (2 control plane nodes and 1 haproxy instance) are on the same network as the first two instances (1 control plane and 1 worker nodes).

Regards,
-Chris