Lab 16.2. Detailed Steps with 2 CPs

dsfraser

I've completed this lab but with just two CP's, however when I shutdown one of the CP's the kube-apiserver works for a while and then stops/shutsdown. I know there's a "Quorum" that etchd has to have but not sure why a single etcd instance can't work on its own. Is there any documentation on this?
Error after one of the CP's is shutdown:
kubectl get pods The connection to the server master1:6443 was refused - did you specify the right host or port?

chrispokorni

Hi @dsfraser,

A key detail can be found on the High Availability -> Collocated Databases page:

join at least two more cp servers to the cluster

By following the recommendation to add two control plane nodes to an existing cluster, yields a cluster with three control plane nodes - the minimum number of control plane nodes to implement a highly available control plane.

What you have experienced with only two control plane nodes is normal behavior. It is the outcome of the RAFT algorithm leveraged by the etcd members to elect a new cluster leader in the case of the leader's failure. Essentially a two node control plane is no better than a single node control plane, therefore the recommendation of minimum three control plane nodes for quorum purposes.

Additional reads:

Options for HA topology:
https://kubernetes.io/docs/setup/production-environment/tools/kubeadm/ha-topology/

Etcd FAQ:
https://etcd.io/docs/v3.2/faq/

Raft consensus algorithm:
https://raft.github.io/

Regards,
-Chris