Cillium in multi-interface deployments

stralex

I had issues when trying to join a node into the cluster. My on prem-VMs have dedicated interfaces to access internet which also have the same IP on every host, which is not a problem itself, but rather that interface was picked by kubeadm, so the configuration that I have ended up with:

---
kind: InitConfiguration
apiVersion: kubeadm.k8s.io/v1beta4
nodeRegistration:
  criSocket: unix:///run/containerd/containerd.sock
localAPIEndpoint:
  advertiseAddress: 10.0.0.1
  bindPort: 6443

On the hosts (Ubuntu LTS):

echo 'KUBELET_EXTRA_ARGS="--node-ip 10.0.0.1"' > /etc/default/kubelet
systemctl daemon-reload
systemctl restart kubelet.service

10.0.0.1 is internal L2 network. Using ontrolPlaneEndpoint: was simply not enough, kubeadm kept picking the first interface, with lowest interrface id

Additionally I reduced a number of operator replicas in cilium-cni.yaml to 1, can be scaled out later, if there is a case or a lab task.

chrispokorni

Hi @stralex,

The course included lab guide does not explore advanced Cilium installation and configuration methods. It deploys the CNI plugin assuming a single interface per VM (cloud and local VMs), where simply declaring the control plane endpoint as an alias is sufficient for all the lab exercises.

To tailor both Kubernetes and Cilium to your virtual infrastructure please consult the official documentations of Kubernetes and Cilium respectively, to ensure both tools are properly configured to provide you with a working lab environment.

Regards,
-Chris

stralex

Hi Chris, I just wanted to spread a knowledge if someone else runs into the same topic.