CKS Lab - failing to install Cluster

mihaitanasescu

Hi all,

I'm having 2 x VMs on Azure, Ubuntu 20, disabled AppArmor, no other iptables settings exist.
Node1 VNET IP CIDR 10.0.0.0/16
Node2 VNET IP CIDR 10.1.0.0/16
VNET Peering so the Nodes do see and reach each other + tested.

I installed the master node via the k8scp.sh script and all seems good:

~$ kubectl get node
NAME   STATUS     ROLES           AGE   VERSION
k8s1   Ready      control-plane   33m   v1.30.1
k8s2   NotReady   <none>          19m   v1.30.1

I installed the Worker Node and joined it and status flaps between Ready / NotReady.
Logs show:

Jul 22 21:19:05 K8S2 kernel: Dead loop on virtual device cilium_vxlan, fix it urgently!
Jul 22 21:19:05 K8S2 kernel: Dead loop on virtual device cilium_vxlan, fix it urgently!

I have a feeling this related to the POD network being with 10 and the main interfaces of the VMs being also with 10...

Am I right?

mihaitanasescu

Changed IPs so far it seems stable.
That error did look like the main CIDR of eth0 trying to be routed over VXLAN Cilium as it had a little overlap with the pod network.

chrispokorni

Hi @mihaitanasescu,

You are correct in your findings: distinct (non-overlapping) subnets are recommended for the cluster. This means that the Node IP subnet (managed by cloud or local hypervisor), the Services subnet (default 10.96.0.0/12), and the Pod IP subnet (managed by the CNI plugin) should not overlap. If the subnets overlap, then iptables and ebpf may cause packets to be sent to an incorrect destination.

Regards,
-Chris