Lab 3.3 - coredns CrashLoopBackoff
Following hardware issues I had to reinstall Kubernetes master and workers on another PC. I'm running a Ubuntu 20.04 based host with QEMU / kvm Ubuntu 18.04 server guests: master, worker1 to worker4.
Things I did in addition to the lab tutorial: Comment out swap creation in /etc/fstab. Networking is done by NetworkManager using static IP. The master also runs a bind9 DNS server (see further down).
All VMs are connected to a bridged network bridge0. ufw firewall is disabled on the VMs and the host.
Here is the output of
kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-578894d4cd-mrj4l 1/1 Running 4 25h kube-system calico-node-9vsvc 0/1 Init:CrashLoopBackOff 7 25h kube-system calico-node-g9q9x 0/1 Running 4 134m kube-system calico-node-knppq 0/1 Completed 2 93m kube-system calico-node-wpfzq 1/1 Running 4 24h kube-system coredns-66bff467f8-gqgt9 0/1 Completed 0 57m kube-system coredns-66bff467f8-qnsjk 0/1 CrashLoopBackOff 11 56m kube-system etcd-master 1/1 Running 4 25h kube-system kube-apiserver-master 1/1 Running 6 25h kube-system kube-controller-manager-master 1/1 Running 7 25h kube-system kube-proxy-8wshb 1/1 Running 4 134m kube-system kube-proxy-gxnjw 0/1 Error 2 93m kube-system kube-proxy-hr92t 1/1 Running 4 24h kube-system kube-proxy-z8cx6 1/1 Running 4 25h kube-system kube-scheduler-master 1/1 Running 7 25h
I now disabled the ufw firewall on the host and deleted the calico-node... nodes. Then I deleted the coredns-... nodes and this is the result:
kubectl get pods --all-namespaces NAMESPACE NAME READY STATUS RESTARTS AGE kube-system calico-kube-controllers-578894d4cd-mrj4l 1/1 Running 4 25h kube-system calico-node-knppq 1/1 Running 4 110m kube-system calico-node-kt5lh 1/1 Running 0 2m25s kube-system calico-node-wpfzq 1/1 Running 4 24h kube-system calico-node-z8h2t 1/1 Running 0 107s kube-system coredns-66bff467f8-9sjgq 0/1 CrashLoopBackOff 1 16s kube-system coredns-66bff467f8-hfk5b 0/1 Running 0 33s
I removed the bind9 DNS server on master but this led to other problems, among others it sometimes resolves names, at other times not. Right now name resolving doesn't work, though I tried to reverse the steps and have systemd-resolve up and running.
I guess I will be reinstalling the host, then the VMs and see if that solves the issues. I'm afraid the bind9 server on the master VM didn't help.
The other possible issue could be libvirt networking. I had manually configured a bridged network which usually works fine when editing the xml guest config files to enable bridged networking. Next time I will try to configure the bridge within virt-manager and see if it makes a difference.
Any suggestions as to the above CrashLoopBackoff errors are welcome. Perhaps I'm looking in the wrong place altogether.