Lab 3.1. Install Kubernetes - Cilium creating two operators

fasmy

Hi,

I almost completed the Kubernetes installation process. However, the final step failed it seems. I must have missed something, but I cannot figure out what.

Feel free to ask for config files and other info. I'm not sure what to share, there is so much.

Here is my issue: I have two cilium operators, instead of one. One of them is forever pending!

What I did:

• If followed the tutorial to the letter, and used the course file to config cilium: ./LFS458/SOLUTIONS/s_03/cilium-cni.yaml
• I did not modify the file in the slightest

Additional info:

kubectl version: 
    Client Version: v1.32.2
    Kustomize Version: v5.5.0
    Server Version: v1.32.2

.

kubeadm version
    kubeadm version: &version.Info{Major:"1", Minor:"32", GitVersion:"v1.32.2", GitCommit:"67a30c0adcf52bd3f56ff0893ce19966be12991f", GitTreeState:"clean", BuildDate:"2025-02-12T21:24:52Z", GoVersion:"go1.23.6", Compiler:"gc", Platform:"linux/amd64"}

.
kubelet --version
Kubernetes v1.32.2
.

fasmy

Also, I know how to use kubectl describe, and kubectl logs but I ma not really at east witht hose function, and I would love to get some tips for debugging and troubleshooting on my own, so I do not have to ask others for help.

chrispokorni

Hi @fasmy,

In order to troubleshoot the cilium plugin issue, you could start by assessing the state of the cluster. Beginning with the VMs, whether cloud or local, (required CPU, RAM, Disk), network interface and any firewalls filtering inbound traffic to the VMs, the required OS distribution and release. Having sufficient compute resources and all inbound traffic allowed to the VMs are critical. The next thing you could validate is the source of the course content - there seems to be a mismatch between this forum's course number (LFS258) and the course number prefixing your resources tarball (LFS458).

In addition, the installed Kubernetes components versions (v1.32.2) does not match the version recommended in the lab guide (installed v1.30.1, upgraded to v1.31.1).

Have you tried deleting the cilium-operator controller of the pending pod? Or perhaps rebooting the VMs hosting the cluster?

What is the status of your cluster nodes? Please provide the output here, in an extended format to show their IP addresses, OS, kernel, and container runtime. Are all the cluster pods in a running state? Please provide the output of the pods listing, in an extended format to show their IP addresses and assigned nodes.

These outputs could help to determine if there are possible IP range conflicts. Are there any other discrepancies between the cluster configuration and the lab guide?

Regards,
-Chris

fasmy

Thank you for your answer Chris!

I'll follow your recommendations and keep you up-to-date. I just did a full reinstall today. I'm setting this up on my own VMs.

fasmy

Problem solved ! It works!

I cannot verify for sure what caused the issue, but I have a few guesses:

1. Lack of resources: I assigned 8GB of storage to my virtual machine (I like them lean and mean). However, newer versions of ubuntu-server take much, much more space than I remembered. 98% of my storage was filled. I assigned 12GB, and it now sits around a safe 59%:
   

2. Network configs: In my Virtual Machines, I use two connections : A NAT network, so that my virtual machines can have an easy access to the internet and so that I can connect to my VMs with ssh, to simulate a real-world environment by remote logging. And a Host-only Adapter network, which links all my VMs, and is used by Kubernetes nodes to communicate between one another.

   Question: Why not just use the Nat Network? Because I would have to setup port forwarding for VM-to-VM communication, which is messy. Just go with the easy option.
   NAT: VM ↔internet
   Host-only: VM ↔ VM

   In short: here are the three (3) different places where you want to set your IP, so that Kubernetes doesn't miss it:

   • In /etc/hosts, set your host to the desired IP (my machine is called cp, for control plane):
     
     The Host-only Adapter network I setup in Virtual Box is 192.168.100.0/24, and my cp Node is set as 192.168.100.12.
     In the above example, both the name k8scp and cp are set to 192.168.100.12. Before I changed CP, it was, by default, set to localhost (127.0.0.1). Which as you might guess is a loopback IP going nowhere.

   • When initializing kubernetes, I used a config file for the EndPoints, I had to inform the IP twice:
     In InitConfig, where we have to write the IP (not a hostname).
     
     and still in the same file, the ClusterConfiguration, which used the host, k8scp, I defined earlier in /etc/hosts. Technically a simple IP would have done the job, but I think we'll later use it in the exercises (for load balancing?)