hi @chrispokorni ,
thank you for answering.

adjust subnets

After applying the following reset procedure, i applied the subnets you suggested (10.200.0.0/16) both on cilium-cni and kubeadm-config.
but no changes in the result .

new kubeadm command

hence, inspired by similar thread i launched again the reset procedure and used the command:

kubeadm init --pod-network-cidr 192.168.0.0/16 --node-name k8scp --upload-certs

with that i been able to complete the cluster initialization and the laboratory .

I'm attaching below the prompt collected during the troubleshooting, hopefully gonna be useful for somebody else.

If you (or somebody else) wanna spent two words for the reset procedure, would be really appreciable.

thank you for the support.

see you next problem.

Reset procedure

sudo kubeadm reset --force
sudo rm -rf /etc/cni/net.d
sudo rm -rf /var/lib/etcd
sudo rm -rf /var/lib/kubelet/*
sudo rm -rf /etc/kubernetes/*
rm -rf $HOME/.kube
sudo rm -rf /etc/cni/net.d
sudo systemctl stop kubelet
sudo systemctl stop containerd
sudo iptables -F
sudo ipvsadm --clear
sudo apt-get purge kubeadm kubectl kubelet kubernetes-cni kube*
sudo apt-get autoremove
sudo ctr containers list | awk '{print $1}' | xargs -r sudo ctr containers delete
sudo ctr snapshots list | awk '{print $1}' | xargs -r sudo ctr snapshots remove
sudo iptables -F && sudo iptables -X
sudo iptables -t nat -F && sudo iptables -t nat -X
sudo iptables -t raw -F && sudo iptables -t raw -X
sudo iptables -t mangle -F && sudo iptables -t mangle -X
sudo rm -rf /var/lib/docker /etc/docker /var/run/docker.sock
sudo rm -rf /var/lib/containerd
sudo rm -rf /run/containerd/containerd.sock
sudo rm -rf /run/docker.sock
rm -rf ~/.kube
rm -rf ~/.docker

prompts from subnets adjusting

kubeadm-config.yaml

substitute "10.200.0.0/16" instead of "192.168.0.0/16"
in /root/kubeadm-config.yaml

apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: 1.32.1 # <-- Use the word stable for newest version
controlPlaneEndpoint: "k8scp:6443" #<-- Use the alias we put in /etc/hosts not >
networking:
 podSubnet: 10.200.0.0/16 #192.168.0.0/16

cilium-cni.yaml

substitute "10.200.0.0/16" instead of "192.168.0.0/16"
in /home/luigi/LFS258/SOLUTIONS/s_03/cilium-cni.yaml

  hubble-tls-key-file: /var/lib/cilium/tls/hubble/server.key
  hubble-tls-client-ca-files: /var/lib/cilium/tls/hubble/client-ca.crt
  ipam: "cluster-pool"
  ipam-cilium-node-update-rate: "15s"
  cluster-pool-ipv4-cidr: "10.200.0.0/16" #instead of "192.168.0.0/16"  
  cluster-pool-ipv4-mask-size: "24"
  egress-gateway-reconciliation-trigger-interval: "1s"
  enable-vtep: "false"

Good morning @chrispokorni ,

Actually enrolled in LFS258 Kubernetes Fundamentals,

I'm running anything on a raspberry pi 5 (16GB ram + 512 GB ssd) with ubuntu on it.

Thank you for the support

Hi @chrispokorni ,
Resume of today's troubleshooting in the intent of grow the cluster (LAB3.2):
at new power on of the machine i found:

• kubelet down
• Api-server down
• etcd down

after some troubleshooting, i found the swap on.
so, resettled swapoff -a, now anything (etcd/api-server/kubelet) on and ready to work.

i been learning and reading all the commands used for that troubleshooting.
will continue tomorrow with the lab.

Will follow some prompt for checks:

hostname

root@luigispi:~# hostname
luigispi

Node up

luigi@luigispi:~$ sudo kubectl get nodes
NAME    STATUS   ROLES           AGE   VERSION
k8scp   Ready    control-plane   26h   v1.32.1

checking yaml files

cilium-cni.yaml

[...]
  ipam: "cluster-pool"
  ipam-cilium-node-update-rate: "15s"
  cluster-pool-ipv4-cidr: "10.200.0.0/16" #instead of "192.168.0.0/16"
[...]

kubeadm-config.yaml

root@luigispi:~# cat kubeadm-config.yaml 
apiVersion: kubeadm.k8s.io/v1beta4
kind: ClusterConfiguration
kubernetesVersion: 1.32.1
controlPlaneEndpoint: "k8scp:6443"
networking:
  podSubnet: 10.200.0.0/16 # instead of 192.168.0.0/16

Conclusions

That troubleshooting is definitely interesting and will improve the knowledge retain and understanding,
but i m not sure will continue on the single node (same physical machine) cluster,
it takes a lot of time to troubleshoot.

What do you suggests?
Better follow the Labs and move on GCE?

Thank you for the support.

Hi @chrispokorni ,
here the results of today troubleshooting, with the help of chatgpt (probably gonna move to claude soon).

At power on of the machine, etcd was again down

so, i settled a 3 terminals view (down the promt out):

• terminal 1: kubectl get pods -A -o wide
• terminal 2: crictl events
• terminal 3: sudo lsof -i :6443

i saw the etcd container go up and down,
SYN_SENT on port 6443 and timeout during dial tcp 192.168.105.156:6443.

so i started a post mortem analysis, etcd exited, containerd collected the garbage ( ) and crashloopbackoff reported from jounalctl.

so, diving in the etcd manifest i found:

[...]
    - --initial-advertise-peer-urls=https://192.168.105.156:2380
    - --initial-cluster=k8scp=https://192.168.105.156:2380
    - --key-file=/etc/kubernetes/pki/etcd/server.key
    - --listen-client-urls=https://127.0.0.1:2379,https://192.168.105.156:2379
    - --listen-metrics-urls=http://127.0.0.1:2381
    - --listen-peer-urls=https://192.168.105.156:2380
[...]

but, ip a | grep inet returned:

[...]
    inet 192.168.237.156/24 brd 192.168.237.255 scope global dynamic noprefixroute wlan0
[...]

hence, seems i have problems with ip addressing of all the jazz...

moreover, i started the cluster (see above) with:

kubeadm init --pod-network-cidr 192.168.0.0/16 --node-name k8scp --upload-certs

not sure about that,
but my opinion is that etcd is looking for 192.168.105.156 but it does not answer because of new ip.
Am i right?
Am i missing something else?

In the attached file 'prompt-out-commands.txt' you can find all the output of the following commands :

• for the 3 terminals view:

kubectl get pods -A -o wide
sudo crictl events
sudo lsof -i :6443

• for the post mortem analysis:

sudo crictl ps -a | grep etcd
sudo crictl logs 45d40286716af
sudo journalctl -u kubelet -xe | grep etcd
sudo cat /etc/kubernetes/manifests/etcd.yaml
ip a | grep inet

Hi @chrispokorni ,
Sorry for the late answer...
Thank you for the support,
Definitely got your point in the reapply anything...
Probably the static IP and the correct init procedure would be enough to start the cluster.

But, just decided that will post pone the implementation in the single node.
Really interesting the troubleshooting and the "side" understanding,
But will need tons of time more,
and actually i'm running a bit out of it.

To close the post,
I also found online a guide about a single node rpi implementation,
hoping gonna be useful for somebody link here.

See you next problem!
take care...

hi @chrispokorni ,
All good here.
VMs up and nginx running a lot
i been able to complete all the labs from chapter 3.
just one remark, i was not able to access to the control-plane' nginx from outside (point 6 of LAB 3.5).

instead, i was able to curl from localhost and from the worker node (using the clusterIP).

having a look on the web, seems that the GCP firewall is blocking the request,

so i tried a port-forward, but no luck.

any suggestions?

curl from control-plane

lc@control-plane-1-lfs258-n2:~$ curl localhost:32341
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

curl from worker node

lc@wn1-lfs258:~$ curl 10.198.0.2:32341
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>

port forwarding

kubectl port-forward service/nginx 32341:80
Forwarding from 127.0.0.1:32341 -> 80
Forwarding from [::1]:32341 -> 80