Lab 3.6 - node is not ready once /etc/containerd/config.toml is changed the second time

mark.kharitonov

The instructions in containerd-setup.txt tells first to set /etc/containerd/config.toml to

version = 2
#disabled_plugins = ["cri"]
[plugins."io.containerd.runtime.v1.linux"]
  shim_debug = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

And then later on to

disabled_plugins = ["restart"]
[plugins.linux]
  shim_debug = true
[plugins.cri.containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"

My problem is that the node becomes not ready with the second version.

Please, observe:

The first variant - all is good

student@master:/etc/containerd$ cat config.toml
version = 2
#disabled_plugins = ["cri"]
[plugins."io.containerd.runtime.v1.linux"]
  shim_debug = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
student@master:/etc/containerd$ systemctl status containerd.service | head -5
● containerd.service - containerd container runtime
     Loaded: loaded (/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2022-12-18 20:35:41 UTC; 54min ago
       Docs: https://containerd.io
    Process: 145675 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
student@master:/etc/containerd$ k get node
NAME     STATUS   ROLES                  AGE     VERSION
master   Ready    control-plane,master   3h35m   v1.23.1
student@master:/etc/containerd$

The second variant - node becomes not ready

student@master:/etc/containerd$ sudo ln --force -s config.toml.v2 config.toml
student@master:/etc/containerd$ cat config.toml
disabled_plugins = ["restart"]
[plugins.linux]
  shim_debug = true
[plugins.cri.containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
student@master:/etc/containerd$ sudo systemctl restart containerd.service
student@master:/etc/containerd$ systemctl status containerd.service | head -5
● containerd.service - containerd container runtime
     Loaded: loaded (/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2022-12-18 21:31:11 UTC; 4s ago
       Docs: https://containerd.io
    Process: 176547 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
student@master:/etc/containerd$ k get node
NAME     STATUS   ROLES                  AGE     VERSION
master   Ready    control-plane,master   3h36m   v1.23.1
student@master:/etc/containerd$ sleep 30
student@master:/etc/containerd$ k get node
NAME     STATUS     ROLES                  AGE     VERSION
master   NotReady   control-plane,master   3h37m   v1.23.1
student@master:/etc/containerd$

Switch back to the first variant - all is good

student@master:/etc/containerd$ sudo ln --force -s config.toml.v1 config.toml
student@master:/etc/containerd$ cat config.toml
version = 2
#disabled_plugins = ["cri"]
[plugins."io.containerd.runtime.v1.linux"]
  shim_debug = true
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
  runtime_type = "io.containerd.runc.v2"
[plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runsc]
  runtime_type = "io.containerd.runsc.v1"
student@master:/etc/containerd$ sudo systemctl restart containerd.service
student@master:/etc/containerd$ systemctl status containerd.service | head -5
● containerd.service - containerd container runtime
     Loaded: loaded (/lib/systemd/system/containerd.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2022-12-18 21:33:03 UTC; 4s ago
       Docs: https://containerd.io
    Process: 176798 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
student@master:/etc/containerd$ k get node
NAME     STATUS   ROLES                  AGE     VERSION
master   Ready    control-plane,master   3h38m   v1.23.1
student@master:/etc/containerd$

When the second variant is selected:

student@master:/etc/containerd$ k describe node | tail -5
  Normal   NodeHasSufficientPID     3m22s                    kubelet  Node master status is now: NodeHasSufficientPID
  Normal   NodeAllocatableEnforced  3m21s                    kubelet  Updated Node Allocatable limit across pods
  Normal   NodeReady                3m21s                    kubelet  Node master status is now: NodeReady
  Normal   NodeNotReady             59s (x2 over 3m22s)      kubelet  Node master status is now: NodeNotReady
  Warning  ContainerGCFailed        22s (x2 over 82s)        kubelet  rpc error: code = Unimplemented desc = unknown service runtime.v1alpha2.RuntimeService
student@master:/etc/containerd$

Your help is greatly appreciated.

chrispokorni

Hi @mark.kharitonov,

I would recommend continuing the lab with the config.toml that allows the node to become ready.

Regards,
-Chris

mark.kharitonov

I guess I have no other choice, but I would like to understand what is wrong. After all, these are the instructions. I hope this will not impede me from doing the lab.

dennis6

Hi @mark.kharitonov,

Yes, I got the same issue.
May any one help to explain more detail about this and how to fix this issue?

Thank you.

chrispokorni

Hi @dennis6,

Please use the config.toml configuration that brings the node to a ready state, as suggested above.

Regards,
-Chris