Control plane deployment fails due to kubelet boot up timeout
Hi, I am trying to follow the first lab assignment. Concretely, the lab2.2 where we supposed to set up the environment and create two "master" and "worker" nodes. I cannot get through the "master" node setup.
Running the course "k8scp.sh" script fails on Ubuntu 20.04.
[kubelet-start] Starting the kubelet
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[kubelet-check] Initial timeout of 40s passed.
Unfortunately, an error has occurred:
timed out waiting for the condition
Checking the kubelet system status
kubelet.service - kubelet: The Kubernetes Node Agent
Loaded: loaded (/lib/systemd/system/kubelet.service; enabled; vendor preset: enabled)
Drop-In: /etc/systemd/system/kubelet.service.d
└─10-kubeadm.conf
Active: active (running) since Fri 2023-12-15 09:39:26 CET; 6min ago
Docs: https://kubernetes.io/docs/home/
Main PID: 7456 (kubelet)
Tasks: 19 (limit: 9513)
Memory: 59.1M
CPU: 8.799s
CGroup: /system.slice/kubelet.service
└─7456 [rosetta] /usr/bin/kubelet /usr/bin/kubelet --bootstrap-kubeconfig=/etc/kubernetes/bootstrap-kubelet.conf --kubeconfig=/etc/kubernetes/kubelet.conf --config=/var/lib/kubelet/config.yaml --container-runtime-endpoint=unix:///var/run/containerd/containerd.sock
The kubelet logs contain rpc error: code = Unknown desc = failed to generate sanbdox container spec options: failed to generate seccomp spec opts: seccomp is not supported\ and "Unable to register node with API server" err="Post \"https://198.19.249.189:6443/api/v1/nodes\": dial tcp 198.19.249.189:6443: connect: connection refused" node="kfd-master" which is probably caused by the former.
I checked if the kernel supports seccomp. It does, why the error then?
Checking the journal logs journalctl -xeu kubelet
7456 kubelet_node_status.go:70] "Attempting to register node" node="kfd-master" 7456 kubelet_node_status.go:92] "Unable to register node with API server" err="Post \"https://198.19.249.189:6443/api/v1/nodes\": dial tcp 198.19.249.189:6443: connect: connection refused" node="kfd-master" 7456 remote_runtime.go:193] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to generate sanbdox container spec options: failed to generate seccomp spec opts: seccomp is not supported" 7456 kuberuntime_sandbox.go:72] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to generate sanbdox container spec options: failed to generate seccomp spec opts: seccomp is not supported" pod="kube-system/kube-controller-manager-kfd-master"
Again, seccomp is not supported?
I have installed the seccomp packages and ran the following script:
#include <stdio.h>
#include <seccomp.h>
int main() {
if (seccomp_api_get() > 0) {
printf("Seccomp is supported.\n");
} else {
printf("Seccomp is not supported.\n");
}
return 0;
}
The script outputs
./check_seccomp Seccomp is supported.
What can be the issue here?
I should mention though, that I am running this on a local machine using the Linux virtual machine on a MacOS. The OS is Ubuntu 20.04, linux kernel is 6.5.13. The virtualization software I use allows to create several Linux machines the shared network for both machines will be provided by the software, so setting up the master and worker nodes should be possible (in theory).
Answers
-
Hi @tastyminerals,
For networking purposes I would recommend configuring the hypervisor with a DHCP server with a private subnet (10.0.0.0/16 or 172.16.0.0/12 for example).
For each VM a single bridged network interface would suffice, on the private subnet of the DHCP server, and enable all inbound (ingress) traffic to your VMs (this should be a setting on your hypervisor). Using public IP addresses or multiple network interfaces per VM requires additional configuration options when bootstrapping Kubernetes, that are beyond the scope of the course. In addition with public IP addresses you may need to account for external firewall rules (from ISP, enterprise, etc.) blocking protocols used by Kubernetes and its plugins.
Watching the two video guides from the introductory chapter may help. Although targeting AWS and GCP clouds, networking requirements are similar for on-prem VMs.
Regards,
-Chris0 -
@tastyminerals Have the same issue. Are you able to figure out the reason and fix it?
0 -
@oleksandrsettlemint I skipped
k8scp.shand created a node, worker environment via minikube:minikue start --nodes 2 -p my-cluster-name. You only need it for setting up the cluster nodes anyway. It works fine and you don't have a headache with their "custom" scripts that work only on AWS machines.0 -
@tastyminerals said:
@oleksandrsettlemint I skippedk8scp.shand created a node, worker environment via minikube:minikue start --nodes 2 -p my-cluster-name.@tastyminerals Have encountered any issues with the rest of course while using
minikube?0 -
Hi @ivanr5,
Please be aware than on Minikube, depending on the selected driver, certain aspects of the lab exercises may not work as presented in the lab guide. Also, considering the intermediate level of the course, it is highly recommended working with VMs (local or cloud) instead of Minikube.
Regards,
-Chris0
Categories
- All Categories
- 177 LFX Mentorship
- 177 LFX Mentorship: Linux Kernel
- 754 Linux Foundation IT Professional Programs
- 374 Cloud Engineer IT Professional Program
- 170 Advanced Cloud Engineer IT Professional Program
- 74 DevOps IT Professional Program - Discontinued
- 5 DevOps & GitOps IT Professional Program
- 100 Cloud Native Developer IT Professional Program
- 7.6K Training Courses & Learning Paths
- 2 AI & ML Training
- 1 Blockchain & Decentralized Identity Training
- 5 Cloud & Containers Training
- 1 Cybersecurity Training
- 2 DevOps & Site-Reliability Training
- 1 Linux Kernel Development Training
- 1 Networking Training
- 2 Open Source Best Practice Training
- 2 System Administration Training
- 1 System Engineering Training
- 1 Web & Application Development Training
- 794 Hardware
- 202 Drivers
- 68 I/O Devices
- 37 Monitors
- 95 Multimedia
- 173 Networking
- 91 Printers & Scanners
- 89 Storage
- 769 Linux Distributions
- 81 Debian
- 68 Fedora
- 22 Linux Mint
- 13 Mageia
- 24 openSUSE
- 150 Red Hat Enterprise
- 31 Slackware
- 13 SUSE Enterprise
- 356 Ubuntu
- 465 Linux System Administration
- 31 Cloud Computing
- 73 Command Line/Scripting
- Github systems admin projects
- 98 Linux Security
- 78 Network Management
- 101 System Management
- 46 Web Management
- 112 Mobile Computing
- 20 Android
- 77 Development
- 1.2K New to Linux
- 1K Getting Started with Linux
- 393 Off Topic
- 121 Introductions
- 182 Small Talk
- 29 Study Material
- 977 Programming and Development
- 310 Kernel Development
- 649 Software Development
- 990 Software
- 382 Applications
- 182 Command Line
- 5 Compiling/Installing
- 68 Games
- 317 Installation
- Archived
- 2 LFD140 Class Forum
- 1.4K LFS258 Class Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)