Lab 2.2 - 'kubadm init' fails

Hello,
When deploying a Control Plane Node using kubeadm (exercise 2.2), I'm launching 'sudo kubeadm init --config=$(find / -name kubeadm.yaml 2>/dev/null )' (command taken from k8scp.sh) which ultimately fails with:
I1129 18:10:45.586621 1407 checks.go:855] pulling: k8s.gcr.io/coredns:v1.8.4 [preflight] Some fatal errors occurred: [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:v1.8.4: output: time="2021-11-29T18:10:48+01:00" level=fatal msg="pulling image: rpc error: code = Unknown desc = reading manifest v1.8.4 in k8s.gcr.io/coredns: manifest unknown: Failed to fetch \"v1.8.4\" from request \"/v2/coredns/manifests/v1.8.4\"." , error: exit status 1
Any suggestion to fix it ?
Thanks.
Answers
-
Hi @fbui,
Is it possible there is a firewall that blocks access to the Google Container Registry? This could be a guest system firewall, and/or an infrastructure level firewall (at hypervisor level, VPC, etc.). Sometimes corporate VPNs may block such connection attempts.
Regards,
-Chris0 -
Hi,
I turned the firewall off as advised in the exercise preliminary, so I don't think the problem is related.
Pulling the image manually gave me the same error:
crictl -r unix:///var/run/crio/crio.sock pull k8s.gcr.io/coredns:v1.8.4 FATA[0000] pulling image: rpc error: code = Unknown desc = reading manifest v1.8.4 in k8s.gcr.io/coredns: manifest unknown: Failed to fetch "v1.8.4" from request "/v2/coredns/manifests/v1.8.4".
It seems that the issue is more related to the version of the manifest.
0 -
Hi @fbui,
Your
pull
command seems to be incomplete.I attempted to bootstrap a control-plane node following all the steps from the lab guide as provided - and it was successful. I also tried to pull the image with
crictl
andpodman
and none of them complained either:sudo crictl pull k8s.gcr.io/coredns/coredns:v1.8.4
sudo podman image pull k8s.gcr.io/coredns/coredns:v1.8.4
If these are not successful, I would start by looking at the networking configuration of the VMs, the local hypervisor or the cloud VPC, any associated firewalls, security groups, etc. I would also recommend watching the AWS/GCP setup videos from the introductory chapter, as they include essential tips for setting up the environment.
Regards,
-Chris0 -
Hi,
Both commands
crictl pull k8s.gcr.io/coredns/coredns:v1.8.4
podman image pull k8s.gcr.io/coredns/coredns:v1.8.4
succeeded.
Therefore it seems that the issue is due to
sudo kubeadm init --config=$(find / -name kubeadm.yaml 2>/dev/null )
The command (taken from k8scp.sh) launches crictl using the wrong path, ie
k8s.gcr.io/coredns:v1.8.4
instead ofk8s.gcr.io/coredns/coredns:v1.8.4
0 -
Hi @fbui,
Can you provide the line where
crictl
is called from yourkubeadm.yaml
file? In my version of the lab guide and SOLUTIONS there is no such call, but it all works as expected every single time I run thekubeadm init
command.Since this behavior cannot be reproduced, can you describe your environment? What VM's are you using, on what platform (what hypervisor, what cloud, what instance types, CPU, MEM, OS), what type of VM networking you have configured, any firewalls, etc? Do you have a history of commands all the way up to the step where you noticed the error? Can you provide the
cp.out
file?Regards,
-Chris0 -
I have the same issue on ubuntu-1804-lts (running in GCP), uploaded the cp.out file, could you pls help?
$ sudo kubeadm init --config=$(find / -name kubeadm.yaml 2>/dev/null )
W0320 23:41:47.568929 8885 utils.go:69] The recommended value for "resolvConf" in "KubeletConfiguration" is: /run/systemd/resolve/resolv.conf; the provided value is: /run/systemd/resolve/resolv.conf
[init] Using Kubernetes version: v1.23.1
[preflight] Running pre-flight checks
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR CRI]: container runtime is not running: output: time="2022-03-20T23:41:49Z" level=fatal msg="connect: connect endpoint 'unix:///var/run/crio/crio.sock', make sure you are running as root and the endpoint has been started: context deadline exceeded"
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with--ignore-preflight-errors=...
To see the stack trace of this error execute with --v=5 or higher0 -
@bvssnraju, @fbui You must use Ubuntu 20.04 as the base OS.
0 -
Thanks, @ankovi > @akovi said:
@bvssnraju, @fbui You must use Ubuntu 20.04 as the base OS.
I think this is quite an important one. Did you consider adding a remark on the lab page?
I was confused when the cluster stopped working for me without a reason.Also, did you notice that for Ubuntu 20.04 there is a warning message when running the init script?
Warning: policy/v1beta1 PodDisruptionBudget is deprecated in v1.21+, unavailable in v1.25+; use policy/v1 PodDisruptionBudget
poddisruptionbudget.policy/calico-kube-controllers created0 -
Hello,
The lab guide does call out the use of Ubuntu 20.04. It is mentioned a few places, but notably in the Overview section before the installation lab.
As well the warning you see is from CNCF software. I assume they are aware of the warning and will be updating what kubeadm uses soon. You can search through issues under https://github.com/kubernetes and add one if there isn't already work being done to take care of the warning.
Regards,
0 -
Hello there,
I'm also experiencing this kind of problem.
My setup is:
- Vagrant VirtualBox Ubuntu 20.04
- apparmor stopped and disabled
- ENV prepared with company proxy, all (http|https|no)_proxy variables set
Running
kubeadm init ...
fails to pull the necessary images whereas podman instead is able to pull them.I finally ended up in
1. runningkubeadm init
and let it fail
2. pull the image with podman
3. repeat at 1.Any advise to overcome this is highly appreciated
0 -
Hi @juetten,
What errors did you see while
kubeadm init
was running?After the cluster initialized, were you able to run the basicpod application and the firstpod deployment as instructed by Lab exercises 2.3 and 2.5 respectively?
Regards,
-Chris0
Categories
- All Categories
- 51 LFX Mentorship
- 104 LFX Mentorship: Linux Kernel
- 576 Linux Foundation IT Professional Programs
- 304 Cloud Engineer IT Professional Program
- 125 Advanced Cloud Engineer IT Professional Program
- 53 DevOps Engineer IT Professional Program
- 61 Cloud Native Developer IT Professional Program
- 5 Express Training Courses
- 5 Express Courses - Discussion Forum
- 2.1K Training Courses
- 19 LFC110 Class Forum
- 7 LFC131 Class Forum
- 27 LFD102 Class Forum
- 158 LFD103 Class Forum
- 20 LFD121 Class Forum
- 1 LFD137 Class Forum
- 61 LFD201 Class Forum
- 1 LFD210 Class Forum
- LFD210-CN Class Forum
- 1 LFD213 Class Forum - Discontinued
- 128 LFD232 Class Forum
- LFD237 Class Forum
- 23 LFD254 Class Forum
- 611 LFD259 Class Forum
- 105 LFD272 Class Forum
- 1 LFD272-JP クラス フォーラム
- 1 LFD273 Class Forum
- 2 LFS145 Class Forum
- 25 LFS200 Class Forum
- 739 LFS201 Class Forum
- 1 LFS201-JP クラス フォーラム
- 11 LFS203 Class Forum
- 75 LFS207 Class Forum
- 300 LFS211 Class Forum
- 54 LFS216 Class Forum
- 47 LFS241 Class Forum
- 41 LFS242 Class Forum
- 37 LFS243 Class Forum
- 11 LFS244 Class Forum
- 37 LFS250 Class Forum
- 1 LFS250-JP クラス フォーラム
- LFS251 Class Forum
- 141 LFS253 Class Forum
- LFS254 Class Forum
- 1.1K LFS258 Class Forum
- 10 LFS258-JP クラス フォーラム
- 93 LFS260 Class Forum
- 132 LFS261 Class Forum
- 33 LFS262 Class Forum
- 80 LFS263 Class Forum
- 15 LFS264 Class Forum
- 11 LFS266 Class Forum
- 18 LFS267 Class Forum
- 18 LFS268 Class Forum
- 23 LFS269 Class Forum
- 203 LFS272 Class Forum
- 1 LFS272-JP クラス フォーラム
- LFS274 Class Forum
- LFS281 Class Forum
- 236 LFW211 Class Forum
- 172 LFW212 Class Forum
- 7 SKF100 Class Forum
- SKF200 Class Forum
- 903 Hardware
- 219 Drivers
- 74 I/O Devices
- 44 Monitors
- 116 Multimedia
- 209 Networking
- 101 Printers & Scanners
- 85 Storage
- 763 Linux Distributions
- 88 Debian
- 66 Fedora
- 15 Linux Mint
- 13 Mageia
- 24 openSUSE
- 142 Red Hat Enterprise
- 33 Slackware
- 13 SUSE Enterprise
- 357 Ubuntu
- 479 Linux System Administration
- 41 Cloud Computing
- 70 Command Line/Scripting
- Github systems admin projects
- 95 Linux Security
- 78 Network Management
- 108 System Management
- 49 Web Management
- 68 Mobile Computing
- 23 Android
- 30 Development
- 1.2K New to Linux
- 1.1K Getting Started with Linux
- 538 Off Topic
- 131 Introductions
- 217 Small Talk
- 22 Study Material
- 826 Programming and Development
- 278 Kernel Development
- 514 Software Development
- 928 Software
- 260 Applications
- 184 Command Line
- 3 Compiling/Installing
- 76 Games
- 316 Installation
- 61 All In Program
- 61 All In Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)