----->>k8scp.sh install script issues <<<------

Hello,
The install script stopped working this morning. From initial research there was an update to some support software, and the kubeadm init is failing with a strange node error, that has some interesting output. Working on it. May be a bug that will be fixed soon. Updates as possible.
Regards,
Comments
-
runc and crio package recently updated. Am investigating a work around. May include a script to run cluster with Docker until this issue is fixed upstream.
These are the exact issues you will find in production. Which is why we chose to download code live, not use a sanitized and near worthless demo type environment. Better to learn here then be surprised and seem clueless when you're in production. The downside is sometimes things break. With such an active community usually the fix is done quickly as well.
More updates as I find them.
1 -
Hello tutor!
I was wondering if the issue we are facing is related to the "metacopy=on" setting:
https://github.com/cri-o/cri-o/issues/4574From my side after I applied this change --> "sed -i 's/,metacopy=on//g' /etc/containers/storage.conf" , removing basically the metacopy=on setting from the storage.conf file, i have managed to successfully execute the "k8scp.sh" script in the master/cp node.
Best regards,
EfthimisPS: Info from the logs after the change
etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 22.002670 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.21" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node cp as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node cp as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]0 -
Thank you! I had been tracking down lots of other dead-ends, and adding Docker as an alternate engine in case the bug persists. Appreciate you letting me know. I'll test and add this to the course. Could be something fixed soon, but good troubleshooting in any case.
Thanks again!
1 -
Hello Tutor!
From my side after removing from the "/etc/containers/storage.conf" the "metacopy=on" option I have successfully executed the "k8scp.sh" script and installed the software in the master/cp node.
The Workaround has been found here:
https://github.com/cri-o/cri-o/issues/4574"sed -i 's/,metacopy=on//g' /etc/containers/storage.conf"
Best regards,
EfthimisPS: The Info from the terminal after applying it is as follows:
[kubeconfig] Using kubeconfig folder "/etc/kubernetes"
[kubeconfig] Writing "admin.conf" kubeconfig file
[kubeconfig] Writing "kubelet.conf" kubeconfig file
[kubeconfig] Writing "controller-manager.conf" kubeconfig file
[kubeconfig] Writing "scheduler.conf" kubeconfig file
[kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
[kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
[kubelet-start] Starting the kubelet
[control-plane] Using manifest folder "/etc/kubernetes/manifests"
[control-plane] Creating static Pod manifest for "kube-apiserver"
[control-plane] Creating static Pod manifest for "kube-controller-manager"
[control-plane] Creating static Pod manifest for "kube-scheduler"
[etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
[wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
[apiclient] All control plane components are healthy after 22.002670 seconds
[upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
[kubelet] Creating a ConfigMap "kubelet-config-1.21" in namespace kube-system with the configuration for the kubelets in the cluster
[upload-certs] Skipping phase. Please see --upload-certs
[mark-control-plane] Marking the node cp as control-plane by adding the labels: [node-role.kubernetes.io/master(deprecated) node-role.kubernetes.io/control-plane node.kubernetes.io/exclude-from-external-load-balancers]
[mark-control-plane] Marking the node cp as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]0 -
Hello again,
I added the sed command to the script, and it worked!! I'll update the script and have the updated labs out soon.
Thanks!
1 -
Hi,
@serewicz said:
Hello again,I added the sed command to the script, and it worked!! I'll update the script and have the updated labs out soon.
Thanks!
It seems that the change is still missing in the
k8sSecond.sh
script. While I can get the control plane node up successfully, the worker node is still giving mefailed to mount overlay for metacopy check with \"nodev,metacopy=on\" options: invalid argument
message. An indeed the storage.conf has not been updated on the worker node.I guess a workaround would be to recommend a newer kernel in course documentation instead of using the pretty old 4.15.x that is installed on the ubuntu image by default.
0 -
Hi,
Same here. Two Ubuntu 18.04 VM's on a laptop. The scripts ran fine but because the metacopy fix is missing from the k8sSecond.sh script I could not get pods running on the worker node. After removing the metacopy=on from the mountopt line in /etc/containers/storage.conf I was able to get my worker node working.
Found the issue by comparing the scripts. Should have checked the forum first :-)
0 -
This sounds like the same issue I reported back in October. More information that might be helpful here: https://forum.linuxfoundation.org/discussion/comment/32059
0
Categories
- 10.1K All Categories
- 35 LFX Mentorship
- 88 LFX Mentorship: Linux Kernel
- 504 Linux Foundation Boot Camps
- 279 Cloud Engineer Boot Camp
- 103 Advanced Cloud Engineer Boot Camp
- 48 DevOps Engineer Boot Camp
- 41 Cloud Native Developer Boot Camp
- 2 Express Training Courses
- 2 Express Courses - Discussion Forum
- 1.8K Training Courses
- 17 LFC110 Class Forum
- 5 LFC131 Class Forum
- 20 LFD102 Class Forum
- 148 LFD103 Class Forum
- 13 LFD121 Class Forum
- 61 LFD201 Class Forum
- LFD210 Class Forum
- 1 LFD213 Class Forum - Discontinued
- 128 LFD232 Class Forum
- 23 LFD254 Class Forum
- 569 LFD259 Class Forum
- 100 LFD272 Class Forum
- 1 LFD272-JP クラス フォーラム
- 1 LFS145 Class Forum
- 23 LFS200 Class Forum
- 739 LFS201 Class Forum
- 1 LFS201-JP クラス フォーラム
- 1 LFS203 Class Forum
- 45 LFS207 Class Forum
- 298 LFS211 Class Forum
- 53 LFS216 Class Forum
- 46 LFS241 Class Forum
- 41 LFS242 Class Forum
- 37 LFS243 Class Forum
- 10 LFS244 Class Forum
- 27 LFS250 Class Forum
- 1 LFS250-JP クラス フォーラム
- 131 LFS253 Class Forum
- 998 LFS258 Class Forum
- 10 LFS258-JP クラス フォーラム
- 87 LFS260 Class Forum
- 126 LFS261 Class Forum
- 31 LFS262 Class Forum
- 79 LFS263 Class Forum
- 15 LFS264 Class Forum
- 10 LFS266 Class Forum
- 17 LFS267 Class Forum
- 17 LFS268 Class Forum
- 21 LFS269 Class Forum
- 200 LFS272 Class Forum
- 1 LFS272-JP クラス フォーラム
- 212 LFW211 Class Forum
- 154 LFW212 Class Forum
- 899 Hardware
- 217 Drivers
- 74 I/O Devices
- 44 Monitors
- 115 Multimedia
- 208 Networking
- 101 Printers & Scanners
- 85 Storage
- 749 Linux Distributions
- 88 Debian
- 64 Fedora
- 14 Linux Mint
- 13 Mageia
- 24 openSUSE
- 133 Red Hat Enterprise
- 33 Slackware
- 13 SUSE Enterprise
- 355 Ubuntu
- 473 Linux System Administration
- 38 Cloud Computing
- 69 Command Line/Scripting
- Github systems admin projects
- 94 Linux Security
- 77 Network Management
- 108 System Management
- 49 Web Management
- 63 Mobile Computing
- 22 Android
- 27 Development
- 1.2K New to Linux
- 1.1K Getting Started with Linux
- 528 Off Topic
- 127 Introductions
- 213 Small Talk
- 20 Study Material
- 794 Programming and Development
- 262 Kernel Development
- 498 Software Development
- 923 Software
- 258 Applications
- 182 Command Line
- 2 Compiling/Installing
- 76 Games
- 316 Installation
- 54 All In Program
- 54 All In Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)