Can not remove node.kubernetes.io/not-ready:NoSchedule taint

In Chapter 2 .1 Labs/Exercise 2.1: Deploy a New Cluster I am trying to remove the node.kubernetes.io/not-ready:NoSchedule taint but no success.
I ran the kubectl taint nodes --all node.kubernetes.io/not-ready-
many times and get
node/kubemaster untainted node/kubeworker untainted
as the output. But then when I run kubectl describe nodes |grep -i Taint
I get
Taints: node.kubernetes.io/not-ready:NoSchedule Taints: node.kubernetes.io/not-ready:NoSchedule
and kubectl get nodes
returns NotReady:
NAME STATUS ROLES AGE VERSION kubemaster NotReady master 78m v1.12.1 kubeworker NotReady <none> 76m v1.12.1
Comments
-
Resetting it with
sudo kubadm reset
and re-running the whole config fixed it.0 -
Thank you for the update.
There seems to be a bit of a feature that it takes about a minute between attempts for the taint to be fully removed. I believe in the notes it mentions this feature in the 1.12.x versions.
Regards,
0 -
@oliveriom
A few times I ran into similar behavior, but most times the taint removal worked as expected. When it did not, a reset and reconfiguration worked without any issues.
-Chris0 -
I have this problem right now and I did a a "sudo kubeadm reset". But I now cannot run the k8scp.sh file to set up the master again.
[email protected]:~$ rm cp.out
[email protected]:~$ bash k8scp.sh | tee $HOME/cp.out
WARNING!
/k8scp_run exists. Script has already been run on control plane.
Can someone please help me?[Edit] I deleted the file /k8scp_run and ran the bash script again, but the problem still persists.
0 -
Hi @madhu91s,
Unlike the
control-plane
andmaster
taints, thenot-ready
taint you are seeing is not removable. It is placed on nodes as result of misconfiguration - it simply means that none of the nodes are ready to run control plane or worker tasks. Once the issues are fixed, the taints will automatically be lifted and the nodes will reach the ready status.What are the outputs of
kubectl get nodes -o wide
kubectl get pods -A -o wide
Also, where are you running your cluster - on cloud VMs, local VMs, what are the sizes of your VMs, what guest OS is running your VMs?
Did you happen o watch the demo videos from the intro chapter that show the VM provisioning process together with all recommended network settings?Regards,
-Chris0 -
Hello Chris, thank you for such a quick response. I am also just going through the other posts in this forum as I am new here. You mentioned somewhere that one could delete the core-dns pod. I did that but the container is still in "ContainerCreating" state. The exact result is -
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Warning FailedScheduling 3m10s (x3 over 13m) default-scheduler 0/2 nodes are available: 2 node(s) had untolerated taint {node.kubernetes.io/not-ready: }. preemption: 0/2 nodes are available: 2 Preemption is not helpful for scheduling.
Normal Scheduled 29s default-scheduler Successfully assigned kube-system/coredns-565d847f94-cjcsv to master
Warning FailedMount 14s (x6 over 29s) kubelet MountVolume.SetUp failed for volume "config-volume" : object "kube-system"/"coredns" not registered
Warning NetworkNotReady 1s (x15 over 29s) kubelet network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: cni plugin not initializedI am doing exactly as what is instructed in the video, I currently have set up my lab on console.cloud.google.com with the same configuration mentioned by the instructor. I skipped the video explaining setting up of lab on AWS since I am not using that environment.
[Edit] I did not disable the apparmor. I probably missed this as I did not watch the AWS set up video. Do you recommend me to do this?
0 -
Hi @madhu91s,
Please run the following commands on your control plane node as the
student
user, and then provide the outputs from the commands requested in my previous response:kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yaml
Regards,
-Chris0 -
error: unable to read URL "https://docs.projectcalico.org/manifests/calico.yaml", server reported 404 Not Found, status code=404
I figured out what the problem was -
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.25.0/manifests/calico.yamlthis fixed my problem. Thank you!
0
Categories
- 9.9K All Categories
- 29 LFX Mentorship
- 82 LFX Mentorship: Linux Kernel
- 463 Linux Foundation Boot Camps
- 266 Cloud Engineer Boot Camp
- 93 Advanced Cloud Engineer Boot Camp
- 43 DevOps Engineer Boot Camp
- 28 Cloud Native Developer Boot Camp
- 1 Express Training Courses
- 1 Express Courses - Discussion Forum
- 1.6K Training Courses
- 18 LFC110 Class Forum
- 3 LFC131 Class Forum
- 19 LFD102 Class Forum
- 130 LFD103 Class Forum
- 9 LFD121 Class Forum
- 60 LFD201 Class Forum
- 1 LFD213 Class Forum - Discontinued
- 128 LFD232 Class Forum
- 23 LFD254 Class Forum
- 544 LFD259 Class Forum
- 100 LFD272 Class Forum
- 1 LFD272-JP クラス フォーラム
- 1 LFS145 Class Forum
- 20 LFS200 Class Forum
- 739 LFS201 Class Forum
- 1 LFS201-JP クラス フォーラム
- 1 LFS203 Class Forum
- 35 LFS207 Class Forum
- 295 LFS211 Class Forum
- 53 LFS216 Class Forum
- 45 LFS241 Class Forum
- 39 LFS242 Class Forum
- 33 LFS243 Class Forum
- 10 LFS244 Class Forum
- 27 LFS250 Class Forum
- 1 LFS250-JP クラス フォーラム
- 131 LFS253 Class Forum
- 962 LFS258 Class Forum
- 10 LFS258-JP クラス フォーラム
- 85 LFS260 Class Forum
- 124 LFS261 Class Forum
- 29 LFS262 Class Forum
- 78 LFS263 Class Forum
- 15 LFS264 Class Forum
- 10 LFS266 Class Forum
- 17 LFS267 Class Forum
- 16 LFS268 Class Forum
- 14 LFS269 Class Forum
- 193 LFS272 Class Forum
- 1 LFS272-JP クラス フォーラム
- 205 LFW211 Class Forum
- 148 LFW212 Class Forum
- 890 Hardware
- 212 Drivers
- 74 I/O Devices
- 44 Monitors
- 115 Multimedia
- 206 Networking
- 99 Printers & Scanners
- 85 Storage
- 747 Linux Distributions
- 88 Debian
- 64 Fedora
- 13 Linux Mint
- 13 Mageia
- 24 openSUSE
- 133 Red Hat Enterprise
- 33 Slackware
- 13 SUSE Enterprise
- 354 Ubuntu
- 468 Linux System Administration
- 38 Cloud Computing
- 67 Command Line/Scripting
- Github systems admin projects
- 93 Linux Security
- 77 Network Management
- 107 System Management
- 48 Web Management
- 61 Mobile Computing
- 22 Android
- 25 Development
- 1.2K New to Linux
- 1.1K Getting Started with Linux
- 525 Off Topic
- 127 Introductions
- 211 Small Talk
- 19 Study Material
- 782 Programming and Development
- 256 Kernel Development
- 492 Software Development
- 919 Software
- 255 Applications
- 181 Command Line
- 2 Compiling/Installing
- 76 Games
- 316 Installation
- 46 All In Program
- 46 All In Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)