Help with lab - Upgrade the Cluster

I've some problems with lab 4.1 Upgrade The Cluster, after I finished the CP node upgrade the Calico Controller Manager does not start.
- NAMESPACE NAME READY STATUS RESTARTS AGE
- kube-system calico-kube-controllers-5f6cfd688c-pgm6x 0/1 CrashLoopBackOff 5 4m50s
- kube-system calico-node-bbtns 1/1 Running 0 7m59s
- kube-system calico-node-lhxc4 1/1 Running 0 28m
- kube-system coredns-558bd4d5db-g2l9k 0/1 Running 0 92s
- kube-system coredns-558bd4d5db-z84b5 0/1 Running 0 92s
- kube-system coredns-74ff55c5b-d2v8d 0/1 Running 0 4m50s
- kube-system etcd-cp 1/1 Running 1 64s
- kube-system kube-apiserver-cp 1/1 Running 1 63s
- kube-system kube-controller-manager-cp 1/1 Running 0 63s
- kube-system kube-proxy-7x9gp 1/1 Running 0 14s
- kube-system kube-proxy-95lcf 1/1 Running 0 38s
- kube-system kube-scheduler-cp 1/1 Running 0 64s
- kube-system upgrade-health-check-8bws2 0/1 Completed 0 38s
I'm just follow the instructions and do it the same with worker node. When I drain worker node this happens:
- error when evicting pods/"calico-kube-controllers-5f6cfd688c-pgm6x" -n "kube-system" (will retry after 5s): Cannot evict pod as it would violate the pod's disruption budget.
I've check the Pods and Calico Kube Constrollers doest not start.
- students@cp:~$ kubectl get pods --all-namespaces
- NAMESPACE NAME READY STATUS RESTARTS AGE
- kube-system calico-kube-controllers-5f6cfd688c-pgm6x 0/1 Running 10 10m
- kube-system calico-node-bbtns 1/1 Running 0 13m
- kube-system calico-node-lhxc4 1/1 Running 0 34m
- kube-system coredns-558bd4d5db-csnp8 1/1 Running 0 3m21s
- kube-system coredns-558bd4d5db-x2g5s 1/1 Running 0 3m21s
- kube-system etcd-cp 1/1 Running 1 6m32s
- kube-system kube-apiserver-cp 1/1 Running 1 6m31s
- kube-system kube-controller-manager-cp 1/1 Running 0 6m31s
- kube-system kube-proxy-7x9gp 1/1 Running 0 5m42s
- kube-system kube-proxy-95lcf 1/1 Running 0 6m6s
- kube-system kube-scheduler-cp 1/1 Running 0 6m32s
When i inspect the logs from calico-kube-controller, I've got this:
main.go 118: Failed to initialize Calico datastore error=Get https://10.96.0.1:443/apis/crd.projectcalico.org/v1/clusterinformations/default: context deadline exceeded
I think the calico does not connect with coredns and i'v delete thoose pods. After that all pods running fine. But i dont know why this happens.
I've tried 2 times this lab and this happens twice. But on the second I've succed and solved the problem.
Comments
-
I had this issue, calico-kube-controllers stuck in a crashloop. Deleting the pod fixed the issue, i was then able to drain the worker
0
Categories
- All Categories
- 207 LFX Mentorship
- 145 LFX Mentorship: Linux Kernel
- 819 Linux Foundation IT Professional Programs
- 371 Cloud Engineer IT Professional Program
- 167 Advanced Cloud Engineer IT Professional Program
- 66 DevOps Engineer IT Professional Program
- 132 Cloud Native Developer IT Professional Program
- 143 Express Training Courses & Microlearning
- 143 Express Courses - Discussion Forum
- Microlearning - Discussion Forum
- 6.7K Training Courses
- 40 LFC110 Class Forum - Discontinued
- 66 LFC131 Class Forum
- 39 LFD102 Class Forum
- 238 LFD103 Class Forum
- 17 LFD110 Class Forum
- 34 LFD121 Class Forum
- 1 LFD123 Class Forum
- LFD125 Class Forum
- 17 LFD133 Class Forum
- 6 LFD134 Class Forum
- 18 LFD137 Class Forum
- 70 LFD201 Class Forum
- 3 LFD210 Class Forum
- 2 LFD210-CN Class Forum
- 2 LFD213 Class Forum - Discontinued
- 128 LFD232 Class Forum - Discontinued
- 1 LFD233 Class Forum
- 3 LFD237 Class Forum
- 24 LFD254 Class Forum
- 727 LFD259 Class Forum
- 110 LFD272 Class Forum
- 3 LFD272-JP クラス フォーラム
- 10 LFD273 Class Forum
- 262 LFS101 Class Forum
- LFS111 Class Forum
- 2 LFS112 Class Forum
- 1 LFS116 Class Forum
- 3 LFS118 Class Forum
- 1 LFS120 Class Forum
- 3 LFS142 Class Forum
- 3 LFS144 Class Forum
- 3 LFS145 Class Forum
- 1 LFS146 Class Forum
- 16 LFS148 Class Forum
- 8 LFS151 Class Forum
- 1 LFS157 Class Forum
- 72 LFS158 Class Forum
- LFS158-JP クラス フォーラム
- 12 LFS162 Class Forum
- 1 LFS166 Class Forum
- 7 LFS167 Class Forum
- 1 LFS170 Class Forum
- 1 LFS171 Class Forum
- 2 LFS178 Class Forum
- 2 LFS180 Class Forum
- 1 LFS182 Class Forum
- 4 LFS183 Class Forum
- 30 LFS200 Class Forum
- 737 LFS201 Class Forum - Discontinued
- 2 LFS201-JP クラス フォーラム
- 17 LFS203 Class Forum
- 118 LFS207 Class Forum
- 1 LFS207-DE-Klassenforum
- LFS207-JP クラス フォーラム
- 301 LFS211 Class Forum
- 55 LFS216 Class Forum
- 50 LFS241 Class Forum
- 43 LFS242 Class Forum
- 37 LFS243 Class Forum
- 13 LFS244 Class Forum
- 6 LFS245 Class Forum
- LFS246 Class Forum
- LFS248 Class Forum
- 45 LFS250 Class Forum
- 1 LFS250-JP クラス フォーラム
- LFS251 Class Forum
- 145 LFS253 Class Forum
- LFS254 Class Forum
- LFS255 Class Forum
- 6 LFS256 Class Forum
- 1 LFS257 Class Forum
- 1.2K LFS258 Class Forum
- 9 LFS258-JP クラス フォーラム
- 116 LFS260 Class Forum
- 156 LFS261 Class Forum
- 41 LFS262 Class Forum
- 82 LFS263 Class Forum - Discontinued
- 15 LFS264 Class Forum - Discontinued
- 11 LFS266 Class Forum - Discontinued
- 23 LFS267 Class Forum
- 18 LFS268 Class Forum
- 29 LFS269 Class Forum
- 8 LFS270 Class Forum
- 200 LFS272 Class Forum
- 1 LFS272-JP クラス フォーラム
- 2 LFS147 Class Forum
- 2 LFS274 Class Forum
- 4 LFS281 Class Forum
- 7 LFW111 Class Forum
- 257 LFW211 Class Forum
- 179 LFW212 Class Forum
- 12 SKF100 Class Forum
- SKF200 Class Forum
- 2 SKF201 Class Forum
- 797 Hardware
- 199 Drivers
- 68 I/O Devices
- 37 Monitors
- 98 Multimedia
- 174 Networking
- 91 Printers & Scanners
- 85 Storage
- 754 Linux Distributions
- 82 Debian
- 67 Fedora
- 16 Linux Mint
- 13 Mageia
- 23 openSUSE
- 147 Red Hat Enterprise
- 31 Slackware
- 13 SUSE Enterprise
- 351 Ubuntu
- 465 Linux System Administration
- 39 Cloud Computing
- 71 Command Line/Scripting
- Github systems admin projects
- 91 Linux Security
- 78 Network Management
- 101 System Management
- 47 Web Management
- 56 Mobile Computing
- 19 Android
- 28 Development
- 1.2K New to Linux
- 1K Getting Started with Linux
- 366 Off Topic
- 114 Introductions
- 171 Small Talk
- 26 Study Material
- 534 Programming and Development
- 293 Kernel Development
- 223 Software Development
- 1.8K Software
- 263 Applications
- 182 Command Line
- 3 Compiling/Installing
- 405 Games
- 311 Installation
- 79 All In Program
- 79 All In Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)