Lab 3.1 Cilium pods not being pulled

varmesto

Cilium is not working for me.
I am installing 2 ubuntu nodes in VirtualBox in my Windows. I configured student and root bash_profile to avoid my PC proxy error. What else could I check?

If I list the pod I see errors in the status of cilium and coredns pods:

NAMESPACE     NAME                                        READY   STATUS                  RESTARTS   AGE
default       nginx-7854ff8877-7g8jq                      0/1     Pending                 0          13m
kube-system   cilium-f78th                                0/1     Init:ImagePullBackOff   0          41m
kube-system   cilium-npgg9                                0/1     Init:ImagePullBackOff   0          41m
kube-system   cilium-operator-788c4f69bc-4t8kh            0/1     ImagePullBackOff        0          41m
kube-system   cilium-operator-788c4f69bc-b4ngr            0/1     ImagePullBackOff        0          23m
kube-system   coredns-5dd5756b68-85sr9                    0/1     Pending                 0          40m
kube-system   coredns-5dd5756b68-rs4fg                    0/1     Pending                 0          40m

When doing describe for some cilium pods I see in the events:

Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 40m default-scheduler Successfully assigned kube-system/cilium-npgg9 to secondarynodeubuntu
Warning Failed 40m kubelet Failed to pull image "quay.io/cilium/cilium:v1.14.1@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72": failed to pull and unpack image "quay.io/cilium/cilium@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72": failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://cdn03.quay.io/quayio-production-s3/sha256/33/33a5be5e9ebc036a2df5ec286fb5027a471e65550c3231ca7a92a17aa6428a47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI5LUAQGPZRPNKSJA/20240617/us-east-1/s3/aws4_request&X-Amz-Date=20240617T134013Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=87bc545a27e0623f3ef5897210c5547c990e98d5a6fd436cf0ac4eacd06020b4&cf_sign=J1gkLA5+o4WN9XUWTju6GfdKgIVHupgchWUWS+qqfpZEHvpCeK3lAyhSJPRjfKlhStx7guTWPbpBCymQda5z4BX++I4M4P/tTJddYFLfQxZHyQL0yQhUunumShhhqhCIwbdq/1gEx4JzpRIskvqAQXfgB1vmtIYguuzeZrg2G8B09f6YJ4q/zcfbSz0EDno+YBALKP72d+3hctSSwRTUZOuWRdbAckMME/87TWWBiMmoLbRrKpnlNq/07lUIaB02PoDuHCF3MaflbkMwYDG1BscXXvrN/H1Tj8GwmOljF0YcfxKhO4yNEq0Me5xnCX29QOTv6loBuhb3e6nxFpMFLA==&cf_expiry=1718632213&region=us-east-1&namespace=cilium&repo_name=cilium": tls: failed to verify certificate: x509: certificate signed by unknown authority
Warning Failed 39m kubelet Failed to pull image "quay.io/cilium/cilium:v1.14.1@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72": failed to pull and unpack image "quay.io/cilium/cilium@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72": failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://cdn03.quay.io/quayio-production-s3/sha256/33/33a5be5e9ebc036a2df5ec286fb5027a471e65550c3231ca7a92a17aa6428a47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI5LUAQGPZRPNKSJA/20240617/us-east-1/s3/aws4_request&X-Amz-Date=20240617T134031Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=83d517b63c13a7115bc6f1f9745747f55cc32385a040ab95b5c3f5836020bb2b&cf_sign=U/s2UMkK4cftLmBS/Q7+ZdUJdSuvqK06+WFFukkMhdA5+4xy2fLw+ZPcRmVcpLCZQCFFEZqJadat5ZAE4zWIbkVYt7hGqeAOUw/vjtue6q2Ad67Th5tFYfH65+s7zW+GWnqzvn529CID8OC9sjwQCrjD3F5Mfoj/h8jI8fVx17/wTw+fYBhdM48enT7yLIkUb31rmU1fyVGnkDYpERMQHi/b3K5n8zLJ6wEZoH2HO5vMbF3vJbQa3Z9aZ1A9bZsqvqAdoG3Okxhxe+VekjUbQx4q878DXQpkiW3fMTUEFUQ+5dH/B1i01X38mXTnqUyXI6yIOu5qYskLTdGMWxoeLQ==&cf_expiry=1718632231&region=us-east-1&namespace=cilium&repo_name=cilium": tls: failed to verify certificate: x509: certificate signed by unknown authority
Normal Pulling 39m (x3 over 40m) kubelet Pulling image "quay.io/cilium/cilium:v1.14.1@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72"
Warning Failed 39m (x3 over 40m) kubelet Error: ErrImagePull
Warning Failed 39m kubelet Failed to pull image "quay.io/cilium/cilium:v1.14.1@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72": failed to pull and unpack image "quay.io/cilium/cilium@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72": failed to copy: httpReadSeeker: failed open: failed to do request: Get "https://cdn03.quay.io/quayio-production-s3/sha256/33/33a5be5e9ebc036a2df5ec286fb5027a471e65550c3231ca7a92a17aa6428a47?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAI5LUAQGPZRPNKSJA/20240617/us-east-1/s3/aws4_request&X-Amz-Date=20240617T134057Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=6773fde5dadc33c052c47b46d6a47c0b3644efc3775e723b71635c931c2bc744&cf_sign=cQk2z2PNIxZm3D7+lG3pNLdCry/3WBB9j1cNTXRat2rRai2RmfEeo+9ukGrYPOUCxX7/X5Pmpf3ny+ewXnzwazssf0nI69el4TkqZOz0k0esyNaiyzyZjWMo2OsyckrZV2csfuvDjXpb1g+NkVn82V+9FbL2l6tYxqXYfP5YQdrVPqfzGX73kOiuXn3qPCgSqtrreGZ26NxMVD661NEsHCEh9FNDoP+g20YTLC5THd+r5zLlS2jQCgr9kKiXLCsL6TjK9qYFkG8mXaDBjAN7U8OzuYWqBG9nCV/UdjCV8yMGKV3Kf1E52C7dGU8BK1xYheK+bm5Ok1x/slDIKRncFQ==&cf_expiry=1718632257&region=us-east-1&namespace=cilium&repo_name=cilium": tls: failed to verify certificate: x509: certificate signed by unknown authority
Normal BackOff 38m (x4 over 40m) kubelet Back-off pulling image "quay.io/cilium/cilium:v1.14.1@sha256:edc1d05ea1365c4a8f6ac6982247d5c145181704894bb698619c3827b6963a72"
Warning Failed 38m (x4 over 40m) kubelet Error: ImagePullBackOff
Warning DNSConfigForming 4m59s (x162 over 40m) kubelet Nameserver limits were exceeded, some nameservers have been omitted, the applied nameserver line is:

chrispokorni

Hi @varmesto,

The cilium CNI plugin is critical even for cluster components, such as the coredns server.

For local VirtualBox setups, please ensure all VMs have a single bridged network interface with promiscuous mode enabled to allow all inbound traffic to the VMs.

In addition, configure your container runtime for the proxy to allow it to retrieve container images.

Regards,
-Chris

varmesto

Thank you for the hint. cilium images were pulled after adding the proxy configuration in the containerd and dockerd services