7.1.15 No pods showing up

mikaelohlsson

kubectl get pod --all-namespaces |grep nginx

Does not show any pods. Then when I run
kubectl create -f ingress.yaml

i get the following error
Error from server (InternalError): error when creating "ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": failed to call webhook: Post "https://myingress-ingress-nginx-controller-admission.default.svc:443/networking/v1/ingresses?timeout=10s": dial tcp 10.100.32.57:443: connect: connection refused

fazlur.khan

@mikaelohlsson,

Did you execute the step 14 "helm install myingress ." (Note there is a . (dot/period) at the end of the command).? If not, then please retry the step 14 and rerun the commands you should be fine.

mikaelohlsson

Yes I did, it started the service but not the pods

chrispokorni

Hi @mikaelohlsson,

I am wondering about the overall state of your cluster.
What are the outputs of:

kubectl get nodes -o wide
kubectl get pods -A -o wide
kubectl get deployments -A
kubectl get daemonsets -A

Regards,
-Chris

mikaelohlsson

ubuntu@ip-172-31-24-56:~$ kubectl get nodes -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
ip-172-31-19-196 Ready 7d14h v1.31.1 172.31.19.196 Ubuntu 24.04.1 LTS 6.8.0-1016-aws containerd://1.7.22
ip-172-31-24-56 Ready control-plane 7d14h v1.31.1 172.31.24.56 Ubuntu 24.04.1 LTS 6.8.0-1016-aws containerd://1.7.22

ubuntu@ip-172-31-24-56:~$ kubectl get pods -A -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
default broken-58f89b548f-qq74l 1/1 Running 1 (3m46s ago) 41h 10.0.1.118 ip-172-31-19-196
default nginx 1/1 Running 1 (3m46s ago) 45h 10.0.1.241 ip-172-31-19-196
default registry-666dd8b56f-4kmr5 1/1 Running 5 (3m46s ago) 7d14h 10.0.1.10 ip-172-31-19-196
default second 2/2 Running 5 (3m46s ago) 44h 10.0.1.150 ip-172-31-19-196
default troubleshoot-6d76676cdf-ptjmm 1/1 Running 1 (3m46s ago) 42h 10.0.1.22 ip-172-31-19-196
default webtwo 1/1 Running 1 (3m46s ago) 45h 10.0.1.192 ip-172-31-19-196
kube-system cilium-8htjz 1/1 Running 7 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system cilium-bfzwr 1/1 Running 5 (3m46s ago) 7d14h 172.31.19.196 ip-172-31-19-196
kube-system cilium-envoy-6wfp9 1/1 Running 5 (3m46s ago) 7d14h 172.31.19.196 ip-172-31-19-196
kube-system cilium-envoy-qdtnc 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system cilium-operator-5c7867ccd5-dthzk 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system coredns-7c65d6cfc9-5dxrl 1/1 Running 7 (3m46s ago) 7d14h 10.0.0.188 ip-172-31-24-56
kube-system coredns-7c65d6cfc9-gj4gp 1/1 Running 7 (3m46s ago) 7d14h 10.0.0.13 ip-172-31-24-56
kube-system etcd-ip-172-31-24-56 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system kube-apiserver-ip-172-31-24-56 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system kube-controller-manager-ip-172-31-24-56 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system kube-proxy-jvkz6 1/1 Running 5 (3m46s ago) 7d14h 172.31.19.196 ip-172-31-19-196
kube-system kube-proxy-w9q7f 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kube-system kube-scheduler-ip-172-31-24-56 1/1 Running 8 (3m46s ago) 7d14h 172.31.24.56 ip-172-31-24-56
kubectl-8039 update-demo-nautilus-4dncx 1/1 Running 1 (3m46s ago) 40h 10.0.1.27 ip-172-31-19-196
kubectl-8039 update-demo-nautilus-p98nl 1/1 Running 1 40h 10.0.1.130 ip-172-31-19-196
linkerd-viz metrics-api-8448d5b8f8-fgjw8 2/2 Running 4 (81s ago) 44h 10.0.1.1 ip-172-31-19-196
linkerd-viz prometheus-674cc7b6d5-qgc2m 2/2 Running 2 (3m46s ago) 44h 10.0.1.142 ip-172-31-19-196
linkerd-viz tap-85c5f5dcb-m6rkw 2/2 Running 3 (104s ago) 44h 10.0.1.86 ip-172-31-19-196
linkerd-viz tap-injector-d79586df-bfzhx 2/2 Running 2 (3m46s ago) 44h 10.0.1.77 ip-172-31-19-196
linkerd-viz web-7bc7b85ccb-l5qjc 2/2 Running 2 (3m46s ago) 44h 10.0.1.24 ip-172-31-19-196
linkerd linkerd-destination-6bd54f5c54-jd84x 4/4 Running 4 (3m46s ago) 44h 10.0.1.214 ip-172-31-19-196
linkerd linkerd-identity-7849f4b847-7r2qw 2/2 Running 2 (3m46s ago) 44h 10.0.1.32 ip-172-31-19-196
linkerd linkerd-proxy-injector-6886d84cf6-htx2m 2/2 Running 2 (3m46s ago) 44h 10.0.1.161 ip-172-31-19-196
multitenant mainapp-5489f85574-9v9pj 2/2 Running 2 (3m46s ago) 44h 10.0.1.148 ip-172-31-19-196
multitenant mainapp-5489f85574-js98n 2/2 Running 2 (3m46s ago) 44h 10.0.1.70 ip-172-31-19-196
multitenant mainapp-5489f85574-kpmz5 2/2 Running 2 (3m46s ago) 44h 10.0.1.149 ip-172-31-19-196
multitenant mainapp-5489f85574-lgrrf 2/2 Running 2 (3m46s ago) 44h 10.0.1.82 ip-172-31-19-196
multitenant mainapp-5489f85574-ljfmh 2/2 Running 2 (3m46s ago) 44h 10.0.1.134 ip-172-31-19-196
sonobuoy sonobuoy 0/1 Unknown 0 41h ip-172-31-19-196
sonobuoy sonobuoy-e2e-job-e8ad6fe12ad344ff 0/2 Unknown 0 41h ip-172-31-19-196
sonobuoy sonobuoy-systemd-logs-daemon-set-dd4302d4569a47d3-24sbv 2/2 Running 2 (3m46s ago) 41h 172.31.24.56 ip-172-31-24-56
sonobuoy sonobuoy-systemd-logs-daemon-set-dd4302d4569a47d3-pltzz 1/2 CrashLoopBackOff 4 (20s ago) 41h 172.31.19.196 ip-172-31-19-196

mikaelohlsson

ubuntu@ip-172-31-24-56:~$ kubectl get deployments -A
NAMESPACE NAME READY UP-TO-DATE AVAILABLE AGE
default broken 1/1 1 1 41h
default registry 1/1 1 1 7d14h
default troubleshoot 1/1 1 1 42h
kube-system cilium-operator 1/1 1 1 7d14h
kube-system coredns 2/2 2 2 7d14h
linkerd-viz metrics-api 1/1 1 1 44h
linkerd-viz prometheus 1/1 1 1 44h
linkerd-viz tap 1/1 1 1 44h
linkerd-viz tap-injector 1/1 1 1 44h
linkerd-viz web 1/1 1 1 44h
linkerd linkerd-destination 1/1 1 1 44h
linkerd linkerd-identity 1/1 1 1 44h
linkerd linkerd-proxy-injector 1/1 1 1 44h
multitenant mainapp 5/5 5 5 44h

ubuntu@ip-172-31-24-56:~$ kubectl get daemonsets -A
NAMESPACE NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
kube-system cilium 2 2 2 2 2 kubernetes.io/os=linux 7d14h
kube-system cilium-envoy 2 2 2 2 2 kubernetes.io/os=linux 7d14h
kube-system kube-proxy 2 2 2 2 2 kubernetes.io/os=linux 7d14h
sonobuoy sonobuoy-systemd-logs-daemon-set-dd4302d4569a47d3 2 2 2 2 2 kubernetes.io/os=linux 41h

mikaelohlsson

ubuntu@ip-172-31-24-56:~$ k get all
NAME READY STATUS RESTARTS AGE
pod/broken-58f89b548f-qq74l 1/1 Running 1 (7m19s ago) 41h
pod/nginx 1/1 Running 1 (7m19s ago) 45h
pod/registry-666dd8b56f-4kmr5 1/1 Running 5 (7m19s ago) 7d14h
pod/second 2/2 Running 5 (7m19s ago) 44h
pod/troubleshoot-6d76676cdf-ptjmm 1/1 Running 1 (7m19s ago) 42h
pod/webtwo 1/1 Running 1 (7m19s ago) 45h

NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
service/kubernetes ClusterIP 10.96.0.1 443/TCP 7d14h
service/myingress-ingress-nginx-controller NodePort 10.101.46.219 80:31989/TCP,443:31066/TCP 42h
service/myingress-ingress-nginx-controller-admission ClusterIP 10.101.211.22 443/TCP 42h
service/registry ClusterIP 10.97.40.62 5000/TCP 7d14h
service/secondapp LoadBalancer 10.99.95.69 80:32000/TCP 44h
service/webone-svc ClusterIP 10.104.73.208 80/TCP 45h
service/webtwo-svc ClusterIP 10.109.28.90 80/TCP 45h

NAME READY UP-TO-DATE AVAILABLE AGE
deployment.apps/broken 1/1 1 1 41h
deployment.apps/registry 1/1 1 1 7d14h
deployment.apps/troubleshoot 1/1 1 1 42h

NAME DESIRED CURRENT READY AGE
replicaset.apps/broken-58f89b548f 1 1 1 41h
replicaset.apps/registry-666dd8b56f 1 1 1 7d14h
replicaset.apps/troubleshoot-6d76676cdf 1 1 1 42h

chrispokorni

Hi @mikaelohlsson,

You are correct, the myingress services are deployed but you have no endpoints because the expected DaemonSet is not deployed. You need to revisit Step 4 and edit the values.yaml manifest to set the expected controller - DaemonSet around line 218 prior to deploying the myingress release.

Regards,
-Chris

mikaelohlsson

I had misspelled it

Thanks