Welcome to the Linux Foundation Forum!

LAB_7.2 ingress-nginx-controller-admission not found

Options
eg8888
eg8888 Posts: 11
in LFD259 Class Forum

ubuntu@controlPlane:~$
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$ kubectl create -f ingress.yaml
Error from server (InternalError): error when creating "ingress.yaml": Internal error occurred: failed calling webhook "validate.nginx.ingress.kubernetes.io": failed to call webhook: Post "https://ingress-nginx-controller-admission.ingress-nginx.svc:443/networking/v1/ingresses?timeout=10s": service "ingress-nginx-controller-admission" not found
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$ kubectl get ingress --all-namespaces
No resources found
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$ kubectl --namespace default get services -o wide myingress-ingress-nginx-controller
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
myingress-ingress-nginx-controller LoadBalancer 10.106.16.160 80:30824/TCP,443:32340/TCP 5m54s app.kubernetes.io/component=controller,app.kubernetes.io/instance=myingress,app.kubernetes.io/name=ingress-nginx
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$ kubectl get pod --all-namespaces |grep nginx
default myingress-ingress-nginx-controller-cxj55 1/1 Running 0 6m5s
default myingress-ingress-nginx-controller-fkwrc 1/1 Running 0 6m5s
ubuntu@controlPlane:~$
ubuntu@controlPlane:~$

Comments

  • eg8888
    eg8888 Posts: 11
    Options

    kubectl get nodes -o wide

    NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
    controlplane Ready control-plane 13d v1.29.1 10.19.107.197 Ubuntu 20.04.6 LTS 5.4.0-173-generic containerd://1.6.28
    worker Ready 13d v1.29.1 10.19.107.156 Ubuntu 20.04.6 LTS 5.4.0-173-generic containerd://1.6.28

  • eg8888
    eg8888 Posts: 11
    Options

    myingress vs ingress ?

    kubectl get ingressclasses.networking.k8s.io nginx -o yaml

    apiVersion: networking.k8s.io/v1
    kind: IngressClass
    metadata:
    annotations:
    meta.helm.sh/release-name: myingress
    meta.helm.sh/release-namespace: default
    creationTimestamp: "2024-03-14T23:26:50Z"
    generation: 1
    labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: myingress
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.10.0
    helm.sh/chart: ingress-nginx-4.10.0
    name: nginx
    resourceVersion: "645062"
    uid: 0784e1a8-a5c7-466b-bd76-9a20dc3239cd
    spec:
    controller: k8s.io/ingress-nginx

  • eg8888
    eg8888 Posts: 11
    Options

    kubectl --namespace default get services -o wide myingress-ingress-nginx-controller myingress-ingress-nginx-controller-admission

    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
    myingress-ingress-nginx-controller LoadBalancer 10.106.16.160 80:30824/TCP,443:32340/TCP 14h app.kubernetes.io/component=controller,app.kubernetes.io/instance=myingress,app.kubernetes.io/name=ingress-nginx
    myingress-ingress-nginx-controller-admission ClusterIP 10.98.138.176 443/TCP 14h app.kubernetes.io/component=controller,app.kubernetes.io/instance=myingress,app.kubernetes.io/name=ingress-nginx

  • chrispokorni
    chrispokorni Posts: 2,178
    edited March 15
    Options

    Hi @eg8888,

    Asking a clear question would be helpful. However, based only on the outputs provided, without any explanation, one may only guess that your myingress-ingress-nginx-controller-admission service cannot be reached.
    Perhaps there is a network policy that blocks access to TCP port 8443 of your myingress-ingress-nginx-controller pods?

    Regards,
    -Chris

  • eg8888
    eg8888 Posts: 11
    Options

    hi @chrispokorni,

    context is "LAB_7.2 ingress-nginx-controller-admission not found"

    with the error:

    it's supposed to connect to?:

    • ingress-nginx-controller-admission
      or

    • myingress-ingress-nginx-controller-admission

    kubectl exec -it secondapp -c busy -- sh
    ~ $
    ~ $ nc -vz myingress-ingress-nginx-controller-admission 443
    myingress-ingress-nginx-controller-admission (10.98.138.176:443) open
    ~ $
    ~ $ nc -vz myingress-ingress-nginx-controller 443
    myingress-ingress-nginx-controller (10.106.16.160:443) open
    ~ $
    ~ $ exit

    for situations like this, what's the best way to investigate, debug?

  • chrispokorni
    chrispokorni Posts: 2,178
    edited March 15
    Options

    Hi @eg8888,

    Per my comment:

    Perhaps there is a network policy that blocks access to TCP port 8443 of your myingress-ingress-nginx-controller pods?

    For this, would you be able to check if the network policy from exercise 6.5 is still operational? The timeout on the myingress-ingress-nginx-controller-admission service could suggest that traffic is blocked from reaching the myingress-ingress-nginx-controller pods.

    The service is there, the pods are running, the service displays endpoints, conditions that otherwise suffice for traffic to reach the desired pods. However, considering that we deployed a network policy in the earlier exercise 6.5, that could block traffic to the myingress-ingress-nginx-controller pods.

    Regards,
    -Chris

  • eg8888
    eg8888 Posts: 11
    Options

    ubuntu@controlPlane:~$ kubectl get networkpolicies.networking.k8s.io
    No resources found in default namespace.
    ubuntu@controlPlane:~$

    ubuntu@controlPlane:~$ kubectl exec -it secondapp -c busy -- sh
    ~ $
    ~ $ nc -vz ingress-ingress-nginx-controller-admission 8443
    nc: bad address 'ingress-ingress-nginx-controller-admission'
    ~ $
    ~ $ nc -vz ingress-ingress-nginx-controller 8443
    nc: bad address 'ingress-ingress-nginx-controller'
    ~ $
    ~ $
    ~ $ nc -vz myingress-ingress-nginx-controller-admission 8443
    ~ $
    ~ $ nc -vz myingress-ingress-nginx-controller 8443
    ~ $

    ~ $
    ~ $ nslookup ingress-ingress-nginx-controller-admission
    Server: 10.96.0.10
    Address: 10.96.0.10:53

    ** server can't find ingress-ingress-nginx-controller-admission.default.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.default.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.multipass: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.multipass: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller-admission.cluster.local: NXDOMAIN

    ~ $ nslookup ingress-ingress-nginx-controller
    Server: 10.96.0.10
    Address: 10.96.0.10:53

    ** server can't find ingress-ingress-nginx-controller.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.default.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.default.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.svc.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.cluster.local: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.multipass: NXDOMAIN

    ** server can't find ingress-ingress-nginx-controller.multipass: NXDOMAIN

    ~ $ nslookup myingress-ingress-nginx-controller-admission
    Server: 10.96.0.10
    Address: 10.96.0.10:53

    Name: myingress-ingress-nginx-controller-admission.default.svc.cluster.local
    Address: 10.98.138.176

    ** server can't find myingress-ingress-nginx-controller-admission.svc.cluster.local: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller-admission.svc.cluster.local: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller-admission.cluster.local: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller-admission.cluster.local: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller-admission.multipass: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller-admission.multipass: NXDOMAIN

    ~ $ nslookup myingress-ingress-nginx-controller
    Server: 10.96.0.10
    Address: 10.96.0.10:53

    ** server can't find myingress-ingress-nginx-controller.svc.cluster.local: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller.cluster.local: NXDOMAIN

    Name: myingress-ingress-nginx-controller.default.svc.cluster.local
    Address: 10.106.16.160

    ** server can't find myingress-ingress-nginx-controller.svc.cluster.local: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller.multipass: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller.multipass: NXDOMAIN

    ** server can't find myingress-ingress-nginx-controller.cluster.local: NXDOMAIN

    ~ $

  • chrispokorni
    chrispokorni Posts: 2,178
    Options

    Hi @eg8888,

    You seem to have bigger problems, since your DNS does not work.

    Please provide the outputs of the following:

    kubectl -n kube-system get pods -o wide
kubectl -n kube-system get cm coredns -o yaml

    Regards,
    -Chris

  • eg8888
    eg8888 Posts: 11
    Options

    ubuntu@controlPlane:~$ kubectl -n kube-system get pods -o wide
    NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
    cilium-2vlw4 1/1 Running 6 (3h38m ago) 14d 10.19.107.156 worker
    cilium-mc9p4 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane
    cilium-operator-5cddcb98d5-6drp8 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane
    coredns-76f75df574-7hrqn 1/1 Running 6 (3h38m ago) 14d 10.0.0.201 controlplane
    coredns-76f75df574-dn2hr 1/1 Running 6 (3h38m ago) 14d 10.0.0.48 controlplane
    etcd-controlplane 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane
    kube-apiserver-controlplane 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane
    kube-controller-manager-controlplane 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane
    kube-proxy-hdc2c 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane
    kube-proxy-pj5kn 1/1 Running 6 (3h38m ago) 14d 10.19.107.156 worker
    kube-scheduler-controlplane 1/1 Running 6 (3h38m ago) 14d 10.19.107.197 controlplane

    ubuntu@controlPlane:~$ kubectl -n kube-system get cm coredns -o yaml
    apiVersion: v1
    data:
    Corefile: |
    .:53 {
    errors
    health {
    lameduck 5s
    }
    ready
    kubernetes cluster.local in-addr.arpa ip6.arpa {
    pods insecure
    fallthrough in-addr.arpa ip6.arpa
    ttl 30
    }
    prometheus :9153
    forward . /etc/resolv.conf {
    max_concurrent 1000
    }
    cache 30
    loop
    reload
    loadbalance
    }
    kind: ConfigMap
    metadata:
    creationTimestamp: "2024-03-01T16:12:56Z"
    name: coredns
    namespace: kube-system
    resourceVersion: "254"
    uid: 66dbe4c3-9a27-4e23-a466-4fe9b55f4795
    ubuntu@controlPlane:~$

Categories

Upcoming Training