Hostname resolution on nginx/ubuntu container (excerise 7.2, step#11)

furqanbaqai

Hi Guys,

While doing exercise 7.2, step#11, i am getting following error while updating apt repositories:

root@thirdpage:/# apt-get update
Err:1 http://deb.debian.org/debian stretch InRelease Temporary failure resolving 'deb.debian.org'
Err:2 http://security.debian.org/debian-security stretch/updates InRelease Temporary failure resolving 'security.debian.org'

My container's resolv.conf have following content:

root@thirdpage:/# cat /etc/resolv.conf  
nameserver 10.96.0.10  
search default.svc.cluster.local svc.cluster.local cluster.local localdomain  
options ndots:5

NOTE My Host OS /etc/network/interfaces file is configured as:

baqai@lfdtru1804n01:/etc/init.d$ cat /etc/network/interfaces
# This file describes the network interfaces available on your system
# and how to activate them. For more information, see interfaces(5).

source /etc/network/interfaces.d/*

# The loopback network interface
auto lo
iface lo inet loopback

# The primary network interface
auto ens32
iface ens32 inet dhcp

NOTE: I am doing all these exercises on local VMs created on VMWare Workstation.

Appreciate your help..

Thanks.
Furqan Baqai

furqanbaqai

Hi Serewicz,

Thank you for your response.

Responding to your questions inline:
1. Does apt-get commands on the host work ?
Yes apt-get is working and ping to a DNS is working as well
2. Output of kubectl get pods -o wide:

baqai@lfdtru1804n01:~$ kubectl get svc -o wide
NAME         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE   SELECTOR
kubernetes   ClusterIP      10.96.0.1        <none>        443/TCP        21d   <none>
nginx        ClusterIP      10.100.147.5     <none>        443/TCP        16d   io.kompose.service=nginx
registry     ClusterIP      10.102.89.186    <none>        5000/TCP       16d   io.kompose.service=registry
secondapp    LoadBalancer   10.102.242.149   <pending>     80:32000/TCP   34h   example=second
thirdpage    NodePort       10.107.220.47    <none>        80:31325/TCP   24h   example=third

3. Output of kubectl get svc:

baqai@lfdtru1804n01:~$ kubectl get svc
NAME         TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP      10.96.0.1        <none>        443/TCP        21d
nginx        ClusterIP      10.100.147.5     <none>        443/TCP        16d
registry     ClusterIP      10.102.89.186    <none>        5000/TCP       16d
secondapp    LoadBalancer   10.102.242.149   <pending>     80:32000/TCP   34h
thirdpage    NodePort       10.107.220.47    <none>        80:31325/TCP   24h

4. Also ensure that the coredns and calico-node pods are properly running.
   I checked if the pods are running by issuing command kubectl get -n kube-system podsand found out that the pods are seeemd to be working. Not sure if it is something to do with their configuration. Please find the relevant output

baqai@lfdtru1804n01:~$ kubectl get -n kube-system pods
NAME                                    READY   STATUS    RESTARTS   AGE
calico-node-k5tv2                       2/2     Running   48         21d
calico-node-zwm9z                       2/2     Running   56         21d
coredns-86c58d9df4-62h6j                1/1     Running   28         21d
coredns-86c58d9df4-fzhk6                1/1     Running   28         21d
etcd-lfdtru1804n01                      1/1     Running   29         21d
kube-apiserver-lfdtru1804n01            1/1     Running   31         21d
kube-controller-manager-lfdtru1804n01   1/1     Running   29         21d
kube-proxy-8cs5r                        1/1     Running   25         21d
kube-proxy-qfkcc                        1/1     Running   28         21d
kube-scheduler-lfdtru1804n01            1/1     Running   29         21d
traefik-ingress-controller-qhv4s        1/1     Running   1          24h
traefik-ingress-controller-vv8nh        1/1     Running   1          24h

Thanks in advance for your help...

Regards,
Furqan Baqai

furqanbaqai

Hi Guys,

I think i found out the issue. As i am running all labs on VMWare workstation, kubernetes core DNS services are pointing to my vmware gateway for DNS resolution which is evidently failing. Please check below logs:

baqai@lfdtru1804n01:~$ kubectl logs --namespace=kube-system coredns-86c58d9df4-62h6j
.:53
2019-05-20T17:54:20.286Z [INFO] CoreDNS-1.2.6
2019-05-20T17:54:20.286Z [INFO] linux/amd64, go1.11.2, 756749c
CoreDNS-1.2.6
linux/amd64, go1.11.2, 756749c
 [INFO] plugin/reload: Running configuration MD5 = f65c4821c8a9b7b5eb30fa4fbc167769
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:47806->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:49965->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:58770->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:37151->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:42153->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:46402->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:35809->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:40866->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:33442->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 325794039783830212.4647384507963668824. HINFO: unreachable backend: read udp 192.168.0.163:54784->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. AAAA: unreachable backend: read udp 192.168.0.163:47129->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. A: unreachable backend: read udp 192.168.0.163:47656->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. A: unreachable backend: read udp 192.168.0.163:56256->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. AAAA: unreachable backend: read udp 192.168.0.163:42880->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. A: unreachable backend: read udp 192.168.0.163:60223->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. AAAA: unreachable backend: read udp 192.168.0.163:57832->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. A: unreachable backend: read udp 192.168.0.163:56050->192.168.159.2:53: i/o timeout
 [ERROR] plugin/errors: 2 www.linuxfoundation.org. AAAA: unreachable backend: read udp 192.168.0.163:33869->192.168.159.2:53: i/o timeout

Appreciate if somebody help us out to resolve this.