Welcome to the Linux Foundation Forum!

nameserver 10.96.0.10 not responding

teggyteggy Posts: 6
in LFS258 Class Forum

Regarding exercice 9.3, I was getting issues to run the testing pod created by nettool.yaml. If I do as instructed, I can't get 'apt update' to work.

After some googling, I have changed the nameserver and added the following to nettool.yaml:
dnsPolicy: "None"
dnsConfig:
nameservers:

8.8.8.8
This tweak has helped me move forward to get curl and dnsutils installed and hence get ping working. But when I try to ping 10.96.0.10 (which was the default nameserver had I not changed the nameserver) from the pod, the ping fails. ping 10.96.0.10 also fails when I run it directly from the master node host.

I understand for, at least for the rest of the exercice, I need to use 10.96.0.10 as dns. So how can I fix this issue ? Thanks for any light you can provide

Kind regards,
Teggy

Comments

  • teggyteggy Posts: 6

    ----8<----------------
    Couple of investigations done so far
    ----8<----------------

    coredns pods seems to be ok:

    14:46 [email protected]:~ $ kubectl get pods --all-namespaces
    NAMESPACE NAME READY STATUS RESTARTS AGE
    accounting nginx-one-5b5d85886b-2dpw2 1/1 Running 0 3h55m
    accounting nginx-one-5b5d85886b-nrngj 1/1 Running 0 3h55m
    default nett 1/1 Running 0 7m35s
    default ubuntu 1/1 Running 0 36m
    kube-system calico-kube-controllers-77c4b7448-pgz7p 1/1 Running 10 41d
    kube-system calico-node-lhd49 1/1 Running 17 241d
    kube-system calico-node-zrtqf 1/1 Running 15 239d
    kube-system coredns-f9fd979d6-xxfcg 1/1 Running 10 41d
    kube-system coredns-f9fd979d6-zsxgp 1/1 Running 10 41d

    14:49 [email protected]:~ $ kubectl get svc
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    kubernetes ClusterIP 10.96.0.1 443/TCP 241d

    14:49 [email protected]:~ $ kubectl describe service kubernetes
    Name: kubernetes
    Namespace: default
    Labels: component=apiserver
    provider=kubernetes
    Annotations:
    Selector:
    Type: ClusterIP
    IP: 10.96.0.1
    Port: https 443/TCP
    TargetPort: 6443/TCP
    Endpoints: 10.132.0.5:6443
    Session Affinity: None
    Events:

    14:50 [email protected]:~ $ sudo netstat -nlp |grep 6443
    tcp6 0 0 :::6443 :::* LISTEN 3421/kube-apiserver

    14:50 [email protected]:~ $ ping 10.132.0.5
    PING 10.132.0.5 (10.132.0.5) 56(84) bytes of data.
    64 bytes from 10.132.0.5: icmp_seq=1 ttl=64 time=0.066 ms
    ^C
    — 10.132.0.5 ping statistics —
    2 packets transmitted, 2 received, 0% packet loss, time 1007ms
    rtt min/avg/max/mdev = 0.064/0.065/0.066/0.001 ms

    14:51 [email protected]:~ $ ping lab-3-1-1
    PING lab-3-1-1.europe-west1-b.c.lfs258-lfs258.internal (10.132.0.5) 56(84) bytes of data.
    64 bytes from k8smaster (10.132.0.5): icmp_seq=1 ttl=64 time=0.072 ms
    ^C
    — lab-3-1-1.europe-west1-b.c.lfs258-lfs258.internal ping statistics —
    3 packets transmitted, 3 received, 0% packet loss, time 2044ms
    rtt min/avg/max/mdev = 0.059/0.063/0.072/0.010 ms

  • chrispokornichrispokorni Posts: 925

    Hi @teggy,

    Not all services respond to ping. The fact that ping returns no response from 10.96.0.10 does not mean that your container cannot talk to it when needed. In the lab exercise there are no references to ping. A different command is used instead dig - are you able to run dig and see expected the outputs?

    Regards,
    -Chris

  • timothyawtimothyaw Posts: 14
    edited November 2020

    I can verify that teggy is correct, I had to add Google DNS (8.8.8.8) to the yaml file. Once installed I can ping and I can telnet (telnet 10.96.0.10 53). I get responses from both. If I do a dig @10.96.0.10 redhat.com, in the answer section I get "WARNING: recursion requested but not available" Any ideas? FYI - I'm running the cluster on personal vm's. Cloud isn't involved.

  • chrispokornichrispokorni Posts: 925

    Hi @timothyaw,

    A hypervisor will play some role in the networking configuration of your hosts, which will impact the overall cluster networking behavior. On GCE, where the labs have been tested, no additional config options were needed for the DNS exercise.

    What local hypervisor are you using and how is the host networking configured by the hypervisor?

    Regards,
    -Chris

  • timothyawtimothyaw Posts: 14
    edited November 2020

    I'm using KVM. I'm using the default network that is setup when you install KVM. I have ip forwarding turned on in the kernel.

  • teggyteggy Posts: 6
    edited November 2020

    Hi @chrispokorni ,

    Thanks for seeing into that issue. You're right about ping; I should have thought about that....

    I've tested once again but this time with dig and it's not working any better :(

    From the container which I've forced to use 8.8.8.8, I'm running the following:

    [email protected]:/etc# cat /etc/resolv.conf
nameserver 8.8.8.8
[email protected]:/etc# dig @10.96.0.10

; <<>> DiG 9.16.1-Ubuntu <<>> @10.96.0.10
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

    Another test which I've run is that I've updated resolv.conf and set the same info as other pods:

    [email protected]:/etc# cat /etc/resolv.conf
#nameserver 8.8.8.8
nameserver 10.96.0.10
search default.svc.cluster.local svc.cluster.local cluster.local europe-west1-b.c.lfs258-lfs258.internal c.lfs258-lfs258.internal google.internal
options ndots:5
[email protected]:/etc# dig

; <<>> DiG 9.16.1-Ubuntu <<>>
;; global options: +cmd
;; connection timed out; no servers could be reached

[email protected]:/etc# dig @10.96.0.10

; <<>> DiG 9.16.1-Ubuntu <<>> @10.96.0.10
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

    Any idea what could be wrong ? (in the meantime, I'll go once again through the pre-req and see if I've left something out. I don't think so since I have able to run the exercices so far...)

    For info, I'm running these nodes on GCE...

  • chrispokornichrispokorni Posts: 925

    Hi @teggy,

    This is a strange behavior. I tried replicating it on GCE, but everything worked as described in the lab exercise. In the past I did experience different, yet strange and unexpected behaviors from GCE networking, which I resolved with a custom VPC and firewall rule specifically set for the VMs of this class.

    How is your VPC setup, and what firewall rule(s) do you have associated with it?

    Regards,
    -Chris

  • chrispokornichrispokorni Posts: 925

    Hi @timothyaw,

    Each KVM networking type has its possible limitations, so it helps knowing how yours is setup.
    Is there any benefit from the enabled IP forwarding? I can't remember ever needing to enable it and all worked as expected.

    Regards,
    -Chris

  • teggyteggy Posts: 6

    Hello @chrispokorni

    Thanks for your time looking into that issue. I have been able to move forward, the issue was about the pre-requisites and specially the VPC configuration. I've got these set up correctly and things are working fine now.

    Kind regards,
    Teggy

  • rzadzinsrzadzins Posts: 13

    Hi @teggy
    Would you mind sharing the details of what you had to change in the VPC please?

Sign In or Register to comment.