nameserver 10.96.0.10 not responding

Regarding exercice 9.3, I was getting issues to run the testing pod created by nettool.yaml. If I do as instructed, I can't get 'apt update' to work.
After some googling, I have changed the nameserver and added the following to nettool.yaml:
dnsPolicy: "None"
dnsConfig:
nameservers:
8.8.8.8
This tweak has helped me move forward to get curl and dnsutils installed and hence get ping working. But when I try to ping 10.96.0.10 (which was the default nameserver had I not changed the nameserver) from the pod, the ping fails. ping 10.96.0.10 also fails when I run it directly from the master node host.
I understand for, at least for the rest of the exercice, I need to use 10.96.0.10 as dns. So how can I fix this issue ? Thanks for any light you can provide
Kind regards,
Teggy
Comments
-
----8<----------------
Couple of investigations done so far
----8<----------------coredns pods seems to be ok:
14:46 [email protected]:~ $ kubectl get pods --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
accounting nginx-one-5b5d85886b-2dpw2 1/1 Running 0 3h55m
accounting nginx-one-5b5d85886b-nrngj 1/1 Running 0 3h55m
default nett 1/1 Running 0 7m35s
default ubuntu 1/1 Running 0 36m
kube-system calico-kube-controllers-77c4b7448-pgz7p 1/1 Running 10 41d
kube-system calico-node-lhd49 1/1 Running 17 241d
kube-system calico-node-zrtqf 1/1 Running 15 239d
kube-system coredns-f9fd979d6-xxfcg 1/1 Running 10 41d
kube-system coredns-f9fd979d6-zsxgp 1/1 Running 10 41d14:49 [email protected]:~ $ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 443/TCP 241d14:49 [email protected]:~ $ kubectl describe service kubernetes
Name: kubernetes
Namespace: default
Labels: component=apiserver
provider=kubernetes
Annotations:
Selector:
Type: ClusterIP
IP: 10.96.0.1
Port: https 443/TCP
TargetPort: 6443/TCP
Endpoints: 10.132.0.5:6443
Session Affinity: None
Events:14:50 [email protected]:~ $ sudo netstat -nlp |grep 6443
tcp6 0 0 :::6443 :::* LISTEN 3421/kube-apiserver14:50 [email protected]:~ $ ping 10.132.0.5
PING 10.132.0.5 (10.132.0.5) 56(84) bytes of data.
64 bytes from 10.132.0.5: icmp_seq=1 ttl=64 time=0.066 ms
^C
— 10.132.0.5 ping statistics —
2 packets transmitted, 2 received, 0% packet loss, time 1007ms
rtt min/avg/max/mdev = 0.064/0.065/0.066/0.001 ms14:51 [email protected]:~ $ ping lab-3-1-1
PING lab-3-1-1.europe-west1-b.c.lfs258-lfs258.internal (10.132.0.5) 56(84) bytes of data.
64 bytes from k8smaster (10.132.0.5): icmp_seq=1 ttl=64 time=0.072 ms
^C
— lab-3-1-1.europe-west1-b.c.lfs258-lfs258.internal ping statistics —
3 packets transmitted, 3 received, 0% packet loss, time 2044ms
rtt min/avg/max/mdev = 0.059/0.063/0.072/0.010 ms0 -
Hi @teggy,
Not all services respond to
ping
. The fact thatping
returns no response from10.96.0.10
does not mean that your container cannot talk to it when needed. In the lab exercise there are no references toping
. A different command is used insteaddig
- are you able to rundig
and see expected the outputs?Regards,
-Chris0 -
I can verify that teggy is correct, I had to add Google DNS (8.8.8.8) to the yaml file. Once installed I can ping and I can telnet (telnet 10.96.0.10 53). I get responses from both. If I do a dig @10.96.0.10 redhat.com, in the answer section I get "WARNING: recursion requested but not available" Any ideas? FYI - I'm running the cluster on personal vm's. Cloud isn't involved.
0 -
Hi @timothyaw,
A hypervisor will play some role in the networking configuration of your hosts, which will impact the overall cluster networking behavior. On GCE, where the labs have been tested, no additional config options were needed for the DNS exercise.
What local hypervisor are you using and how is the host networking configured by the hypervisor?
Regards,
-Chris0 -
I'm using KVM. I'm using the default network that is setup when you install KVM. I have ip forwarding turned on in the kernel.
0 -
Hi @chrispokorni ,
Thanks for seeing into that issue. You're right about ping; I should have thought about that....
I've tested once again but this time with dig and it's not working any better
From the container which I've forced to use 8.8.8.8, I'm running the following:
[email protected]:/etc# cat /etc/resolv.conf nameserver 8.8.8.8 [email protected]:/etc# dig @10.96.0.10 ; <<>> DiG 9.16.1-Ubuntu <<>> @10.96.0.10 ; (1 server found) ;; global options: +cmd ;; connection timed out; no servers could be reached
Another test which I've run is that I've updated resolv.conf and set the same info as other pods:
[email protected]:/etc# cat /etc/resolv.conf #nameserver 8.8.8.8 nameserver 10.96.0.10 search default.svc.cluster.local svc.cluster.local cluster.local europe-west1-b.c.lfs258-lfs258.internal c.lfs258-lfs258.internal google.internal options ndots:5 [email protected]:/etc# dig ; <<>> DiG 9.16.1-Ubuntu <<>> ;; global options: +cmd ;; connection timed out; no servers could be reached [email protected]:/etc# dig @10.96.0.10 ; <<>> DiG 9.16.1-Ubuntu <<>> @10.96.0.10 ; (1 server found) ;; global options: +cmd ;; connection timed out; no servers could be reached
Any idea what could be wrong ? (in the meantime, I'll go once again through the pre-req and see if I've left something out. I don't think so since I have able to run the exercices so far...)
For info, I'm running these nodes on GCE...
0 -
Hi @teggy,
This is a strange behavior. I tried replicating it on GCE, but everything worked as described in the lab exercise. In the past I did experience different, yet strange and unexpected behaviors from GCE networking, which I resolved with a custom VPC and firewall rule specifically set for the VMs of this class.
How is your VPC setup, and what firewall rule(s) do you have associated with it?
Regards,
-Chris0 -
Hi @timothyaw,
Each KVM networking type has its possible limitations, so it helps knowing how yours is setup.
Is there any benefit from the enabled IP forwarding? I can't remember ever needing to enable it and all worked as expected.Regards,
-Chris0 -
Hello @chrispokorni
Thanks for your time looking into that issue. I have been able to move forward, the issue was about the pre-requisites and specially the VPC configuration. I've got these set up correctly and things are working fine now.
Kind regards,
Teggy0
Categories
- 10.1K All Categories
- 35 LFX Mentorship
- 88 LFX Mentorship: Linux Kernel
- 504 Linux Foundation Boot Camps
- 279 Cloud Engineer Boot Camp
- 103 Advanced Cloud Engineer Boot Camp
- 48 DevOps Engineer Boot Camp
- 41 Cloud Native Developer Boot Camp
- 2 Express Training Courses
- 2 Express Courses - Discussion Forum
- 1.7K Training Courses
- 17 LFC110 Class Forum
- 5 LFC131 Class Forum
- 19 LFD102 Class Forum
- 148 LFD103 Class Forum
- 12 LFD121 Class Forum
- 61 LFD201 Class Forum
- LFD210 Class Forum
- 1 LFD213 Class Forum - Discontinued
- 128 LFD232 Class Forum
- 23 LFD254 Class Forum
- 569 LFD259 Class Forum
- 100 LFD272 Class Forum
- 1 LFD272-JP クラス フォーラム
- 1 LFS145 Class Forum
- 23 LFS200 Class Forum
- 739 LFS201 Class Forum
- 1 LFS201-JP クラス フォーラム
- 1 LFS203 Class Forum
- 45 LFS207 Class Forum
- 298 LFS211 Class Forum
- 53 LFS216 Class Forum
- 46 LFS241 Class Forum
- 41 LFS242 Class Forum
- 37 LFS243 Class Forum
- 10 LFS244 Class Forum
- 27 LFS250 Class Forum
- 1 LFS250-JP クラス フォーラム
- 131 LFS253 Class Forum
- 996 LFS258 Class Forum
- 10 LFS258-JP クラス フォーラム
- 87 LFS260 Class Forum
- 126 LFS261 Class Forum
- 31 LFS262 Class Forum
- 79 LFS263 Class Forum
- 15 LFS264 Class Forum
- 10 LFS266 Class Forum
- 17 LFS267 Class Forum
- 17 LFS268 Class Forum
- 21 LFS269 Class Forum
- 200 LFS272 Class Forum
- 1 LFS272-JP クラス フォーラム
- 212 LFW211 Class Forum
- 153 LFW212 Class Forum
- 899 Hardware
- 217 Drivers
- 74 I/O Devices
- 44 Monitors
- 115 Multimedia
- 208 Networking
- 101 Printers & Scanners
- 85 Storage
- 749 Linux Distributions
- 88 Debian
- 64 Fedora
- 14 Linux Mint
- 13 Mageia
- 24 openSUSE
- 133 Red Hat Enterprise
- 33 Slackware
- 13 SUSE Enterprise
- 355 Ubuntu
- 473 Linux System Administration
- 38 Cloud Computing
- 69 Command Line/Scripting
- Github systems admin projects
- 94 Linux Security
- 77 Network Management
- 108 System Management
- 49 Web Management
- 63 Mobile Computing
- 22 Android
- 27 Development
- 1.2K New to Linux
- 1.1K Getting Started with Linux
- 528 Off Topic
- 127 Introductions
- 213 Small Talk
- 20 Study Material
- 794 Programming and Development
- 262 Kernel Development
- 498 Software Development
- 923 Software
- 258 Applications
- 182 Command Line
- 2 Compiling/Installing
- 76 Games
- 316 Installation
- 53 All In Program
- 53 All In Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)