core-dns cannot reach to kubernetes service

kursad.yildirim

Hi,

I am trying to install kubernetes on CentOS VMs by following the document "Kubernets The Hard Way" (https://github.com/kelseyhightower/kubernetes-the-hard-way). Everything went well up to deploying dns.

kubectl get pods -l k8s-app=kube-dns -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-5677dc4cdb-9khwg 0/1 Running 0 2d14h
coredns-5677dc4cdb-mfgcn 0/1 Running 0 2d14h

Pods are at running state but they do not get ready. Logs constantly state that it waits fro "kubernetes" service:

[INFO] plugin/ready: Still waiting on: "kubernetes"
[INFO] plugin/ready: Still waiting on: "kubernetes"
E0311 13:31:54.085156 1 reflector.go:178] pkg/mod/k8s.io/client-go@v0.18.3/tools/cache/reflector.go:125: Failed to list *v1.Endpoints: Get "https://10.32.0.1:443/api/v1/endpoints?limit=500&resourceVersion=0": x509: certificate is valid for 10.142.0.1, 10.141.200.11, 10.141.200.12, 10.141.200.13, 10.141.200.1, 127.0.0.1, not 10.32.0.1

I checked other similar cases but could not proceed further. In some casese it was the DNS cachhing being ebabled on the VM while in some cases it was pod network overlapping with ndoe network.

Any help is much appreciated,

Many thanks in advance,
~K

serewicz

Hello,

As it seems you are not following the course it may be difficult to get help. Every difference could be the cause. For example, what version of CentOS are you using? What version of Kubernetes are you trying to install? Etc. We test the labs using Ubuntu 18.04, which obviously is different than CentOS.

The error you are getting is that 10.32.0.1 is not valid for the x509 certificate. Why is it using that IP instead of the six IP addresses it is configured to use? What are you using to run your lab? How many interfaces does each node have? What are your firewall settings? How is your node to node network configured? Basically without a full understanding of what your cluster is, and how it has been configured there is a massive list of what could be wrong.

If you have a question about the LFS258 course and lab please follow the lab and let us know where you find an issue.

Regards,

kursad.yildirim

Thank you so much or your response @serewicz . I've been working with kubernetes distributions (Icp, rancher and openshift) for few years. I am generally trying not to use distributions speciic functions and as a personal rule of thumb doing everything by using kubernetes resource yaml files. But I have never experienced a raw kubernetes installation. Anyway now I am onto it. I haven't take the course yet but I will do that on May.

I feel myself more confident with CentOS so I started with it but I can switch to Ubuntu also. Right now I am using CentOS 8.3.2011.

10.32.0.1 is from the subnet reserved for services, as
--service-cluster-ip-range=10.32.0.0/24 \

Environment is running on vmware workstation as 9 VMs (3 master 3 infra 3 worker) for kubernetes. I installed other machines for services like http, dns, lb, fw, dhcp, nfs etc. selinux and firewalld has been disabled at all of the machines. Each machine has one network interface which is configred to take ipo from subnet 10.141.0.0/16. While pod cidr is
--cluster-cidr=10.200.0.0/16

There is an haproxy running in front of masters. It is configured as TCP mode. Listening port 6443 and forwarding to masters' 6443 port with roundrobin.

I understand that it is not easy to comment on without having sufficient information, I personally compalin about it when ppl just tell me a problem and ask for a solution. I just was not sure to what extend to describe the environment and state the problem. I do know about doing things at kubernetes and struggling when it comes to fundamentals and I know that it is the worst.

I am not sure about this is the correct platorm to ask about it. I mean without taking the course. Maybe places liek superuser.com is more appropriate. I appreciate any guidance. Many thanks in advance.
~K

kursad.yildirim

I just realizeI just realized that this is the class forum. Asking without taking the class is totally irrelevant at the moment. I am going to close the threadd that this is the class forum. Asking without taking the class is totally irrelevant at the moment. We can close this thread. I will seek for a solution somewhere else and come back here for the cases I face while taking the course.