Welcome to the Linux Foundation Forum!

Lab 3.3 Cluster IP access

RaveendraGuntupalli
RaveendraGuntupalli Posts: 8
in LFS258 Class Forum

kubectl get svc nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
nginx ClusterIP 10.103.189.38 80/TCP 8m
curl 10.103.189.38:80

Using the cluster ip as above, unable to access the curl from master node, but I can access from worker node.
Is there anything I am missing here ? Thanks.

Comments

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi,
    Are you still working on GCP? Do you have a firewall rule to allow all traffic (all ports, all sources, all protocols)?
    If on AWS, you would need a similar firewal rule.
    This may be required to be able to complete this lab and subsequent labs as well.
    Regards,
    -Chris

  • RaveendraGuntupalli
    RaveendraGuntupalli Posts: 8

    Thank you.
    Added below and it starts working. Thanks.

    Name Type Targets Filters Protocols / ports Action Priority Network
    allowall Ingress Apply to all IP ranges: 0.0.0.0/0 all Allow 1000 default

  • EdwardQuick
    EdwardQuick Posts: 4

    Yes I have to say I spent ages trying to troubleshoot on lab 8 until I read this post :rage:
    After adding a firewall rule to allow all ingress traffic, and adding to the master and worker nodes, I'm back on track.

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @EdwardQuick ,
    I am glad you found the solution helpful.
    Did you encounter any similar issues in labs prior to Lab 8?
    Thanks,
    -Chris

  • EdwardQuick
    EdwardQuick Posts: 4

    I can't recall to be honest Chris. This is on GCP by the way. Perhaps I've got a poor memory but was there a LAB on how to set up the hosts on GCP and did it mention adding firewall rules?

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @EdwardQuick ,
    For some reason, I remember some instructions to open all ports for traffic, but I may be thinking of another course. On the K8s.io documentation site there are some notes about port ranges used by K8s, but last time I checked that list was incomplete (creating fw rules for those specific port ranges still produced errors). So for the purpose of these labs, I just opened all ports to all protocols from all sources.
    -Chris

  • gotojeff
    gotojeff Posts: 1

    is it updated in the introduction chapter? I didn't find it and spent hours with the local IP tables rule until I found this post.. :s

  • fcioanca
    fcioanca Posts: 2,613

    It will be included in the next course version, but it has not yet been done.

  • blackpiglet
    blackpiglet Posts: 5

    I also faced this problem. I really didn't make it work, even I tried the firewall rule mentioned in this post. At last, I followed the kuernetes-the-hard-way to set up a working cluster. Could you provide more specific commands?

    The following is the glcoud commands I used to create network and instances on GCP. What did I miss?

    $ gcloud compute networks create cka --subnet-mode custom
$ gcloud compute networks subnets create kubernetes --network cka --range 10.240.0.0/24
$ gcloud compute firewall-rules create cka-external --allow tcp:22,tcp:6443,icmp --network cka --source-ranges 0.0.0.0/0
$ gcloud compute firewall-rules create cka-internal --network cka --allow tcp,udp,icmp --source-ranges 192.168.0.0/16,10.240.0.0/16
$ gcloud compute instances create controller-1 --async --boot-disk-size 200GB --can-ip-forward --image-family ubuntu-1804-lts --image-project ubuntu-os-cloud --machine-type n1-standard-1 --private-network-ip 10.240.0.11 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet kubernetes --tags cka,controller
$ gcloud compute instances create worker-1 --async --boot-disk-size 200GB --can-ip-forward --image-family ubuntu-1804-lts --image-project ubuntu-os-cloud --machine-type n1-standard-1 --private-network-ip 10.240.0.21 --scopes compute-rw,storage-ro,service-management,service-control,logging-write,monitoring --subnet kubernetes --tags cka,worker
  • chrispokorni
    chrispokorni Posts: 2,605

    Hi,
    Kubernetes uses many port ranges for agent-to-agent communication and for external communication. As I mentioned above, create a FW rule to allow all traffic: all protocols, all ports, all sources. The GCP documentation on firewall rules shows a sample that you can use to create this. If you are using the console, just navigate to VPC network -> Firewall rules and from there you can create the same rule.
    Regards,
    -Chris

  • blackpiglet
    blackpiglet Posts: 5

    Thanks, Chris. It worked.
    Kubernetes-the-hard-way works fine with such firewall rule setting, but our lesson doesn't.
    The only difference here I can inspect is the network plugin. Kubernetes-the-hard-way uses GCP route and our lesson uses Calico.
    I doubt we can make it work with much less permissive firewall rule. Maybe, just add BGP protocol permission is enough.

  • demian711
    demian711 Posts: 1

    Hello!
    I Have the same issue. I am using 2 virtual machines in Azure, and curl is not responding:
    Some useful info:

    $ kubectl get svc nginx-7db75b8b78-8gtzs
    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    nginx-7db75b8b78-8gtzs ClusterIP 10.111.205.50 80/TCP 5h45m
    $ curl 10.111.205.50:80

    At this point nothing happens until timeout. I Already opened all ports in Azure portal (inbound / outbound rules).
    Master and node hosts have Internet access, but seems that pods doesn't. Or at least, can't resolve DNS, because I can pin google's IP, but can't to google.com:

    **/ # ping 172.217.168.206
    PING 172.217.168.206 (172.217.168.206): 56 data bytes
    64 bytes from 172.217.168.206: seq=0 ttl=48 time=3.478 ms
    64 bytes from 172.217.168.206: seq=1 ttl=48 time=3.292 ms
    ^C
    --- 172.217.168.206 ping statistics ---
    2 packets transmitted, 2 packets received, 0% packet loss
    round-trip min/avg/max = 3.292/3.385/3.478 ms
    / #
    / # ping google.com
    ^C
    **

    Also, all pods are running:

    $ kubectl get pod --all-namespaces
    NAMESPACE NAME READY STATUS RESTARTS AGE
    default bosybox-56bbfc6d5c-627wp 1/1 Running 1 6h5m
    default nginx-7db75b8b78-8gtzs 1/1 Running 0 24h
    kube-system calico-node-2k4qt 2/2 Running 2 3d23h
    kube-system calico-node-bkbc4 2/2 Running 2 4d
    kube-system coredns-869f847d58-ldtm9 1/1 Running 1 4d2h
    kube-system coredns-869f847d58-nfdjq 1/1 Running 1 4d2h
    kube-system etcd-demian-test 1/1 Running 1 3d23h
    kube-system kube-apiserver-demian-test 1/1 Running 1 3d23h
    kube-system kube-controller-manager-demian-test 1/1 Running 1 3d23h
    kube-system kube-proxy-6ddgr 1/1 Running 1 4d2h
    kube-system kube-proxy-gp2t4 1/1 Running 1 3d23h
    kube-system kube-scheduler-demian-test 1/1 Running 1 3d23h

    any ideas? Thanks

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @demian711 ,
    You can try to restart your DNS pods, and see if that resolves your DNS issue.
    If not, then here is a tutorial on how to setup DNS for Kubernetes on Azure:
    https://github.com/kubernetes-incubator/external-dns/blob/master/docs/tutorials/azure.md
    Regards,
    -Chris

Categories

Upcoming Training