cannot complete lab 10.1

echouserdevnull

using lab v2018-11-13

ive run through this a couple times, removing the created resources from the cluster and reapplying. when i attempt to complete step 11, i do not get a response from the load balancer. i can query the deployment successfully.

# k get ingresses.extensions ingress-test 
NAME           HOSTS             ADDRESS   PORTS   AGE
ingress-test   www.example.com             80      2d22h

i believe the failure is related to the lack of a bound address above (though i could be wrong, obv). netstat/lsof show no bound listeners on port 80 on any interface.

im using a 3 virtualbox instances on OSX, deployed with kubectl - one (untainted) master and two worker nodes.

can someone help me understand whats going on here?

serewicz

Hello,

Thank you for the detailed question. It helps try to troubleshoot. In this case you mention not getting a response from a load balancer, do you mean the load balancer service? Only if you have an external load balancer would you get that IP entry. If you send curl -H "Host: www.example.com" http://10.128.xx.yy/ where you replace the .xx.yy with your IP, do you get a nginx web server response?

When you try the same curl but to the public IP of your VMs (which may be the same, I don't know what networking options you configured in VirtualBox) is it different?

Regards,

echouserdevnull

none of the interfaces on the kube master or worker instances match 10.128.x.x.

i can query the pod address (10.38.x.x), and the service address(10.106.x.x) and get the nginx response. neither of these require the header to bet set to succeed. when i attempt to query either of the virtualized phyiscal nics, the curl fails.

i have to set up port forwarding to query the vbox host IP, which i havent done because neither virtual nic is answering, so i dont know where to send the traffic.

serewicz

Hello,

If you are not getting a 404 error, which would indicate you are reaching the ingress controller, but it is not configured to pass along the message to the proper back-end service.

If a query to the ClusterIP works and the Pod we know the web server container is running and responding to messages. Next would be to test the ingress controller. To test that you send the header to port 80 of the primary interface IP, not the pod directly. Does that work, if not what error do you get if any?

If neither vnic is answering it sounds like an issue with VirtualBox network settings. My guess would be routing. When you query the primary interface, port 80 and using the suggested header, from one node to the other do you get a 404 response, or passed along to the expected service?

Regards,

echouserdevnull

To test that you send the header to port 80 of the primary interface IP, not the pod directly. Does that work, if not what error do you get if any?

this is where it is failing, per above. i get a connection refused message on the master node. it looks like it works if i query the worker node directly it works after deleting and redeploying (i didnt taint the master to run pods). it might have done that before, i dont recall if i tested that or not.

If neither vnic is answering it sounds like an issue with VirtualBox network settings. My guess would be routing. When you query the primary interface, port 80 and using the suggested header, from one node to the other do you get a 404 response, or passed along to the expected service?

when i query the worker nodes, the 404 occurs correctly if the header is omitted from the curl, so i think that means its working correctly?

i might be misunderstanding how the loadbalancer is supposed to work - should any node in the cluster be able to route those queries, or only the nodes running the pod?

serewicz

Hello,

The way that Calico works is that iptables rules are updated on every node. Regardless of which worker node you make the request to, External-IP:High-Port-Number the combination of iptables and Calico will get the request to the pod, even if the pod is running on a different worker.

The LoadBalancer service type is a NodePort plus an asynchronous request to use an exterior load balancer. If one has not been configured on the exterior there is no response. The kubectl svc get output shows a "Pending" in the exterior IP field. To be more clear the creation of a load balancer does not create a load balancer, it only requests to use one.

If the worker nodes respond to its public IP and port 80 when the request is from within the node it would narrow down if the issue is with the ingress controller OR the networking external to the node.

If you query your outside facing IP address port 80 from within the node, and you still do not get at least a 404 then the issue is with the ingress controller.

If that works (You get the expected web page or at least a 404) we know the ingress controller is working so if you cannot get the same from outside the node then it is the networking of your virtualization choice.

Regards,

echouserdevnull

Ok that makes sense. it looks like i dont have a traefik controller running on my master because its untainted. the worker nodes running traefik work as described in the exercise.

so i think this is mostly my environment not matching the initial cluster exactly.