Welcome to the Linux Foundation Forum!

lab 11.2 ingress balancing and not routing based on host header

dsfraser
dsfraser Posts: 9
edited June 2025 in LFS258 Class Forum

I've got an issue where after i've setup lab 11.2 ingress with two nginx pods and the corresponding ingress controllers and ingress rules etc. despite curling to load-balancer IP using the host header, they traffic is always balanced between the two, the host header seems to be ignored.

kubectl get pods NAME READY STATUS RESTARTS AGE myingress-ingress-nginx-controller-2jwkw 2/2 Running 281 (3d23h ago) 5d myingress-ingress-nginx-controller-bjbh9 2/2 Running 0 3d23h nfs-subdir-external-provisioner-6b958d8dd5-fqt4z 1/1 Running 16 (3d23h ago) 68d web-one-6bd47f96d7-5przs 1/1 Running 0 5d2h web-two-6bd47f96d7-4bchp 1/1 Running 1 (3d23h ago) 5d2h

kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 84d myingress-ingress-nginx-controller LoadBalancer 10.100.109.123 <pending> 80:31751/TCP,443:31047/TCP 10d myingress-ingress-nginx-controller-admission ClusterIP 10.97.128.57 <none> 443/TCP 10d web-one ClusterIP 10.102.13.165 <none> 80/TCP 72m web-two ClusterIP 10.104.211.198 <none> 80/TCP 71

kubectl get ingress -o yaml
apiVersion: v1
items:

  • apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
    annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/service-upstream: "true"
    creationTimestamp: "2025-06-18T20:01:34Z"
    generation: 2
    name: ingress-test
    namespace: default
    resourceVersion: "5609621"
    uid: d48f43a7-097b-41cb-ba4d-742f880649bc
    spec:
    ingressClassName: nginx
    rules:
    • host: www.internal.com
      http:
      paths:
      • backend:
        service:
        name: web-two
        port:
        number: 80
        path: /
        pathType: ImplementationSpecific
    • host: www.external.com
      http:
      paths:
      • backend:
        service:
        name: web-one
        port:
        number: 80
        path: /
        pathType: ImplementationSpecific
        status:
        loadBalancer: {}
        kind: List
        metadata:
        resourceVersion: ""

Comments

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @dsfraser,

    Are the two applications web-one and web-two distinct? Do they return different content when accessed individually, and you can tell for sure which application was accessed based on its returned response?

    Regards,
    -Chris

  • dsfraser
    dsfraser Posts: 9
    edited June 2025

    Thanks for the reply chris, they are both nginx web servers and as per the instructions I have adjusted the index.html file to state either EXTERNAL or INTERNAL, but what happens is if I curl to external for example I get both INTERNAL And EXTERNAL output, it's like it's ignoring the host header and just balancing.

    root@master1:~# curl -H "Host: www.external.com" http://10.100.109.123
    <!DOCTYPE html>


    Welcome to INTERNAL!

    root@master1:~# curl -H "Host: www.external.com" http://10.100.109.123
    <!DOCTYPE html>


    Welcome to EXTERNAL!

    -Sam Fraser

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @dsfraser,

    Please provide the outputs for the following commands, using the code formatting for readability:

    kubectl get pod,svc,ep -o wide
kubectl get ingress
kubectl describe ingress ingress-test

    Regards,
    -Chris

  • dsfraser
    dsfraser Posts: 9
    edited July 2025

    `root@master1:~# kubectl get pod,svc,ep -o wide
    NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
    pod/books-54c8659c75-5fjzt 1/1 Running 0 26d 192.168.2.194 worker1
    pod/books-54c8659c75-fn55h 1/1 Running 1 (3d10h ago) 26d 192.168.1.247 worker2
    pod/myingress-ingress-nginx-controller-dl5lj 1/1 Running 0 31m 192.168.2.39 worker1
    pod/myingress-ingress-nginx-controller-lnl2c 1/1 Running 0 31m 192.168.1.241 worker2
    pod/nfs-subdir-external-provisioner-6b958d8dd5-fqt4z 1/1 Running 17 (3d10h ago) 96d 192.168.2.197 worker1
    pod/web-one-6bd47f96d7-5przs 1/1 Running 1 (3d10h ago) 33d 192.168.1.160 worker2
    pod/web-two-6bd47f96d7-4bchp 1/1 Running 1 (31d ago) 33d 192.168.2.251 worker1

    NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
    service/books ClusterIP 10.103.181.252 80/TCP 26d app=books
    service/kubernetes ClusterIP 10.96.0.1 443/TCP 112d
    service/myingress-ingress-nginx-controller LoadBalancer 10.110.174.163 80:32115/TCP,443:32500/TCP 31m app.kubernetes.io/component=controller,app.kubernetes.io/instance=myingress,app.kubernetes.io/name=ingress-nginx
    service/myingress-ingress-nginx-controller-admission ClusterIP 10.111.218.135 443/TCP 31m app.kubernetes.io/component=controller,app.kubernetes.io/instance=myingress,app.kubernetes.io/name=ingress-nginx
    service/web-one ClusterIP 10.102.13.165 80/TCP 28d system=secondary
    service/web-two ClusterIP 10.104.211.198 80/TCP 28d system=secondary

    NAME ENDPOINTS AGE
    endpoints/books 192.168.1.247:80,192.168.2.194:80 26d
    endpoints/cluster.local-nfs-subdir-external-provisioner 104d
    endpoints/kubernetes 172.28.98.3:6443 112d
    endpoints/myingress-ingress-nginx-controller 192.168.1.241:443,192.168.2.39:443,192.168.1.241:80 + 1 more... 31m
    endpoints/myingress-ingress-nginx-controller-admission 192.168.1.241:8443,192.168.2.39:8443 31m
    endpoints/web-one 192.168.1.160:80,192.168.2.251:80 28d
    endpoints/web-two 192.168.1.160:80,192.168.2.251:80 28d
    root@master1:~# kubectl get ingress
    NAME CLASS HOSTS ADDRESS PORTS AGE
    ingress-test nginx www.internal.com,www.external.com 80 33d
    root@master1:~# kubectl describe ingress ingress-test
    Name: ingress-test
    Labels:
    Namespace: default
    Address:
    Ingress Class: nginx
    Default backend:
    Rules:
    Host Path Backends
    ---- ---- --------
    www.internal.com
    / web-two:80 (192.168.1.160:80,192.168.2.251:80)
    www.external.com
    / web-one:80 (192.168.1.160:80,192.168.2.251:80)
    Annotations: kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/service-upstream: true
    Events:
    Type Reason Age From Message
    ---- ------ ---- ---- -------
    Normal Sync 33m nginx-ingress-controller Scheduled for sync
    Normal Sync 33m nginx-ingress-controller Scheduled for sync

  • chrispokorni
    chrispokorni Posts: 2,605
    edited July 2025

    Hi @dsfraser,

    For future reference, ensure the code block is closed also, to help format terminal content for readability.

    From your output, several objects seems to be adequate:

    • 1 pod replica for the web-one deployment, and 1 pod replica for the web-two deployment
    • 1 service web-one (presumably) exposing the web-one deployment, and 1 service web-two (presumably) exposing the web-two deployment

    However, what seems to be inadequate is the following:

    • 2 endpoints behind the web-one service, and 2 endpoints behind the web-two service

    A single pod replica for the web-one deployment should yield a single web-one endpoint behind the web-one service, and the same should be the case for the web-two pod - deployment - endpoint - service. It seems you misconfigured the two services and they cross manage the 2 deployments' replicas.

    I recommend you either troubleshoot and fix the existing web-one and web-two deployments and services, or tear them down and redeploy them from scratch. It is essential that the web-one and web-two objects are managed by distinct labels and selectors.

    Regards,
    -Chris

  • dsfraser
    dsfraser Posts: 9
    edited July 2025

    apiVersion: apps/v1 kind: Deployment metadata: annotations: deployment.kubernetes.io/revision: "1" creationTimestamp: "2025-06-18T19:38:03Z" generation: 2 name: web-two namespace: default resourceVersion: "4988942" uid: 00f56163-5d1e-46c3-ba83-929a622ac3ea spec: progressDeadlineSeconds: 600 replicas: 1 revisionHistoryLimit: 10 selector: matchLabels: system: secondary strategy: rollingUpdate: maxSurge: 25% maxUnavailable: 25% type: RollingUpdate template: metadata: creationTimestamp: null labels: system: secondary spec: containers: - image: nginx:1.20.1 imagePullPolicy: Always name: nginx ports: - containerPort: 80 protocol: TCP resources: {} terminationMessagePath: /dev/termination-log terminationMessagePolicy: File dnsPolicy: ClusterFirst restartPolicy: Always schedulerName: default-scheduler securityContext: {} terminationGracePeriodSeconds: 30

  • dsfraser
    dsfraser Posts: 9
    edited July 2025

    I'm sorry, inserting the terminal output in-between the `` is not working for me.

    This above is the deployment I'm using for web-one and web-two, the only difference being the name. It's a deployment used in a previous lab and thought it would be ok as the labels were being used as a taint and now ineffective. Why is it creating two endpoints? The pod output shows only one node target...

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @dsfraser,

    Aside from distinct names, deployments and their respective services must have distinct/unique labels and selectors. You may be re-using the system: secondary label between the two deployments, and their respective services inheriting it as well. This label was never intended to replace the deployment's own label, it was supposed to be an additional label to help in the context of the earlier exercise.

    Regards,
    -Chris

  • dsfraser
    dsfraser Posts: 9

    Thanks very much @chrispokorni for your advice. I have got this working now.
    The big surprise for me in all this is that a deployment can end up with another deployments pods and endpoints. I created two nginx deployments with 2 replicas each (one on each node) and then created the ingress to point a host header to each deployment. I manipulated each replicas index.html file to display text relevant to that deployments name e.g. "welcome to web-one/welcome to web-two" based on the header included in the curl. Works fine now, but good lessons learned! Thanks.

  • chrispokorni
    chrispokorni Posts: 2,605

    Hi @dsfraser,

    Kubernetes uses labels as a backbone to logically connect the types of objects that together implement an application's lifecycle operations (self-healing, scaling, updates), traffic management (load balancing, DNS) - in this case the deployment, its underlying replicaset, the pod replica(s), and the service.
    The objects launched to manage the web-one application should be logically connected by a unique label, such as app=web-one, to separate them from the objects managing web-two. A common label shared by many deployments causes scaling and update inconsistencies, while a common label shared by multiple services causes undesired traffic load balancing.

    Regards,
    -Chris

Categories

Upcoming Training