Welcome to the Linux Foundation Forum!

Lab 11.1. Service Mesh linkerd viz check

eboyko
eboyko Posts: 10
in LFS258 Class Forum

Hi

When I try "linkerd viz check" I get an error:
container "linkerd-proxy" in pod "metrics-api-54888ffcc5-jkcxm" is not ready

How to resolve this?

Output of "kubectl -n linkerd-viz describe pods metrics-api-54888ffcc5-jkcxm":
Name: metrics-api-54888ffcc5-jkcxm
Namespace: linkerd-viz
Priority: 0
Service Account: metrics-api
Node: cp/10.22.22.114
Start Time: Thu, 11 Jan 2024 15:02:51 +0000
Labels: component=metrics-api
linkerd.io/extension=viz
pod-template-hash=54888ffcc5
Annotations: checksum/config: b73fb1bf343c4203fbab8ee108c5eba2e07d184177e204677dc83d4cad2cd12b
cluster-autoscaler.kubernetes.io/safe-to-evict: true
config.alpha.linkerd.io/proxy-wait-before-exit-seconds: 0
linkerd.io/created-by: linkerd/helm stable-2.14.8
linkerd.io/inject: enabled
Status: Running
SeccompProfile: RuntimeDefault
IP: 192.168.0.22
IPs:
IP: 192.168.0.22
Controlled By: ReplicaSet/metrics-api-54888ffcc5
Containers:
metrics-api:
Container ID: containerd://99208e15ec3f614f49febf02b1e277e7768767b24580f6b5c9bff280845a57cf
Image: cr.l5d.io/linkerd/metrics-api:stable-2.14.8
Image ID: cr.l5d.io/linkerd/metrics-api@sha256:dce7f2085d26d1d526d2c4c2a69c81eda6d89f1abf17404e7746acf5582a901c
Ports: 8085/TCP, 9995/TCP
Host Ports: 0/TCP, 0/TCP
SeccompProfile: RuntimeDefault
Args:
-controller-namespace=linkerd
-log-level=info
-log-format=plain
-cluster-domain=cluster.local
-prometheus-url=http://prometheus.linkerd-viz.svc.cluster.local:9090
-enable-pprof=false
State: Running
Started: Thu, 11 Jan 2024 15:02:53 +0000
Ready: True
Restart Count: 0
Liveness: http-get http://:9995/ping delay=10s timeout=1s period=10s #success=1 #failure=3
Readiness: http-get http://:9995/ready delay=0s timeout=1s period=10s #success=1 #failure=7
Environment:
Mounts:
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-n5lm9 (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
kube-api-access-n5lm9:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional:
DownwardAPI: true
QoS Class: BestEffort
Node-Selectors: kubernetes.io/os=linux
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:

Comments

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @eboyko,

    In situations like this one, the actual Events (not displayed above) are the most helpful for troubleshooting purposes.

    In addition, what is the state of your cluster overall? What infrastructure is hosting your cluster?
    What are the outputs of the following commands:

    kubectl get nodes -o wide
    kubectl get pods -A -o wide

    Regards,
    -Chris

  • chrispokorni
    chrispokorni Posts: 2,606

    Also, @eboyko,

    How did you install the linkerd service mesh? Did you follow the installation sequence from the lab guide?

    Regards,
    -Chris

  • eboyko
    eboyko Posts: 10

    Hi @chrispokorni,

    I did follow the installation sequence in step 1 of lab guide.

    I also tried to reinstall linkerd using:
    linkerd viz uninstall | kubectl delete -f -
    linkerd uninstall | kubectl delete -f -
    and then repeating step 1 of lab guide.

    My VM for CP:
    VMware ESXi Hypervisor
    4 CPUs
    16 GB memory
    200 GB HDD
    Ubuntu 20.04.6 LTS
    1 network adapter

    My VM for Woker:
    VMware ESXi Hypervisor
    4 CPUs
    8 GB memory
    200 GB HDD
    Ubuntu 20.04.6 LTS
    1 network adapter

  • eboyko
    eboyko Posts: 10
    edited January 2024

    Output of following commands in attached file (kubectl.txt):
    kubectl get events --all-namespaces (during installation of linkerd and linkerd-viz)
    kubectl get nodes -o wide
    kubectl get pods -A -o wide

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @eboyko,

    Similar issues reported on github eventually got resolved by opening up the hypervisor firewall. In your situation it would be the ESXi firewall to allow all inbound traffic from all sources to all ports all protocols.

    Regards,
    -Chris

  • eboyko
    eboyko Posts: 10

    Hi, @chrispokorni,

    I managed to solve this issue by adding .cluster.local to no_proxy in /etc/kubernetes/manifests/kube-apiserver.yaml.

    Thank you!

Categories

Upcoming Training