Lab 11.1 Unable to install linkerd

vramperez

Hello,

I'm running lab 11.1, but I can't get linkerd installed, as the pods are stuck in CrashLoopBackOff:

The logs of the pods are as follows:

To see the resolv.conf file being configured in the pods I ran a busybox pod, and the resolv.conf file has the following:

I think the "--" at the end might be the problem, but I don't know how to remove it.

I'm running the cluster with VM virtualBox Ubuntu 20.04
Does anyone else have this problem or know how to solve it?

Thank you!

chrispokorni

Hi @vramperez,

Can you please provide the contents of /etc/resolv.conf from the VirtualBox VM as well?

Regards,
-Chris

vramperez

Hi @chrispokorni,

Thank you for your quick response.

This is my VM's resolv.conf (Ubuntu's 20.04 default):

Thinking that the problem might be the search --, I installed the resolvconf package to edit /etc/resolv.conf in the VM so I could comment out the search --. However, this didn't work because even though the resolv.conf has the line search -- commented out, the pods' resolv.conf still has the -- at the end (I guess it will be added by coreDNS).

(The last screenshot is on the master, but the worker has the same config and /etc/resolv.conf).

Maybe I'm wrong and the problem is not the -- of the resolv.conf...

Thank you.
Br,

chrispokorni

Hi @vramperez,

Thank you for the guest VM details. I remember running these labs successfully on VirtualBox VMs, but I provisioned them manually, not with vagrant. Could you double check your Vagrantfile for any possible typos that may be carried over into the VM?

Also, can you check your workstation's /etc/resolv.conf ?

After manually commenting out (or removing) the search line from the VM's /etc/resolv.conf, did you re-deploy the pod?

Regards,
-Chris

vramperez

Hi @chrispokorni,

Attached is the Vagrantfile. It is a very standard Vagrantfile. I have tried disabling the vagrant-hosts plugin, but it doesn't solve the problem.

Attached is also the /etc/resolv.conf of my workstation (Ubuntu 20.04).

After commenting out the "search --" line in resolv.conf, I restarted not only the linkerd pods, but also the coreDNS pods, but that didn't work either.

It might be useful to compare the /etc/resolv.conf of my pods, with the one of someone who is working, to see what's different (since it's only a couple of lines).

Thanks!

chrispokorni

Hi @vramperez,

My workstation also runs native Ubuntu 20.04 and in my resolv.conf file there is no "search --" entry. Please consult the official docs on resolv.conf configuration to correct your workstation's file before attempting the labs any further.

Regards,
-Chris

vramperez

Hi @chrispokorni,

The error seems to be, as I suspected, in the "--" added to the end of the line in resolv.conf. I could only find the double hyphen "--" used to indicate the end of the search suffix list in the resolv.conf file in Ubuntu, Fedora or RHEL (Red Hat Enterprise Linux), but it is not a standard. However, I don't know why it is being added, nor how I could remove it.

However, since this seemed to be the issue, I was able to deploy linkerd by modifying the dnsPoolicy to None and adding the corresponding dnsConfig based on the resolv.conf values of all deployments:

      dnsPolicy: None
      dnsConfig:
        nameservers:
        - 10.96.0.10
        options:
        - name: ndots
          value: "5"
        searches:
        - linkerd.svc.cluster.local
        - svc.cluster.local
        - cluster.local

With this, all linkerd pods are running and it seems to be working. Tomorrow I hope to do the associated labs, and if I find any problems I'll update this thread.

Thank you!