Unable to add Node - Lab 3.2

Comments

• serewicz Posts: 520

  March 2018

  Hello,

  I would begin by first checking that server IP address matches. Were you using the join statement from the output of kubeadm init?  If it's been over 24 hours you may need to generate a new certificate on the master server.   You may have a firewall inbetween the nodes. 

  If the IP address is accurate and it has been less than 24 hours since you initialied the master, use the nc command to check port 6443 like this: nc 10.10.128.0.4 6443 

  If you get a prompt back, then the worker node cannot reach the server and I would look for a firewall issue. Next I would check that adding the networking policy file worked and all the Pods are running and in good shape with kubectl show pods --all-namespaces -o wide

  Please report back if this works or not. If you get errors along the way please share the output so we can further troubleshoot the issue.

  Regards,

  0
• chrispokorni Posts: 307

  March 2018 edited March 2018

  Hi, 

  I notice you are providing one public key in your kubeadm join command:  e3b0c4429... and there is a different key in the error output: 9b263f52d90b...

  The key in the error output is the expexted key you should have in the kubeadm join command. Try to join with the 9b263f52d90b... key. Basically if you provide the wrong key in the kubeadm join, the error spits out the expected key to join the cluster, which is not something very secure if you asked me...

  I just reproduced the error by providing a different key than the one expected...

  Good luck!

  -Chris

  0
• rthothadri Posts: 7

  March 2018 edited March 2018

  Node1a - Master:

  [email protected]:~$ sudo kubeadm token create
  
  5ed1ee.fc9580d58d001c0e
  
  [email protected]:~$ sudo kubeadm token list
  
  TOKEN                     TTL       EXPIRES                USAGES                   DESCRIPTION   EXTRA GROUPS
  
  5ed1ee.fc9580d58d001c0e   23h       2018-03-21T00:51:28Z   authentication,signing   <none>        system:bootstrappers:kubeadm:default-node-token
  
  [email protected]:~$

  [email protected]:~$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der> /dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
  
  e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
  
  writing RSA key
  
  [email protected]:~$

  Node2a:

  [email protected]:~# kubeadm join --token 5ed1ee.fc9580d58d001c0e 10.128.0.4:6443 --discovery-token-ca-cert-hash \
  
  > sha256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
  
  [preflight] Running pre-flight checks.
  
          [WARNING FileExisting-crictl]: crictl not found in system path
  
  [discovery] Trying to connect to API Server "10.128.0.4:6443"
  
  [discovery] Created cluster-info discovery client, requesting info from "https://10.128.0.4:6443"
  
  [discovery] Failed to connect to API Server "10.128.0.4:6443": cluster CA found in cluster-info configmap is invalid: public key sha256:9b263f52d90b62458a6a6c6d5d415e9110fc6dcb9bf8392f102ddc34bf26e1ac not pinned
  
  [discovery] Trying to connect to API Server "10.128.0.4:6443"
  
  [discovery] Created cluster-info discovery client, requesting info from "https://10.128.0.4:6443"
  
  [discovery] Failed to connect to API Server "10.128.0.4:6443": cluster CA found in cluster-info configmap is invalid: public key sha256:9b263f52d90b62458a6a6c6d5d415e9110fc6dcb9bf8392f102ddc34bf26e1ac not pinned

  [email protected]:~#

  [email protected]:~# kubeadm join --token 5ed1ee.fc9580d58d001c0e 10.128.0.4:6443 --discovery-token-ca-cert-hash \
  
  > sha256:9b263f52d90b62458a6a6c6d5d415e9110fc6dcb9bf8392f102ddc34bf26e1ac
  
  [preflight] Running pre-flight checks.
  
          [WARNING FileExisting-crictl]: crictl not found in system path
  
  [discovery] Trying to connect to API Server "10.128.0.4:6443"
  
  [discovery] Created cluster-info discovery client, requesting info from "https://10.128.0.4:6443"
  
  [discovery] Requesting info from "https://10.128.0.4:6443" again to validate TLS against the pinned public key
  
  [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "10.128.0.4:6443"
  
  [discovery] Successfully established connection with API Server "10.128.0.4:6443"

  This node has joined the cluster:
  
  * Certificate signing request was sent to master and a response
  
    was received.
  
  * The Kubelet was informed of the new secure connection details.

  Run 'kubectl get nodes' on the master to see this node join the cluster.
  
  [email protected]:~#

  
  
  Hi Chris,

  Thanks for your suggestion. I was able to use the key from the error output to join successfully.

  I'm not sure why openssl command from Master Node is not returning the key that I should use for kubeadm join.

  I've provided the output from Node1a and Node2a.

  Regards,

  Ram

   

  0
• chrispokorni Posts: 307

  March 2018 edited March 2018

  Hi Ram, 

  There are a few characters missing from your 'openssl', my guess is due to copy/paste. 

  From https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-join/  and from the course/labs manual, the complete 'openssl' is the following: 

  
  openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'

  Compare it with yours, try both and see what the difference is... I assume this one will generate the expected hash.

  Good luck!

  -Chris

  0
• rthothadri Posts: 7

  March 2018

  Thanks Chris! Yes, it was missing "2" after der. I got the correct public key now!

  [email protected]:~$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa -pubin -outform der 2>/dev/null | openssl dgst -sha256 -hex | sed 's/^.* //'
  
  9b263f52d90b62458a6a6c6d5d415e9110fc6dcb9bf8392f102ddc34bf26e1ac
  
  [email protected]:~$

   

  0
• chrispokorni Posts: 307

  March 2018

  I am glad it worked. With these long commands and lots of piping it is easy to miss a few characters when copy/pasting, and copying from the pdf lab manual is not always seamless either. 

  What helps me in these cases is pasting the content into a text editor, double-checking for accuracy and only then copy from the editor and paste into the terminal. 

  0