Welcome to the Linux Foundation Forum!

adding a worker node to cluster giving error

santoshsub123
santoshsub123 Posts: 18
in LFD259 Class Forum

Hello instructors

i added a worker node to make my cluster from 1 master node 1 worker node to 1 master node 2 worker nodes

i recently added 2nd worker node where kubeadm join is failing. please advise what needs to be done.

Comments

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @santoshsub123,

    There are several reasons why a kubeadm join command may fail. These reasons may be included in the output produced by the command, or they may be deduced from warning and error messages displayed.

    Please provide the output of the failed join command, to help us troubleshoot the issue.

    Regards,
    -Chris

  • santoshsub123
    santoshsub123 Posts: 18

    sudo kubeadm join 172.31.35.59:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:72461e0dace6a289d3dd15ef5cc4a6fca76036fcc992093e844e386579a37557 --v=5
    I0823 09:32:58.630290 7362 join.go:395] [preflight] found NodeName empty; using OS hostname as NodeName
    I0823 09:32:58.630600 7362 initconfiguration.go:115] detected and using CRI socket: /var/run/crio/crio.sock
    [preflight] Running pre-flight checks
    I0823 09:32:58.630681 7362 preflight.go:90] [preflight] Running general checks
    I0823 09:32:58.630730 7362 checks.go:250] validating the existence and emptiness of directory /etc/kubernetes/manifests
    I0823 09:32:58.630784 7362 checks.go:287] validating the existence of file /etc/kubernetes/kubelet.conf
    I0823 09:32:58.630797 7362 checks.go:287] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
    I0823 09:32:58.630807 7362 checks.go:103] validating the container runtime
    I0823 09:32:58.642908 7362 checks.go:377] validating the presence of executable crictl
    I0823 09:32:58.643568 7362 checks.go:336] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
    I0823 09:32:58.643890 7362 checks.go:336] validating the contents of file /proc/sys/net/ipv4/ip_forward
    I0823 09:32:58.644106 7362 checks.go:654] validating whether swap is enabled or not
    I0823 09:32:58.644570 7362 checks.go:377] validating the presence of executable conntrack
    I0823 09:32:58.644785 7362 checks.go:377] validating the presence of executable ip
    I0823 09:32:58.645002 7362 checks.go:377] validating the presence of executable iptables
    I0823 09:32:58.645215 7362 checks.go:377] validating the presence of executable mount
    I0823 09:32:58.645428 7362 checks.go:377] validating the presence of executable nsenter
    I0823 09:32:58.645642 7362 checks.go:377] validating the presence of executable ebtables
    I0823 09:32:58.645836 7362 checks.go:377] validating the presence of executable ethtool
    I0823 09:32:58.645900 7362 checks.go:377] validating the presence of executable socat
    I0823 09:32:58.646086 7362 checks.go:377] validating the presence of executable tc
    I0823 09:32:58.646568 7362 checks.go:377] validating the presence of executable touch
    I0823 09:32:58.646608 7362 checks.go:525] running all checks
    I0823 09:32:58.662878 7362 checks.go:408] checking whether the given node name is valid and reachable using net.LookupHost
    I0823 09:32:58.663608 7362 checks.go:623] validating kubelet version
    I0823 09:32:58.768408 7362 checks.go:129] validating if the "kubelet" service is enabled and active
    I0823 09:32:58.784302 7362 checks.go:202] validating availability of port 10250
    I0823 09:32:58.784503 7362 checks.go:287] validating the existence of file /etc/kubernetes/pki/ca.crt
    I0823 09:32:58.784524 7362 checks.go:437] validating if the connectivity type is via proxy or direct
    I0823 09:32:58.784567 7362 join.go:465] [preflight] Discovering cluster-info
    I0823 09:32:58.784597 7362 token.go:78] [discovery] Created cluster-info discovery client, requesting info from "172.31.35.59:6443"
    I0823 09:32:58.795930 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again

  • santoshsub123
    santoshsub123 Posts: 18

    main.main
    _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
    runtime.main
    /usr/local/go/src/runtime/proc.go:225
    runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1371
    couldn't validate the identity of the API Server

  • santoshsub123
    santoshsub123 Posts: 18

    main.main
    _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
    runtime.main
    /usr/local/go/src/runtime/proc.go:225
    runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1371
    error execution phase preflight

    main.main
    _output/dockerized/go/src/k8s.io/kubernetes/cmd/kubeadm/kubeadm.go:25
    runtime.main
    /usr/local/go/src/runtime/proc.go:225
    runtime.goexit
    /usr/local/go/src/runtime/asm_amd64.s:1371

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @santoshsub123,

    How did you generate the token abcdef.0123456789abcdef and the discovery token hash 72461e0dace6a289d3dd15ef5cc4a6fca76036fcc992093e844e386579a37557 prior to running the join command?

    Regards,
    -Chris

  • santoshsub123
    santoshsub123 Posts: 18

    sudo kubeadm join 172.31.35.59:6443 --token abcdef.0123456789abcdef --discovery-token-ca-cert-hash sha256:72461e0dace6a289d3dd15ef5cc4a6fca76036fcc992093e844e386579a37557 --v=5

    I0823 09:32:58.784524 7362 checks.go:437] validating if the connectivity type is via proxy or direct
    I0823 09:32:58.784567 7362 join.go:465] [preflight] Discovering cluster-info
    I0823 09:32:58.784597 7362 token.go:78] [discovery] Created cluster-info discovery client, requesting info from "172.31.35.59:6443"

    I0823 09:34:24.465385 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:34:30.175613 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:34:35.606899 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:34:41.052016 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:34:47.074828 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:34:52.408849 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:34:57.722085 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:03.268472 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:09.198803 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:15.498788 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:20.947894 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:26.398686 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:32.535410 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:38.466388 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:44.778274 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:50.832257 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:35:56.626164 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:01.677345 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:06.924151 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:12.842566 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:19.310145 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:24.434379 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:30.331614 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:35.429836 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:41.475208 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:46.940728 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:52.205340 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:36:58.030695 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:03.855385 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:09.277700 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:14.922959 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:20.724888 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:26.109930 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:31.541856 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:37.734133 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:43.894861 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:50.220400 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    I0823 09:37:55.673063 7362 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "abcdef", will try again
    could not find a JWS signature in the cluster-info ConfigMap for token ID "abcdef"

  • MassimilianoGullusci
    MassimilianoGullusci Posts: 11

    Hi @chrispokorni and @serewicz ,

    I've the same issue on worker node after a successful run of k8sSecond.sh.
    In my case the discovery token hash was generated by control-plane node.

    I've preserved the CP installation output as suggested in your guide.
    Attached CP cluster-info and worker join command ouput too.

    Please help me to understand where I'm wrong.

    I've stopped to go on with this course cause I've become father on May, but I want to solve all start/preparation issues and go on.

    Thanks a lot.

    Massimiliano

  • chrispokorni
    chrispokorni Posts: 2,606

    Hi @MassimilianoGullusci,

    From the timestamps in your outputs you may have an expired bootstrap token. Try running the following commands in the order provided below, to cleanup the worker, generate a new token/hash combination on the cp node, and finally join the worker with the cp node:

    1 - On worker node: $ sudo kubeadm reset

    2 - On cp node: $ sudo kubeadm token create --print-join-command

    3 - On worker node: $ sudo kubeadm join ... <as generated by the command above>

    Regards,
    -Chris

  • MassimilianoGullusci
    MassimilianoGullusci Posts: 11

    Hi @chrispokorni,

    even if I had a great hope on your suggestion success, it fails creating a new token.

    Output of failed commends in attachment.

    I share with you my ".kube/config" file got from CP node.

    Is there something wrong?
    It was generated automatically.

    Which "condition" it expects during token generation?

    Thanks.

    Massimiliano

  • MassimilianoGullusci
    MassimilianoGullusci Posts: 11

    Hi @chrispokorni,

    to avoid possible errors due to out of time joining of my worker node, I've executed again all steps from the start.

    Now I can see CP and WORKER in the cluster.

    I'm very happy.

    But I have a question:

    What is missing to fix errors risen by execution of your suggested commands?

    Thanks

    Massimiliano

  • MassimilianoGullusci
    MassimilianoGullusci Posts: 11

    Sorry. Found it.
    Thanks a lot.

Categories

Upcoming Training