Welcome to the Linux Foundation Forum!

Lab 3.2 new token not having a JWS signature in cluster-info ConfigMap

Hi,

I started few days ago the LFS258 training and I am stuck on lab exercise 3.2.
I created the 2 VMs, master and worker, with VirtualBox (host-only + NAT) and set static IPs

From 3.1, I created the cluster with the kubeadm-config.yaml below:

  1. apiVersion: kubeadm.k8s.io/v1beta2
  2. kind: InitConfiguration
  3. localAPIEndpoint:
  4. advertiseAddress: 192.168.56.100
  5. bindPort: 6443
  6. ---
  7. apiVersion: kubeadm.k8s.io/v1beta2
  8. kind: ClusterConfiguration
  9. kubernetesVersion: 1.18.1
  10. controlPlaneEndpoint: "k8smaster:6443"
  11. networking:
  12. podSubnet: 192.168.0.0/16

I successfully applied calico config to the cluster as well.
You can find the kubeadm-init.out in attachment.

From 3.2, I created a new token from the master

student@master:~$ sudo kubeadm --v=5 token create --print-join-command
I0510 08:57:08.914031 27949 token.go:121] [token] validating mixed arguments
I0510 08:57:08.914123 27949 token.go:130] [token] getting Clientsets from kubeconfig file
I0510 08:57:08.914149 27949 cmdutil.go:79] Using kubeconfig file: /home/student/.kube/config
I0510 08:57:08.919229 27949 token.go:243] [token] loading configurations
I0510 08:57:08.920355 27949 interface.go:400] Looking for default routes with IPv4 addresses
I0510 08:57:08.921316 27949 interface.go:405] Default route transits interface "enp0s3"
I0510 08:57:08.922168 27949 interface.go:208] Interface enp0s3 is up
I0510 08:57:08.923092 27949 interface.go:256] Interface "enp0s3" has 2 addresses :[192.168.56.100/24 fe80::a00:27ff:fef8:907f/64].
I0510 08:57:08.923849 27949 interface.go:223] Checking addr 192.168.56.100/24.
I0510 08:57:08.924634 27949 interface.go:230] IP found 192.168.56.100
I0510 08:57:08.925321 27949 interface.go:262] Found valid IPv4 address 192.168.56.100 for interface "enp0s3".
I0510 08:57:08.925966 27949 interface.go:411] Found active IP 192.168.56.100
W0510 08:57:08.926707 27949 configset.go:202] WARNING: kubeadm cannot validate component configs for API groups [kubelet.config.k8s.io kubeproxy.config.k8s.io]
I0510 08:57:08.927506 27949 token.go:255] [token] creating token
kubeadm join k8smaster:6443 --token 8j77yo.vrv09tpb6wqpf2p9 --discovery-token-ca-cert-hash sha256:d21ca296b30091b304dfa03fb3b600e32eb67cef13c3f51badd835e25dfad1ba

student@master:~$ kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
8j77yo.vrv09tpb6wqpf2p9 23h 2020-05-11T08:57:08Z authentication,signing system:bootstrappers:kubeadm:default-node-token

and when I use it to join the cluster from the worker, I get the following issue:

root@worker:/home/student# kubeadm join --v=5 k8smaster:6443 --token 8j77yo.vrv09tpb6wqpf2p9 --discovery-token-ca-cert-hash sha256:d21ca296b30091b304dfa03fb3b600e32eb67cef13c3f51badd835e25dfad1ba
W0510 08:59:12.753901 1121 join.go:346] [preflight] WARNING: JoinControlPane.controlPlane settings will be ignored when control-plane flag is not set.
I0510 08:59:12.754736 1121 join.go:371] [preflight] found NodeName empty; using OS hostname as NodeName
I0510 08:59:12.755402 1121 initconfiguration.go:103] detected and using CRI socket: /var/run/dockershim.sock
[preflight] Running pre-flight checks
I0510 08:59:12.756629 1121 preflight.go:90] [preflight] Running general checks
I0510 08:59:12.757284 1121 checks.go:249] validating the existence and emptiness of directory /etc/kubernetes/manifests
I0510 08:59:12.758224 1121 checks.go:286] validating the existence of file /etc/kubernetes/kubelet.conf
I0510 08:59:12.758814 1121 checks.go:286] validating the existence of file /etc/kubernetes/bootstrap-kubelet.conf
I0510 08:59:12.759434 1121 checks.go:102] validating the container runtime
I0510 08:59:12.854290 1121 checks.go:128] validating if the service is enabled and active
[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
I0510 08:59:13.009580 1121 checks.go:335] validating the contents of file /proc/sys/net/bridge/bridge-nf-call-iptables
I0510 08:59:13.009770 1121 checks.go:335] validating the contents of file /proc/sys/net/ipv4/ip_forward
I0510 08:59:13.009860 1121 checks.go:649] validating whether swap is enabled or not
I0510 08:59:13.009893 1121 checks.go:376] validating the presence of executable conntrack
I0510 08:59:13.009923 1121 checks.go:376] validating the presence of executable ip
I0510 08:59:13.009945 1121 checks.go:376] validating the presence of executable iptables
I0510 08:59:13.009963 1121 checks.go:376] validating the presence of executable mount
I0510 08:59:13.009980 1121 checks.go:376] validating the presence of executable nsenter
I0510 08:59:13.009995 1121 checks.go:376] validating the presence of executable ebtables
I0510 08:59:13.010011 1121 checks.go:376] validating the presence of executable ethtool
I0510 08:59:13.010029 1121 checks.go:376] validating the presence of executable socat
I0510 08:59:13.010044 1121 checks.go:376] validating the presence of executable tc
I0510 08:59:13.010062 1121 checks.go:376] validating the presence of executable touch
I0510 08:59:13.010093 1121 checks.go:520] running all checks
I0510 08:59:13.139333 1121 checks.go:406] checking whether the given node name is reachable using net.LookupHost
I0510 08:59:13.141594 1121 checks.go:618] validating kubelet version
I0510 08:59:13.237892 1121 checks.go:128] validating if the service is enabled and active
I0510 08:59:13.262185 1121 checks.go:201] validating availability of port 10250
I0510 08:59:13.262628 1121 checks.go:286] validating the existence of file /etc/kubernetes/pki/ca.crt
I0510 08:59:13.262712 1121 checks.go:432] validating if the connectivity type is via proxy or direct
I0510 08:59:13.262745 1121 join.go:441] [preflight] Discovering cluster-info
I0510 08:59:13.262774 1121 token.go:78] [discovery] Created cluster-info discovery client, requesting info from "k8smaster:6443"
I0510 08:59:13.277069 1121 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "8j77yo", will try again
I0510 08:59:13.283618 1121 token.go:221] [discovery] The cluster-info ConfigMap does not yet contain a JWS signature for token ID "8j77yo", will try again

looking at journalctl on the master, I can only see these errors in loop from the master:

May 10 09:24:56 master kubelet[26506]: E0510 09:24:56.925984 26506 kubelet.go:1663] Failed creating a mirror pod for "kube-apiserver-master_kube-system(5ee8922c9ad7b5a8f3f59abd72df0a78)": Timeout: request did not complete within requested timeout 34s
May 10 09:24:59 master kubelet[26506]: E0510 09:24:59.158408 26506 controller.go:136] failed to ensure node lease exists, will retry in 7s, error: proto: Lease: illegal tag -633754067 (wire type 29289705834)
May 10 09:25:06 master kubelet[26506]: E0510 09:25:06.167262 26506 controller.go:136] failed to ensure node lease exists, will retry in 7s, error: proto: Lease: illegal tag -633754067 (wire type 29289705834)
May 10 09:25:06 master kubelet[26506]: E0510 09:25:06.252399 26506 reflector.go:178] k8s.io/kubernetes/pkg/kubelet/config/apiserver.go:46: Failed to list *v1.Pod: proto: VolumeMount: wiretype end group for non-group

from the worker

May 10 09:26:42 worker systemd[1]: Configuration file /lib/systemd/system/kubelet.service is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.
May 10 09:26:42 worker systemd[1]: Configuration file /etc/systemd/system/kubelet.service.d/10-kubeadm.conf is marked world-inaccessible. This has no effect as configuration data is accessible via APIs without restrictions. Proceeding anyway.

Any help and advice to troubleshoot further would be appreciated

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Categories

Upcoming Training