Lab 3.2 kubeadm join request refused
Hello all,
I am following the instruction to do Lab 3.2 Grow the Cluster. I use my own laptop to setup the environment. Here are teh spec of my environment.
Host OS: Windows 10
VM Hypervisor: VirtualBox
Guest OS: Ubuntu 17.10
I setup two VMs. The master node is the one I regular used for my projects. The second one is dedicated for the project.
At step 5 of the Lab instruction, I cannot pass the settings.
Here I dump the info I capture for this lab.
Master node:
$ hmchen@Prj:ch03(k8s03)$ ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: enp0s3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
link/ether 08:00:27:c1:e5:84 brd ff:ff:ff:ff:ff:ff
inet 10.0.2.15/24 brd 10.0.2.255 scope global dynamic enp0s3
valid_lft 69118sec preferred_lft 69118sec
inet6 fe80::a9ce:981a:d8f:529c/64 scope link
valid_lft forever preferred_lft forever
3: br-2a405b0dbcdc: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:b3:b9:b1:a1 brd ff:ff:ff:ff:ff:ff
inet 172.19.0.1/16 scope global br-2a405b0dbcdc
valid_lft forever preferred_lft forever
4: br-4bb1837b90b1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:1f:16:7c:e3 brd ff:ff:ff:ff:ff:ff
inet 172.18.0.1/16 scope global br-4bb1837b90b1
valid_lft forever preferred_lft forever
5: docker0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default
link/ether 02:42:b2:41:94:1b brd ff:ff:ff:ff:ff:ff
inet 172.17.0.1/16 scope global docker0
valid_lft forever preferred_lft forever
6: flannel.1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UNKNOWN group default
link/ether f2:79:48:46:d8:60 brd ff:ff:ff:ff:ff:ff
inet 10.244.0.0/32 scope global flannel.1
valid_lft forever preferred_lft forever
inet6 fe80::f079:48ff:fe46:d860/64 scope link
valid_lft forever preferred_lft forever
7: cni0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue state UP group default qlen 1000
link/ether 0a:58:0a:f4:00:01 brd ff:ff:ff:ff:ff:ff
inet 10.244.0.1/24 scope global cni0
valid_lft forever preferred_lft forever
inet6 fe80::cc83:4fff:feda:d3a1/64 scope link
valid_lft forever preferred_lft forever
8: veth156a7dc9@if3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1450 qdisc noqueue master cni0 state UP group default
link/ether b2:69:f7:04:6a:20 brd ff:ff:ff:ff:ff:ff link-netnsid 0
inet6 fe80::b069:f7ff:fe04:6a20/64 scope link
valid_lft forever preferred_lft forever
$ sudo kubeadm token list
[sudo] password for hmchen:
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
44d148.2a750f5fe1f90105 23h 2018-03-28T12:09:11-07:00 authentication,signing <none> system:bootstrappers:kubeadm:default-node-token
ba3edb.1b9344bf6ce19d91 29m 2018-03-27T13:17:26-07:00 authentication,signing The default bootstrap token generated by 'kubeadm init'. system:bootstrappers:kubeadm:default-node-token
$ openssl x509 -pubkey -in /etc/kubernetes/pki/ca.crt | openssl rsa \
-pubin -outform der 2>/dev/null | openssl dgst \
-sha256 -hex | sed 's/^.* //'
34e555e3f502b0b0aa0786ea841fa8b62daa66569863d62adc34f8e72dc3da41
Second Node:
$ ping 10.0.2.15 ==> master node reachable
# kubeadm join --ignore-preflight-errors=cri --token 44d148.2a750f5fe1f90105 10.0.2.15:6443 --discovery-token-ca-cert-hash sha256:34e555e3f502b0b0aa0786ea841fa8b62daa66569863d62adc34f8e72dc3da41
[preflight] Running pre-flight checks.
[WARNING Service-Docker]: docker service is not enabled, please run 'systemctl enable docker.service'
[WARNING FileExisting-crictl]: crictl not found in system path
[discovery] Trying to connect to API Server "10.0.2.15:6443"
[discovery] Created cluster-info discovery client, requesting info from "https://10.0.2.15:6443"
[discovery] Failed to request cluster info, will try again: [Get https://10.0.2.15:6443/api/v1/namespaces/kube-public/configmaps/cluster-info: dial tcp 10.0.2.15:6443: getsockopt: connection refused]
[discovery] Failed to request cluster info, will try again: [Get https://10.0.2.15:6443/api/v1/namespaces/kube-public/configmaps/cluster-info: dial tcp 10.0.2.15:6443: getsockopt: connection refused]
I have tried other interfaces but none working even with ping.
Comments
-
Hi, while I have not tried yet doing the labs on local VMs, I can only think of a networking issue between the 2 nodes:
1 - is your master listening and accepting traffic on port 6443? The reason I asked is because you mentioned you have used this node for other projects as well.
2 - can you run successfully a telnet or nc from the second node with the master IP and port 6443?
Good luck!
-Chris
0 -
Hi Fred, although a good learning experience, it is unfortunate that you have to go through all this trouble to setup your environment in order to complete the labs. There was another post in this forum about a similar issue (Ubuntu VMs on a local windows workstation), which did not get resolved. The successful alternative, however, was to complete these labs in the cloud, either on GCP or AWS. They offer 1 free year, and I would think that Azure has a similar offer. I have completed most labs by now on GCP and I only ran into a minor firewall issue, which I resolved with a few simple commands.
Good luck!
-Chris
0 -
Hi Fred, @hmchen47
I was able to run thru lab 3 by setting up Bridged networking adapter on the master and minion VMs, and I posted some of the steps and my findings in the topic below:
https://www.linux.com/forums/lfs258-class-forum/lab-3233-master-node-network-problem
Regards,
-Chris
0 -
I thought it was really interesting that, although I created the worker as essentially a clone of the master by choosing "create similar" in GCP per the instructions, installing docker on it didn't enable it. Here were the instructions on Lab 3.2:
IF you chose Docker on the master:
root@worker: ̃# apt-get install -y docker.io...And here's what I did:
root@lfs258-worker:~# apt-get install -y docker.io Reading package lists... Done Building dependency tree Reading state information... Done The following package was automatically installed and is no longer required: libnuma1 Use 'apt autoremove' to remove it. The following additional packages will be installed: bridge-utils cgroupfs-mount containerd pigz runc ubuntu-fan Suggested packages: ifupdown aufs-tools debootstrap docker-doc rinse zfs-fuse | zfsutils The following NEW packages will be installed: bridge-utils cgroupfs-mount containerd docker.io pigz runc ubuntu-fan 0 upgraded, 7 newly installed, 0 to remove and 0 not upgraded. Need to get 63.7 MB of archives. After this operation, 319 MB of additional disk space will be used. Get:1 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 pigz amd64 2.4-1 [57.4 kB] Get:2 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 bridge-utils amd64 1.5-15ubuntu1 [30.1 kB] Get:3 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic/universe amd64 cgroupfs-mount all 1.4 [6320 B] Get:4 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 runc amd64 1.0.0~rc10-0ubuntu1~18.04.2 [2000 kB] Get:5 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 containerd amd64 1.3.3-0ubuntu1~18.04.2 [21.7 MB] Get:6 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic-updates/universe amd64 docker.io amd64 19.03.6-0ubuntu1~18.04.2 [39.9 MB] Get:7 http://us-west1.gce.archive.ubuntu.com/ubuntu bionic/main amd64 ubuntu-fan all 0.12.10 [34.7 kB] Fetched 63.7 MB in 1s (56.7 MB/s) Preconfiguring packages ... Selecting previously unselected package pigz. (Reading database ... 65498 files and directories currently installed.) Preparing to unpack .../0-pigz_2.4-1_amd64.deb ... Unpacking pigz (2.4-1) ... Selecting previously unselected package bridge-utils. Preparing to unpack .../1-bridge-utils_1.5-15ubuntu1_amd64.deb ... Unpacking bridge-utils (1.5-15ubuntu1) ... Selecting previously unselected package cgroupfs-mount. Preparing to unpack .../2-cgroupfs-mount_1.4_all.deb ... Unpacking cgroupfs-mount (1.4) ... Selecting previously unselected package runc. Preparing to unpack .../3-runc_1.0.0~rc10-0ubuntu1~18.04.2_amd64.deb ... Unpacking runc (1.0.0~rc10-0ubuntu1~18.04.2) ... Selecting previously unselected package containerd. Preparing to unpack .../4-containerd_1.3.3-0ubuntu1~18.04.2_amd64.deb ... Unpacking containerd (1.3.3-0ubuntu1~18.04.2) ... Selecting previously unselected package docker.io. Preparing to unpack .../5-docker.io_19.03.6-0ubuntu1~18.04.2_amd64.deb ... Unpacking docker.io (19.03.6-0ubuntu1~18.04.2) ... Selecting previously unselected package ubuntu-fan. Preparing to unpack .../6-ubuntu-fan_0.12.10_all.deb ... Unpacking ubuntu-fan (0.12.10) ... Setting up runc (1.0.0~rc10-0ubuntu1~18.04.2) ... Setting up cgroupfs-mount (1.4) ... Setting up containerd (1.3.3-0ubuntu1~18.04.2) ... Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /lib/systemd/system/containerd.service. Setting up bridge-utils (1.5-15ubuntu1) ... Setting up ubuntu-fan (0.12.10) ... Created symlink /etc/systemd/system/multi-user.target.wants/ubuntu-fan.service → /lib/systemd/system/ubuntu-fan.service. Setting up pigz (2.4-1) ... Setting up docker.io (19.03.6-0ubuntu1~18.04.2) ... Adding group `docker' (GID 116) ... Done. Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /lib/systemd/system/docker.socket. docker.service is a disabled or a static unit, not starting it. Processing triggers for systemd (237-3ubuntu10.43) ... Processing triggers for man-db (2.8.3-2ubuntu0.1) ... Processing triggers for ureadahead (0.100.0-21) ...
Of note is the tiny notice near the end, "docker.service is a disabled or a static unit, not starting it." Odd that this would happen on the worker but not the control plane.
So I enabled it:
root@lfs258-worker:~# systemctl enable docker.service Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /lib/systemd/system/docker.service.
That seemed to work.
0
Categories
- All Categories
- 177 LFX Mentorship
- 177 LFX Mentorship: Linux Kernel
- 750 Linux Foundation IT Professional Programs
- 373 Cloud Engineer IT Professional Program
- 169 Advanced Cloud Engineer IT Professional Program
- 74 DevOps IT Professional Program - Discontinued
- 4 DevOps & GitOps IT Professional Program
- 99 Cloud Native Developer IT Professional Program
- 7.6K Training Courses & Learning Paths
- 1 AI & ML Training
- 1 Blockchain & Decentralized Identity Training
- 4 Cloud & Containers Training
- 1 Cybersecurity Training
- 2 DevOps & Site-Reliability Training
- 1 Linux Kernel Development Training
- 1 Networking Training
- 2 Open Source Best Practice Training
- 1 System Administration Training
- 1 System Engineering Training
- 1 Web & Application Development Training
- 792 Hardware
- 202 Drivers
- 68 I/O Devices
- 37 Monitors
- 95 Multimedia
- 173 Networking
- 91 Printers & Scanners
- 87 Storage
- 769 Linux Distributions
- 81 Debian
- 68 Fedora
- 22 Linux Mint
- 13 Mageia
- 24 openSUSE
- 150 Red Hat Enterprise
- 31 Slackware
- 13 SUSE Enterprise
- 356 Ubuntu
- 465 Linux System Administration
- 31 Cloud Computing
- 73 Command Line/Scripting
- Github systems admin projects
- 98 Linux Security
- 78 Network Management
- 101 System Management
- 46 Web Management
- 106 Mobile Computing
- 18 Android
- 73 Development
- 1.2K New to Linux
- 1K Getting Started with Linux
- 392 Off Topic
- 121 Introductions
- 181 Small Talk
- 29 Study Material
- 955 Programming and Development
- 310 Kernel Development
- 627 Software Development
- 983 Software
- 375 Applications
- 182 Command Line
- 5 Compiling/Installing
- 68 Games
- 317 Installation
- Archived
- 2 LFD140 Class Forum
- 1.4K LFS258 Class Forum
Upcoming Training
-
August 20, 2018
Kubernetes Administration (LFS458)
-
August 20, 2018
Linux System Administration (LFS301)
-
August 27, 2018
Open Source Virtualization (LFS462)
-
August 27, 2018
Linux Kernel Debugging and Security (LFD440)