Welcome to the Linux Foundation Forum!

Use DNS names vs IP for etcd and kube-apiserver? (re: Lab 3.1)

Options
rilindo
rilindo Posts: 9
in LFS258 Class Forum

It seems that when you install k8, etcd and kube-apiserver is configured to use the IP of the k8 master. That is an issue to me as I do tear down my environment and start it back up.

Is there a way to tell to use the hostname/dns name of the master rather than the IP?

Comments

  • serewicz
    serewicz Posts: 1,000
    Options

    Hello,

    The x509 certificates will only work with the server name passed to kubeadm init. I would suggest starting over and following the entire directions, where you configure the k8smaster alias and use it. This will become essential when you try to get HA to work.

    Regards,

  • chrispokorni
    chrispokorni Posts: 2,177
    Options

    Hi @rilindo,

    In some environments the VMs get to keep their private IP addresses between restarts. In others, you may assign static IP addresses to your VMs.

    Are you using a cloud or local infrastructure?

    Regards,
    -Chris

  • rilindo
    rilindo Posts: 9
    Options

    @serewicz This is how I configure things, per instructions:

    root@k8smaster:~# cat kubeadm-config.yaml 
apiVersion: kubeadm.k8s.io/v1beta2
kind: ClusterConfiguration
kubernetesVersion: stable
controlPlaneEndpoint: "k8smaster:6443"
networking:
  podSubnet: 192.168.0.0/16
root@k8smaster:~# cat /etc/hosts
# Your system has configured 'manage_etc_hosts' as True.
# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.debian.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data
#
127.0.0.1 localhost

# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts

10.70.60.49 k8smaster k8smaster

/usr/bin/kubeadm init --config=/root/kubeadm-config.yaml --upload-certs |  tee /root/kubeadm-init.out

    Essentially, what it says in the lab

    @chrispokorni I am using scaleway as a cloud provider and they don't allow you to assign IP addresses manually. :neutral:

    I am not against having to use Tier 1 provider like AWS (since they provide that capability), but I am hoping that it doesn't come to that.

  • serewicz
    serewicz Posts: 1,000
    edited September 2020
    Options

    Hello,

    Could you indicate where you see it using the IP instead of the k8smaster alias?

    Could you show grep server .kube/config output please.

    Regards,

  • serewicz
    serewicz Posts: 1,000
    Options

    Hello,
    Also what operating system and cloud provider or host are you using?

    I note this in your output:

    Your system has configured 'manage_etc_hosts' as True.

# As a result, if you wish for changes to this file to persist
# then you will need to either
# a.) make changes to the master file in /etc/cloud/templates/hosts.debian.tmpl
# b.) change or remove the value of 'manage_etc_hosts' in
#     /etc/cloud/cloud.cfg or cloud-config from user-data

    Regards,

  • rilindo
    rilindo Posts: 9
    Options

    @serewicz Here is my kube config:

    gstudent@k8smaster:~$ grep server .kube/config 
    server: https://k8smaster:6443
student@k8smaster:~$

    The IPs in question are located here:

    root@k8smaster:~# find /etc/ -type f -exec grep -l 10.68.68.191 {} \;
/etc/hosts
/etc/kubernetes/manifests/etcd.yaml
/etc/kubernetes/manifests/kube-apiserver.yaml

    Also:

    Also what operating system and cloud provider or host are you using?

    I am using Ubuntu Bionic on Scaleway.

    root@k8smaster:~# uname -a
Linux k8smaster 4.15.0-115-generic #116-Ubuntu SMP Wed Aug 26 14:04:49 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
root@k8smaster:~# cat /etc/lsb-release 
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=18.04
DISTRIB_CODENAME=bionic
DISTRIB_DESCRIPTION="Ubuntu 18.04.3 LTS"

    And

    I note this in your output:

    Scaleway uses cloud-init, which does some configuration inject at start. I had it off and turn it back on (since I am using ansible to reconfigure the the host file):

      - /usr/bin/ansible 127.0.0.1 -m lineinfile -a 'dest=/etc/hosts regexp=".*k8smaster.*" line="{{ ipaddress }} k8smaster k8smaster" state=present' -e ipaddress=$(ifdata -pa ens2)
  • rilindo
    rilindo Posts: 9
    Options

    Alright, I bit the bullet and switch over to AWS, which allowed me to set a static private IP. That said. . . . maybe I should try a different approach, since it seems that AWS behavior on static private IPs seems to be inconsistent.

    If the the master somehow start up with a different IP, is it possible to re-run kubeadm init to rebuild the configuration with a different IP?

  • chrispokorni
    chrispokorni Posts: 2,177
    Options

    I have been using Google Cloud GCE VM instances for my Kubernetes clusters for a few years now, and have not encountered any issues when re-starting my clusters. That is probably because GCE keeps the private IPs static in my custom VPC network.

    There is a kubeadm reset command you could run on master and worker nodes if this is something you desire, but the command itself is not perfect, as it leaves behind configuration options which need to be cleaned up manually post-reset, otherwise you will not be able to seamlessly re-join your nodes in a cluster. Additional cleanup steps are described here.

    However, unless this process is scripted/automated, I find this to be a counterproductive approach.

    Regards,
    -Chris

  • rilindo
    rilindo Posts: 9
    Options

    @chrispokorni Sounds like at some point I may need to start using Google Cloud GCE to get some sort of first class support for Kubenetes. FWIW, after some refactoring my lab (enlarging my CIDR to avoid IP address complexity), the AWS setup seems to be working out well, so I will continue on that as of this time.

    Thanks to y'all for your help!

Categories

Upcoming Training