Welcome to the Linux Foundation Forum!

Creating VM nodes on QEMU/KVM with Ubuntu

Posts: 15
edited August 2022 in LFS258 Class Forum

If you have a decent multi-core CPU with enough RAM running Ubuntu there's no reason not to use it for this course instead of a cloud service.

You should already have QEMU/KVM setup and a ssh key created using ssh-keygen -t rsa.

First thing you need to do is make sure you have a virtual bridge. In Ubuntu 22.04 you should already have one called virbr0.

  1. $ ip link show type bridge
  2. $ ip addr show virbr0
  3. 5: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN group default qlen 1000
  4. link/ether 52:54:00:c3:12:65 brd ff:ff:ff:ff:ff:ff
  5. inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
  6. valid_lft forever preferred_lft forever

You can see our static address range is 192.168.122.1/24. So let's add some host names to our /etc/hosts

  1. 192.168.122.191 mycluster-cp1
  2. 192.168.122.192 mycluster-cp2
  3. 192.168.122.193 mycluster-cp3
  4. 192.168.122.194 mycluster-w1
  5. 192.168.122.195 mycluster-w2
  6. 192.168.122.196 mycluster-w3
  7. 192.168.122.197 mycluster-w4

Create a file called user-data.yml and add the following:

  1. #cloud-config
  2. autoinstall:
  3. version: 1
  4. interactive-sections: []
  5. ssh:
  6. install-server: true
  7. allow-pw: true
  8. authorized-keys:
  9. - [SSH_RSA]
  10. user-data:
  11. disable_root: false
  12. identity:
  13. hostname: ubuntu-server
  14. username: ubuntu
  15. password: [ROOT_PASSWORD]
  16. early-commands: []
  17. late-commands:
  18. - swapoff -a
  19. - sed -i '/swap/ s/^\(.*\)$/#\1/g' /target/etc/fstab
  20. - echo 'ubuntu ALL=(ALL) NOPASSWD:ALL' > /target/etc/sudoers.d/ubuntu
  21. - chmod 440 /target/etc/sudoers.d/ubuntu

Paste in your ssh key in [SSH_RSA]:

  1. cat ~/.ssh/id_rsa.pub

Change the user@host to ubuntu@ubuntu-server

To generate a password for root you can run and paste it in the [ROOT_PASSWORD] section:

  1. mkpasswd --method=SHA-512

Now run ubuntu-autoinstall-generator.sh to generate the Ubuntu 22.04 iso:

  1. ./ubuntu-autoinstall-generator.sh -a -u user-data.yml -d ubuntu-autoinstall.iso

Now we use this virt-install.sh bash script to generate our VM's:

  1. #!/bin/bash
  2.  
  3. HOSTNAME=$1
  4. CLIENTIP=$2
  5. VCPUS=${3:-2}
  6. MEMORY=${4:-2048}
  7. DISKSIZE=${5:-30}
  8. DISKPATH=./$HOSTNAME.rawdisk
  9. MACADDR=RANDOM
  10. DEVICE=enp3s0
  11. AUTOCONF=off
  12. BRIDGE=virbr0
  13. SERVERIP=
  14. DNS0IP=192.168.122.1
  15. DNS1IP=
  16. GATEWAYIP=192.168.122.1
  17. NETMASK=255.255.255.0
  18. LOCATION=./ubuntu-autoinstall.iso
  19.  
  20. sudo virt-install \
  21. --connect=qemu:///system \
  22. --name $HOSTNAME \
  23. --memory $MEMORY \
  24. --vcpus $VCPUS \
  25. --bridge=$BRIDGE \
  26. --mac=$MACADDR \
  27. --autostart \
  28. --check-cpu \
  29. --os-type=linux \
  30. --force \
  31. --graphics none \
  32. --virt-type kvm \
  33. --os-variant=ubuntu22.04 \
  34. --location $LOCATION,initrd=casper/initrd,kernel=casper/vmlinuz \
  35. --disk path=$DISKPATH,format=raw,cache=none,bus=virtio,size=$DISKSIZE \
  36. --debug \
  37. --noautoconsole \
  38. --wait=-1 \
  39. --extra-args="ip=$CLIENTIP:$SERVERIP:$GATEWAYIP:$NETMASK:$HOSTNAME:$DEVICE:$AUTOCONF:$DNS0IP:$DNS1IP console=ttyS0 quiet autoinstall ds=nocloud;s=/cdrom/nocloud/"

Change the value of DEVICE to match your ethernet device:

  1. $ hostname -I
  2. 192.168.0.141
  3. $ ifconfig | grep 192.168.0.141 -a -3
  4. TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
  5.  
  6. enp3s0: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
  7. inet 192.168.0.141 netmask 255.255.255.0 broadcast 192.168.0.255
  8. inet6 fe80::5a5:e6a5:9aee:9e43 prefixlen 64 scopeid 0x20<link>
  9. ether a8:a1:59:40:ca:d4 txqueuelen 1000 (Ethernet)
  10. RX packets 45472938 bytes 16695796477 (16.6 GB)

So we should set:

  1. DEVICE=enp3s0

Now we're finally ready to generate our VMs'!

  1. #!/bin/bash
  2. SCRIPT=./virt-install.sh
  3.  
  4. $SCRIPT mycluster-cp1 192.168.122.191 2 2048 30
  5. $SCRIPT mycluster-cp2 192.168.122.192 2 2048 30
  6. $SCRIPT mycluster-cp3 192.168.122.193 2 2048 30
  7. $SCRIPT mycluster-w1 192.168.122.194 1 2048 30
  8. $SCRIPT mycluster-w2 192.168.122.195 1 2048 30
  9. $SCRIPT mycluster-w3 192.168.122.196 1 2048 30
  10. $SCRIPT mycluster-w4 192.168.122.197 1 2048 30

If you only want two control planes' and two workers, for example, comment out the mycluster-cp3, mycluster-w3 and mycluster-w4 lines by prefixing them with a # character.

Once you have your VMs' created you should ssh into each one and set the correct host name and edit the /etc/hosts file to add the same hosts as we did before.

  1. $ sudo nano /etc/hostname
  2. mycluster-x
  3. $ sudo nano /etc/hosts

Comments

  • Posts: 15
    edited August 2022

    I just commit a k8s-cluster repo which contains Ansible scripts for automating the creation of a k8s cluster on your VMs.

    It should be as simple as editing hosts.ini

    1. [control_plane]
    2. mycluster-cp1 ansible_host=192.168.122.191
    3. mycluster-cp2 ansible_host=192.168.122.192
    4. mycluster-cp3 ansible_host=192.168.122.193
    5.  
    6. [workers]
    7. mycluster-w1 ansible_host=192.168.122.194
    8. mycluster-w2 ansible_host=192.168.122.195
    9. mycluster-w3 ansible_host=192.168.122.196
    10.  
    11. [all:vars]
    12. ansible_python_interpreter=/usr/bin/python3

    Then run the install.sh script

    1. #!/bin/bash
    2. set -e
    3. ansible-playbook -i hosts.ini ./init.yml
    4. ansible-playbook -i hosts.ini ./kube-dependencies.yml
    5. ansible-playbook -i hosts.ini ./control-planes.yml
    6. ansible-playbook -i hosts.ini ./workers.yml

    By default it will install containerd CRI but aslo includes scripts for installing cri-o and docker.

  • Posts: 15
    edited March 2023

    The repo has been updated to use Vagrant instead of cloud-config

    k8s-cluster

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Welcome!

It looks like you're new here. Sign in or register to get started.
Sign In

Categories

Upcoming Training