Routing problem on VMs LAB 11.2

k0dard

Hello,

I've done this lab exercise using my laptop and raspberry pi, and everything worked fine. However, when I try to do it on VMs, it just doesn't work.

Here's my setting:

First VM IPs

    inet 192.168.122.223/24 brd 192.168.122.255 scope global dynamic ens3
    inet 192.168.1.100/24 scope global ens3

First VM routing table

default via 192.168.122.1 dev ens3 proto dhcp src 192.168.122.223 metric 100 
172.16.1.0/24 via 192.168.122.98 dev ens3 
192.168.1.0/24 dev ens3 proto kernel scope link src 192.168.1.100 
192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.223 
192.168.122.1 dev ens3 proto dhcp scope link src 192.168.122.223 metric 100

Second VM IPs

    inet 192.168.122.98/24 brd 192.168.122.255 scope global dynamic ens3
    inet 172.16.1.100/24 scope global ens3

Second VM routing table

    default via 192.168.122.1 dev ens3 proto dhcp src 192.168.122.98 metric 100 
    172.16.1.0/24 dev ens3 proto kernel scope link src 172.16.1.100 
    192.168.1.0/24 via 192.168.122.223 dev ens3 
    192.168.122.0/24 dev ens3 proto kernel scope link src 192.168.122.98 
    192.168.122.1 dev ens3 proto dhcp scope link src 192.168.122.98 metric 100

What am I doing wrong ?

Thanks in advance for your help

codmd

Hi in your first VM GW = 192.168.122.98 ?
second VM GW = 192.168.122.223 ? --- two VM use 2 GW different addresses ?

k0dard

Hello codmd,

In order to reach 192.168.1.100 on VM1 the traffic from VM2 has to be routed through 192.168.122.223

192.168.122.98 -> 192.168.122.223 -> 192.168.1.100

And to reach 176.16.1.100 from VM1

192.168.122.223 -> 192.168.122.98 -> 176.16.1.100

lee42x

Thank you for your input.

There is a note at the top of the lab indicating that more than one machine is best for routing exercises,however, the lab should function. Ii will reproduce this lab and post the solution.

Which VM hypervisor are you using?

Lee

lee42x

Please verify that ip_forwarding is enabled on all the machines.

sysctl net.ipv4.ip_forward

k0dard

Hello Lee,

Thanks in advance for your help!

I am using KVM.

I confirm that ip_forwarding is enabled on both machines.

lee42x

The base machine also had ip_forwarding on?
Your configuration looks ok. Something else is causing issues, perhaps a firewall on the main machine?

k0dard

Sorry, by the base machine you mean my host which is running KVM hypervisor ? If that is the case, ip_forwarding is on

Also, the firewalld and ufw services on my host are stopped. If firewall services are stopped, iptables has no effect, right ?

Just in case, here's the relevant part of my FORWARD chain on the host machine

0 0 ACCEPT all -- virbr1 virbr1 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr1 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
0 0 REJECT all -- virbr1 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
134K 174M ACCEPT all -- * virbr0 0.0.0.0/0 192.168.122.0/24 ctstate RELATED,ESTABLISHED
30737 2066K ACCEPT all -- virbr0 * 192.168.122.0/24 0.0.0.0/0
24 7449 ACCEPT all -- virbr0 virbr0 0.0.0.0/0 0.0.0.0/0
0 0 REJECT all -- * virbr0 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable
0 0 REJECT all -- virbr0 * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-port-unreachable

Really appreciate your help !

lee42x

It depends if you are using iptables or nft with systemd.
Please shutdown the firewall and verify there is no rules. The flush option should clean out the rules.

lee42x

On the three machines, Hypervisor,first and second, which of them have the libvirtd service running?

k0dard

Hello Lee,

Sorry for the delay.

I cannot flush iptables because I have a lot of other routing options and filters set on my laptop so it seems potentially dangerous (?) That is why I asked if it suffice to shutdown ufw.service

libvirtd is running on hypervisor machine

lee42x

Ok, I see. How about creating a new “private” network we can experiment on? The idea is only the vms can use this network, and assign ipaddresses manually through the hyper visor management. I do this a lot just to isolate the traffic so wireshark is less busy. Stay away from the default nat network (192.168.122.0/24 and only use it for outside access.

k0dard

Hello Lee,

I've added a new private network 10.11.12.0/24 so now I have another adapter (ens9) on each VM with address 10.11.12.77 on VM1 and 10.11.12.99 on VM2.

After adding routes through new interfaces, the machines can ping each other and everything works as expected.

I'm really puzzled... Is there a way to find out why it wasn't working in the first place (or an explication at least) ?

Thanks a lot for your help !

k0dard

lee42x

The software bridges control the paths the packets take, the default is set so packets are not returned on the same path that they arrived on. This is an option called "hairpin" and the default is off. Rather than messing about with the primary bridge, using another bridge with the option already on is easier.
Regards Lee

k0dard

Cool, thanks for the explanation !