[libvirt-users] Setting up port forwarding to guests on nat network

[Rhys Ferris]

Hello all,

I’m currently trying to figure out how to forward ports to guests that are on a NAT Network. I have followed the directions on https://wiki.libvirt.org/page/Networking under the “Forwarding Incoming Connections” Section and get connection refused when attempting to connect.

System: Ubuntu Server 18.04.1
Virsh / LibVirtd Version: 4.0.0

Here’s the contents of /etc/libvirt/hooks/qemu
 
#!/bin/bash
 
# IMPORTANT: Change the "VM NAME" string to match your actual VM Name.
# In order to create rules to other VMs, just duplicate the below block and configure
# it accordingly.
if [ "${1}" = "testy" ]; then
 
   # Update the following variables to fit your setup
   GUEST_IP='10.128.10.100'
   GUEST_PORT='22'
   HOST_PORT='2588'
 
   if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
        /sbin/iptables -D FORWARD -o virbr0 -d  $GUEST_IP -j ACCEPT
        /sbin/iptables -t nat -D PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
   fi
   if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
        /sbin/iptables -I FORWARD -o virbr0 -d  $GUEST_IP -j ACCEPT
        /sbin/iptables -t nat -I PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
   fi
fi


Here’s my network XML
<network>
  <name>olympus</name>
  <uuid>3b0d968c-8166-42f7-8109-e5f0317cab42</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr1' stp='on' delay='0'/>
  <mac address='52:54:00:bb:18:6b'/>
  <ip address='10.128.10.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.128.10.2' end='10.128.10.254'/>
      <host mac='52:54:00:8d:f5:0c' name='testy' ip='10.128.10.100'/>
    </dhcp>
  </ip>
</network>

And here’s the results of iptables -L -vt nat:
BEFORE VM BOOT:
Chain PREROUTING (policy ACCEPT 46615 packets, 6618K bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain INPUT (policy ACCEPT 46615 packets, 6618K bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain OUTPUT (policy ACCEPT 198K packets, 18M bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain POSTROUTING (policy ACCEPT 198K packets, 18M bytes)
 pkts bytes target     prot opt in     out     source               destination
   24  1812 RETURN     all  --  any    any     10.128.10.0/24       base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     10.128.10.0/24       255.255.255.255
   17  1020 MASQUERADE  tcp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
   15  1700 MASQUERADE  udp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     10.128.10.0/24      !10.128.10.0/24
   22  1666 RETURN     all  --  any    any     192.168.122.0/24     base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     192.168.122.0/24     255.255.255.255
    0     0 MASQUERADE  tcp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    8  1168 MASQUERADE  udp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     192.168.122.0/24    !192.168.122.0/24
 
 
AFTER VM BOOT
Chain PREROUTING (policy ACCEPT 2 packets, 120 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:2588 to:10.128.10.100:22
 
Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain OUTPUT (policy ACCEPT 18 packets, 1263 bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain POSTROUTING (policy ACCEPT 18 packets, 1263 bytes)
 pkts bytes target     prot opt in     out     source               destination
   24  1812 RETURN     all  --  any    any     10.128.10.0/24       base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     10.128.10.0/24       255.255.255.255
   17  1020 MASQUERADE  tcp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
   15  1700 MASQUERADE  udp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     10.128.10.0/24      !10.128.10.0/24
   22  1666 RETURN     all  --  any    any     192.168.122.0/24     base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     192.168.122.0/24     255.255.255.255
    0     0 MASQUERADE  tcp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    8  1168 MASQUERADE  udp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     192.168.122.0/24    !192.168.122.0/24

And lastly heres what actually happens on attempt to SSH:
***@odin:~$ ssh ***@172.16.99.170 -p 2258
ssh: connect to host 172.16.99.170 port 2258: Connection refused
***@odin:~$

The connection refused is instant, not a timeout.

I’ve ensured that ufw is disabled.

Any help appreciated. I just can’t figure this out.

Sent from Mail for Windows 10

[Martin Kletzander]

Post by Rhys Ferris
Hello all,
I’m currently trying to figure out how to forward ports to guests that are on a NAT Network. I have followed the directions on https://wiki.libvirt.org/page/Networking under the “Forwarding Incoming Connections” Section and get connection refused when attempting to connect.
System: Ubuntu Server 18.04.1
Virsh / LibVirtd Version: 4.0.0
Here’s the contents of /etc/libvirt/hooks/qemu
 
#!/bin/bash
 
# IMPORTANT: Change the "VM NAME" string to match your actual VM Name.
# In order to create rules to other VMs, just duplicate the below block and configure
# it accordingly.
if [ "${1}" = "testy" ]; then
 
   # Update the following variables to fit your setup
   GUEST_IP='10.128.10.100'
   GUEST_PORT='22'
   HOST_PORT='2588'
 
   if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
        /sbin/iptables -D FORWARD -o virbr0 -d  $GUEST_IP -j ACCEPT
        /sbin/iptables -t nat -D PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT
   fi
   if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
        /sbin/iptables -I FORWARD -o virbr0 -d  $GUEST_IP -j ACCEPT
        /sbin/iptables -t nat -I PREROUTING -p tcp --dport $HOST_PORT -j DNAT --to $GUEST_IP:$GUEST_PORT

I would do the rules differently, but since it is on the wiki, I'll believe it
works. You probably checked, but just to make sure, what is the output of
`sysctl net.ipv4.ip_forward` ?

Post by Rhys Ferris
   fi
fi
Here’s my network XML
<network>
  <name>olympus</name>
  <uuid>3b0d968c-8166-42f7-8109-e5f0317cab42</uuid>
  <forward mode='nat'>
    <nat>
      <port start='1024' end='65535'/>
    </nat>
  </forward>
  <bridge name='virbr1' stp='on' delay='0'/>
  <mac address='52:54:00:bb:18:6b'/>
  <ip address='10.128.10.1' netmask='255.255.255.0'>
    <dhcp>
      <range start='10.128.10.2' end='10.128.10.254'/>
      <host mac='52:54:00:8d:f5:0c' name='testy' ip='10.128.10.100'/>
    </dhcp>
  </ip>
</network>
Chain PREROUTING (policy ACCEPT 46615 packets, 6618K bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain INPUT (policy ACCEPT 46615 packets, 6618K bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain OUTPUT (policy ACCEPT 198K packets, 18M bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain POSTROUTING (policy ACCEPT 198K packets, 18M bytes)
 pkts bytes target     prot opt in     out     source               destination
   24  1812 RETURN     all  --  any    any     10.128.10.0/24       base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     10.128.10.0/24       255.255.255.255
   17  1020 MASQUERADE  tcp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
   15  1700 MASQUERADE  udp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     10.128.10.0/24      !10.128.10.0/24
   22  1666 RETURN     all  --  any    any     192.168.122.0/24     base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     192.168.122.0/24     255.255.255.255
    0     0 MASQUERADE  tcp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    8  1168 MASQUERADE  udp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     192.168.122.0/24    !192.168.122.0/24
 
 
AFTER VM BOOT
Chain PREROUTING (policy ACCEPT 2 packets, 120 bytes)
 pkts bytes target     prot opt in     out     source               destination
    0     0 DNAT       tcp  --  any    any     anywhere             anywhere             tcp dpt:2588 to:10.128.10.100:22
 

The out:any and

Post by Rhys Ferris
Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain OUTPUT (policy ACCEPT 18 packets, 1263 bytes)
 pkts bytes target     prot opt in     out     source               destination
 
Chain POSTROUTING (policy ACCEPT 18 packets, 1263 bytes)
 pkts bytes target     prot opt in     out     source               destination
   24  1812 RETURN     all  --  any    any     10.128.10.0/24       base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     10.128.10.0/24       255.255.255.255
   17  1020 MASQUERADE  tcp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
   15  1700 MASQUERADE  udp  --  any    any     10.128.10.0/24      !10.128.10.0/24       masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     10.128.10.0/24      !10.128.10.0/24
   22  1666 RETURN     all  --  any    any     192.168.122.0/24     base-address.mcast.net/24
    0     0 RETURN     all  --  any    any     192.168.122.0/24     255.255.255.255
    0     0 MASQUERADE  tcp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    8  1168 MASQUERADE  udp  --  any    any     192.168.122.0/24    !192.168.122.0/24     masq ports: 1024-65535
    0     0 MASQUERADE  all  --  any    any     192.168.122.0/24    !192.168.122.0/24
ssh: connect to host 172.16.99.170 port 2258: Connection refused
The connection refused is instant, not a timeout.
I’ve ensured that ufw is disabled.
Any help appreciated. I just can’t figure this out.
Sent from Mail for Windows 10
_______________________________________________
libvirt-users mailing list
https://www.redhat.com/mailman/listinfo/libvirt-users

[Rhys Ferris]

Thanks for the reply!
output:
net.ipv4.ip_forward = 1

What do you mean "The out:any and"

Anywhere else I can look as to why the connection isn't going?
Do I need some kind of listener at that port on the host? I'm not even
seeing the packet count on the prerouting chain increase when the
connection attempts are made.

Post by Rhys Ferris

Post by Rhys Ferris
Hello all,
I’m currently trying to figure out how to forward ports to guests that

are on a NAT Network. I have followed the directions on
https://wiki.libvirt.org/page/Networking under the “Forwarding Incoming
Connections” Section and get connection refused when attempting to connect.

Post by Rhys Ferris
System: Ubuntu Server 18.04.1
Virsh / LibVirtd Version: 4.0.0
Here’s the contents of /etc/libvirt/hooks/qemu
#!/bin/bash
# IMPORTANT: Change the "VM NAME" string to match your actual VM Name.
# In order to create rules to other VMs, just duplicate the below block

and configure

Post by Rhys Ferris
# it accordingly.
if [ "${1}" = "testy" ]; then
# Update the following variables to fit your setup
GUEST_IP='10.128.10.100'
GUEST_PORT='22'
HOST_PORT='2588'
if [ "${2}" = "stopped" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -D FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -D PREROUTING -p tcp --dport $HOST_PORT -j

DNAT --to $GUEST_IP:$GUEST_PORT

Post by Rhys Ferris
fi
if [ "${2}" = "start" ] || [ "${2}" = "reconnect" ]; then
/sbin/iptables -I FORWARD -o virbr0 -d $GUEST_IP -j ACCEPT
/sbin/iptables -t nat -I PREROUTING -p tcp --dport $HOST_PORT -j

DNAT --to $GUEST_IP:$GUEST_PORT
I would do the rules differently, but since it is on the wiki, I'll believe it
works. You probably checked, but just to make sure, what is the output of
`sysctl net.ipv4.ip_forward` ?

Post by Rhys Ferris
fi
fi
Here’s my network XML
<network>
<name>olympus</name>
<uuid>3b0d968c-8166-42f7-8109-e5f0317cab42</uuid>
<forward mode='nat'>
<nat>
<port start='1024' end='65535'/>
</nat>
</forward>
<bridge name='virbr1' stp='on' delay='0'/>
<mac address='52:54:00:bb:18:6b'/>
<ip address='10.128.10.1' netmask='255.255.255.0'>
<dhcp>
<range start='10.128.10.2' end='10.128.10.254'/>
<host mac='52:54:00:8d:f5:0c' name='testy' ip='10.128.10.100'/>
</dhcp>
</ip>
</network>
Chain PREROUTING (policy ACCEPT 46615 packets, 6618K bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
Chain INPUT (policy ACCEPT 46615 packets, 6618K bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
Chain OUTPUT (policy ACCEPT 198K packets, 18M bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
Chain POSTROUTING (policy ACCEPT 198K packets, 18M bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
24 1812 RETURN all -- any any 10.128.10.0/24

base-address.mcast.net/24

Post by Rhys Ferris
0 0 RETURN all -- any any 10.128.10.0/24

255.255.255.255

Post by Rhys Ferris
17 1020 MASQUERADE tcp -- any any 10.128.10.0/24 !

10.128.10.0/24 masq ports: 1024-65535

Post by Rhys Ferris
15 1700 MASQUERADE udp -- any any 10.128.10.0/24 !

10.128.10.0/24 masq ports: 1024-65535

Post by Rhys Ferris
0 0 MASQUERADE all -- any any 10.128.10.0/24 !

10.128.10.0/24

Post by Rhys Ferris
22 1666 RETURN all -- any any 192.168.122.0/24

base-address.mcast.net/24

Post by Rhys Ferris
0 0 RETURN all -- any any 192.168.122.0/24

255.255.255.255

Post by Rhys Ferris
0 0 MASQUERADE tcp -- any any 192.168.122.0/24 !

192.168.122.0/24 masq ports: 1024-65535

Post by Rhys Ferris
8 1168 MASQUERADE udp -- any any 192.168.122.0/24 !

192.168.122.0/24 masq ports: 1024-65535

Post by Rhys Ferris
0 0 MASQUERADE all -- any any 192.168.122.0/24 !

192.168.122.0/24

Post by Rhys Ferris
AFTER VM BOOT
Chain PREROUTING (policy ACCEPT 2 packets, 120 bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
0 0 DNAT tcp -- any any anywhere

anywhere tcp dpt:2588 to:10.128.10.100:22
The out:any and

Post by Rhys Ferris
Chain INPUT (policy ACCEPT 2 packets, 120 bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
Chain OUTPUT (policy ACCEPT 18 packets, 1263 bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
Chain POSTROUTING (policy ACCEPT 18 packets, 1263 bytes)
pkts bytes target prot opt in out source

destination

Post by Rhys Ferris
24 1812 RETURN all -- any any 10.128.10.0/24

base-address.mcast.net/24

Post by Rhys Ferris
0 0 RETURN all -- any any 10.128.10.0/24

255.255.255.255

Post by Rhys Ferris
17 1020 MASQUERADE tcp -- any any 10.128.10.0/24 !

10.128.10.0/24 masq ports: 1024-65535

Post by Rhys Ferris
15 1700 MASQUERADE udp -- any any 10.128.10.0/24 !

10.128.10.0/24 masq ports: 1024-65535

Post by Rhys Ferris
0 0 MASQUERADE all -- any any 10.128.10.0/24 !

10.128.10.0/24

Post by Rhys Ferris
22 1666 RETURN all -- any any 192.168.122.0/24

base-address.mcast.net/24

Post by Rhys Ferris
0 0 RETURN all -- any any 192.168.122.0/24

255.255.255.255

Post by Rhys Ferris
0 0 MASQUERADE tcp -- any any 192.168.122.0/24 !

192.168.122.0/24 masq ports: 1024-65535

Post by Rhys Ferris
8 1168 MASQUERADE udp -- any any 192.168.122.0/24 !

192.168.122.0/24 masq ports: 1024-65535

Post by Rhys Ferris
0 0 MASQUERADE all -- any any 192.168.122.0/24 !

192.168.122.0/24

Post by Rhys Ferris
ssh: connect to host 172.16.99.170 port 2258: Connection refused
The connection refused is instant, not a timeout.
I’ve ensured that ufw is disabled.
Any help appreciated. I just can’t figure this out.
Sent from Mail for Windows 10
_______________________________________________
libvirt-users mailing list
https://www.redhat.com/mailman/listinfo/libvirt-users

[Martin Kletzander]

Post by Rhys Ferris
Thanks for the reply!
net.ipv4.ip_forward = 1

OK, so that is fine.

Post by Rhys Ferris
What do you mean "The out:any and"

I started writing something and then realized it's not related.

Anyway, you can also

Post by Rhys Ferris
Anywhere else I can look as to why the connection isn't going?
Do I need some kind of listener at that port on the host? I'm not even
seeing the packet count on the prerouting chain increase when the
connection attempts are made.

You can always check with wireshark to see what packets are arriving.

Anyway, my iptables are rusty since I switched to shorewall and then firewalld,
so I'm not sure what the reason is. It looks like it should work.