networking - Martian source in log of machine1 after ssh-DNAT to virtual machine located on machine2 (machine1 and machine2 xover connected)

• 2 physical machines, each with 2 network interfaces (eth0, eth1).


• they act as virtual machine host


• for one machine alone, the networking works with no problems


• first i had two seperated networks and each machine has a own virtual dhcp server running


• then i tried to merge the seperated networks into one


• firewall: shorewall (connection policy for lan = allow on both machines)


• dhcp server: dnsmasq


• both machines can connect to the internet

i would like to connect the two machines over a crossovercable and want them to share one network, so i have only one dhcp server with one network and each server in this lan can connect to each other. does this make sense or are two seperate dhcp server and networks the better way?

below i added some configs and i did some connection tests.

in short:

• machine1 + machine2 can reach ips on machine1


• machine1 + machine2 cannnot reach ips on machine2


• machine1 + machine2 can reach ips on machine2


• external dnat (for example ssh) does work for machine1 (port 5678 -> 10.62.63.20:22)


• external dnat (for example ssh) does not work for machine2 (port 5678 -> 10.62.63.30:22)

if i ssh connect to machine1 port 5678 the connection to 10.62.62.20 works and i only see shorewall log entries from this connection on machine1.
but if i connect to machine2 port 5678 the connection does not work and i can see a martian log on machine1

Nov 29 15:26:57 machine1 kernel: [ 7495.749894] martian source **ssh.client.ip.addr** from **yyy.yyy.yyy.yyy**, on dev br1

machine1 virtual machines:

• dhcp 10.62.63.2


• web1 10.62.63.20

machine2 virtual machines:

• web2 10.62.63.30

Config Files

machine1 /etc/shorewall/rules

***snip***
DNAT:debug  net  lan:10.62.63.20:22  tcp  5678  -  xxx.xxx.xxx.xxx
***snip***

machine2 /etc/shorewall/rules

***snip***

DNAT:debug  net  lan:10.62.63.30:22  tcp  5678  -  yyy.yyy.yyy.yyy
***snip***

machine1 /etc/networking/interfaces

# Loopback device:
auto lo
iface lo inet loopback


# device: eth0
#allow-hotplug eth0 
auto eth0
iface eth0 inet manual


# device: eth1
#allow-hotplug eth1
auto eth1
iface eth1 inet manual 


auto br0
iface br0 inet static
  address   xxx.xxx.xxx.xxx
  broadcast xxx.xxx.xxx.xxx
  netmask   xxx.xxx.xxx.xxx
  gateway   xxx.xxx.xxx.xxx
  bridge_ports eth0
  bridge_fd 0
  bridge_hello 2

  bridge_maxage 12
  bridge_maxwait 0
  bridge_stp off


auto br1
iface br1 inet static
  address 10.62.63.1
  broadcast 10.62.63.255
  netmask 255.255.255.0

  bridge_ports eth1
  bridge_fd 0
  bridge_hello 2
  bridge_maxage 12
  bridge_maxwait 0
  bridge_stp off

machine2 /etc/networking/interfaces

# Loopback device:
auto lo
iface lo inet loopback

# device: eth0
#allow-hotplug eth0 
auto eth0
iface eth0 inet manual



# device: eth1
#allow-hotplug eth1
auto eth1
iface eth1 inet manual 

auto br0
iface br0 inet static
  address   yyy.yyy.yyy.yyy
  broadcast yyy.yyy.yyy.yyy
  netmask   yyy.yyy.yyy.yyy

  gateway   yyy.yyy.yyy.yyy
  bridge_ports eth0
  bridge_fd 0
  bridge_hello 2
  bridge_maxage 12
  bridge_maxwait 0
  bridge_stp off


auto br1

iface br1 inet static
  address 10.62.63.3
  broadcast 10.62.63.255
  netmask 255.255.255.0
  bridge_ports eth1
  bridge_fd 0
  bridge_hello 2
  bridge_maxage 12
  bridge_maxwait 0
  bridge_stp off

machine1 (10.62.63.1)

routes:

ip route show
yyy.yyy.yyy.yyy/yy dev br0  proto kernel  scope link  src yyy.yyy.yyy.yyy

10.62.63.0/24 dev br1  proto kernel  scope link  src 10.62.63.1
default via yyy.yyy.yyy.yyy dev br0

• ping 10.62.63.3 to br1 ip (remote): ok


• ping 10.62.63.1 to br1 ip (local): ok


• ping 10.62.63.2 to dns (local): ok


• ping 10.62.63.20 to web01 (local): ok


• ping 10.62.63.30 to web02 (remote): ok



• ssh 10.62.63.20 to web01 (local): ok


• ssh 10.62.63.30 to web02 (remote): ok

machine2 (10.62.63.3)

routes:

ip route show
yyy.yyy.yyy.yyy/yy dev br0  proto kernel  scope link  src yyy.yyy.yyy.yyy

10.62.63.0/24 dev br1  proto kernel  scope link  src 10.62.63.3
default via yyy.yyy.yyy.yyy dev br0

• ping 10.62.63.3 to br1 ip (local): ok


• ping 10.62.63.1 to br1 ip (remote): ok


• ping 10.62.63.2 to dns (remote): ok


• ping 10.62.63.20 to web01 (remote): ok


• ping 10.62.63.30 to web02 (local): ok



• ssh 10.62.63.20 to web01 (remote): ok


• ssh 10.62.63.30 to web02 (local): ok

Answer

the problem was the outbound packets, not the inbound.

the dhcp server supplied the same gateway for both machines (10.62.63.1), so dnat inbound packets on machine2 had no problem to reach their location (web on machine2), but then the returning packet was sent to the gateway of machine1 (10.62.63.1) and not from where it was comming (10.62.63.3).

so it lands on machine1 as martion packet.

solution was to add tagging on the dns (dnsmasq) so the virtual machines on the different hosts get different gateways:

/etc/dnsmasq.conf

*** snip ***

dhcp-host=set:machine1,ff:ff:ff:ff:ff:ff,web01,10.62.63.20
dhcp-host=set:machine2,ee:ee:ee:ee:ee:ee,web02,10.62.63.30

dhcp-option=tag:machine1,option:router,10.62.63.1

dhcp-option=tag:machine2,option:router,10.62.63.3

*** snip ***