Wednesday, December 31, 2014

ubuntu 10.04 - KVM guests lose connectivity after networking restart



we're setting up an ubuntu server 10.04 host with kvm. The host is setup with a bond and bridged interfaces to allow the guests access to the network without natting.
Our current configuration is working fine, except when we're restarting the network with an /etc/init.d/networking restart.
After restarting the network, the guests lose connectivity. The only way to restore it is to halt the guest and start it again.




I've been looking around but I can't seem to find any known bug/issue/report of this behavior.



Here follows our network configuration file:



auto lo
iface lo inet loopback

auto bond0
iface bond0 inet manual
bond-slaves none

bond-mode active-backup
bond-downdelay 250
bond-updelay 120
bond-miimon 100

auto eth0
iface eth0 inet manual
bond-master bond0
bond-primary eth0 eth1


auto eth1
iface eth1 inet manual
bond-master bond0
bond-primary eth0 eth1

#bridge used by host
auto br-vlan180
iface br-vlan180 inet static
address 10.0.0.200
netmask 255.255.255.0

gateway 10.0.0.1
vlan-raw-device bond0
bridge_ports vlan180
bridge_maxwait 0
bridge_fd 0
bridge_stp off
#bridge without address, used by vm
auto br-vlan120
iface br-vlan120 inet manual
vlan-raw-device bond0

bridge_ports vlan120
bridge_maxwait 0
bridge_fd 0
bridge_stp off


Thank you



ADDENDUM - brctl show output before and after nw restart:




BRCTL SHOW BEFORE NW RESTART



brctl show
bridge name bridge id STP enabled interfaces
br-vlan120 8000.984be1644072 no vlan120
vnet0
vnet1
br-vlan180 8000.984be1644072 no vlan180
virbr0 8000.000000000000 yes



BRCTL SHOW AFTER NW RESTART



brctl show
bridge name bridge id STP enabled interfaces
br-vlan120 8000.984be1644072 no vlan120
br-vlan180 8000.984be1644072 no vlan180
virbr0 8000.000000000000 yes



apparently, the two virtual interfaces fail to come back after network restart.



PS BEFORE NW RESTART



ps -ef | grep qemu
root 1784 1 6 11:45 ? 00:00:40 /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 512 -smp 1 -name test02 -uuid ee6d84b6-dbf8-d93c-b32f-8ae6b7d9b80e -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/test02.monitor,server,nowait -monitor chardev:monitor -boot c -drive file=/dev/sysvg/test02,if=virtio,index=0,boot=on,format=raw -drive file=/root/ubuntu-10.04.2-server-amd64.iso,if=ide,media=cdrom,index=2,format=raw -net nic,macaddr=52:54:00:2c:d1:26,vlan=0,name=nic.0 -net tap,fd=48,vlan=0,name=tap.0 -chardev pty,id=serial0 -serial chardev:serial0 -parallel none -usb -vnc 127.0.0.1:0 -vga cirrus -soundhw es1370
root 2711 1 89 11:55 ? 00:00:14 /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 512 -smp 2 -name nttest -uuid 04ca381e-0510-7d3c-c7e2-8f7d7b6ea58f -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/nttest.monitor,server,nowait -monitor chardev:monitor -localtime -boot c -drive file=/dev/sysvg/nttest,if=ide,index=0,boot=on,format=raw -drive file=/root/SW_DVD5_Windows_Svr_DC_EE_SE_Web_2008_R2_64Bit_English_w_SP1_MLF_X17-22580.ISO,if=ide,media=cdrom,index=2,format=raw -net nic,macaddr=52:54:00:62:1b:2e,vlan=0,name=nic.0 -net tap,fd=51,vlan=0,name=tap.0 -chardev pty,id=serial0 -serial chardev:serial0 -parallel none -usb -usbdevice tablet -vnc 127.0.0.1:1 -vga cirrus -soundhw es1370


PS AFTER NW RESTART




ps -ef | grep qemu
root 1784 1 4 11:45 ? 00:00:59 /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 512 -smp 1 -name test02 -uuid ee6d84b6-dbf8-d93c-b32f-8ae6b7d9b80e -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/test02.monitor,server,nowait -monitor chardev:monitor -boot c -drive file=/dev/sysvg/test02,if=virtio,index=0,boot=on,format=raw -drive file=/root/ubuntu-10.04.2-server-amd64.iso,if=ide,media=cdrom,index=2,format=raw -net nic,macaddr=52:54:00:2c:d1:26,vlan=0,name=nic.0 -net tap,fd=48,vlan=0,name=tap.0 -chardev pty,id=serial0 -serial chardev:serial0 -parallel none -usb -vnc 127.0.0.1:0 -vga cirrus -soundhw es1370
root 2711 1 39 11:55 ? 00:03:51 /usr/bin/kvm -S -M pc-0.12 -enable-kvm -m 512 -smp 2 -name nttest -uuid 04ca381e-0510-7d3c-c7e2-8f7d7b6ea58f -chardev socket,id=monitor,path=/var/lib/libvirt/qemu/nttest.monitor,server,nowait -monitor chardev:monitor -localtime -boot c -drive file=/dev/sysvg/nttest,if=ide,index=0,boot=on,format=raw -drive file=/root/SW_DVD5_Windows_Svr_DC_EE_SE_Web_2008_R2_64Bit_English_w_SP1_MLF_X17-22580.ISO,if=ide,media=cdrom,index=2,format=raw -net nic,macaddr=52:54:00:62:1b:2e,vlan=0,name=nic.0 -net tap,fd=51,vlan=0,name=tap.0 -chardev pty,id=serial0 -serial chardev:serial0 -parallel none -usb -usbdevice tablet -vnc 127.0.0.1:1 -vga cirrus -soundhw es1370

Answer



Well, there is the problem, when you restart the networking, the vnetX tap devices are not reconnected, causing the VMs to lose connectivity with the bridge.



I guess you could manually reconnect them to the bridge, since they are still running, but the right way to do this would be to migrate the VMs away from a host where you make network changes, or take the VMs down, if you're in a single host mode. In most corporate level systems this is called "maintenance mode", and changing the network config is definitely maintenance.


No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...