Friday, December 21, 2018

centos - Mirroring Docker Traffic to Different Port in Same Container



I am using docker containers on CentOS to simulate devices on the network and have two tools that are to run inside each of those containers to simulate communication with actual network devices. Individually, either tool works inside the docker containers and can communicate with the outside world via one port we'll call 8000 here, but due to both attempting to use 8000, they are unable to run simultaneously. To resolve this, I am looking to mirror traffic on 8000 to another port we'll call 8001 here so that one tool can run on 8000 while the other receives its packets on 8001. In my Dockerfile, I EXPOSE both of these (though do not publish them in the run command).



Poking around the internet, I found this. Attempting to adapt it to my purposes, I ended up with the below, which I execute from within docker exec -it bash. By my primitive understanding, it should copy the message to a different localhost IP's port 8000 while letting the original message through, then pass the duplicate packet back to 127.0.0.1:8001.




iptables -t mangle -A PREROUTING -p UDP --dport 8000 -j TEE --gateway 127.0.0.2
iptables -t nat -A PREROUTING -d 127.0.0.2 -p UDP --dport 8000 -j DNAT --to 127.0.0.1:8001


This does not appear to be working. Running a tool on 8001 caused it to miss all packets I threw at 8000 from the outside world.



Similarly, I found this one, and tried adapting it to the below, but it doesn't seem to work either, and trying to get netcat running properly in a container has proven to be quite the headache.



iptables -A PREROUTING -t mangle -p tcp ! -s 127.0.0.1 --dport 8000 -j TEE --gateway 127.0.0.1

iptables -A OUTPUT -t nat -p tcp -s 127.0.0.1/32 --dport 8000 -j DNAT --to 127.0.0.1:8001


I am new to both docker and iptables, so any explanation of the underlying mechanisms of what is going wrong here would be greatly appreciated.


Answer



After a great deal of effort, the following solution was discovered that needs to run inside the container:



sysctl -w net.ipv4.conf.eth0.route_localnet=1
iptables -t mangle -A PREROUTING -i eth0 -p UDP --dport 8000 -j TEE --gateway 127.0.0.1
iptables -t nat -A PREROUTING -p UDP --dport 8000 -j DNAT --to-destination 127.0.0.1:8001



The key thing missing was the sysctl command to be run inside the container to allow things in the first place. Then, it needed to be eth0 instead of the previous solutions as otherwise you would end up with infinite loops.


No comments:

Post a Comment

linux - How to SSH to ec2 instance in VPC private subnet via NAT server

I have created a VPC in aws with a public subnet and a private subnet. The private subnet does not have direct access to external network. S...