From owner-freebsd-net@FreeBSD.ORG Mon Sep 29 05:47:51 2003 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CF2C416A4B3; Mon, 29 Sep 2003 05:47:51 -0700 (PDT) Received: from postfix3-2.free.fr (postfix3-2.free.fr [213.228.0.169]) by mx1.FreeBSD.org (Postfix) with ESMTP id A342843FE5; Mon, 29 Sep 2003 05:47:50 -0700 (PDT) (envelope-from yannick.dambielle@free.fr) Received: from shibuya (nord-5-82-67-198-63.fbx.proxad.net [82.67.198.63]) by postfix3-2.free.fr (Postfix) with SMTP id 6A59CC455; Mon, 29 Sep 2003 14:47:38 +0200 (CEST) Message-ID: <004801c38687$e323d870$3fc64352@shibuya> From: "Dambielle Yannick" To: "Robert Watson" , "Giovanni P. Tirloni" References: Date: Mon, 29 Sep 2003 14:47:37 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 cc: freebsd-net@freebsd.org Subject: Re: tcpdump - tun/tap virtual interfaces (a full working exemple) X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Sep 2003 12:47:52 -0000 Hello everybody, Well, here is the full exemple I've just done to deal with a tap0 interface. I didn't give you enaugh details in my last mail, so this time I've inserted all the step. Tap0 is now fully used in the sens that icmp requests and echos can be dumped from the virtual interface. As you will see in the following details, I had to attach 2 IP address to tap0 (10.0.0.1 and 10.0.0.2) and I had to make the route for 10.0.0.1 passing trough 10.0.0.2 as a gateway. Up today it is the only way I found for reaching that point, because altrought 10.0.0.1 was attached to tap0, it changed to lo0 once the address pinged . Any other change like "route add 10.0.0.1 -interface tap0" didn't have any effect and icmp requests and replies where always seen from lo0. According to wat you said (Mr. Watson - please correct me if I'm wrong), this is because 10.0.0.1 is always considered as a local address. But if I use another IP address (10.0.0.2) and if I attache it to tap0 and if I explicitly say to 10.0.0.1 to use 10.0.0.2 as the gateway, every paquets coming from 10.0.0.1 will go trough tap0. I hope my explication is understandable. Any comments or other method is really really really welcome. Here start the details : ----------------------------------------------------------------------- Before starting. Here is how everything is configured on FreeBSD box. ----------------------------------------------------------------------- shinjuku# ifconfig rl0: flags=8843 mtu 1500 inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:08:02:05:51:e0 media: Ethernet autoselect (100baseTX) status: active lp0: flags=8810 mtu 1500 lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 shinjuku# netstat -rn Routing tables Internet: Destination Gateway Flags Refs Use Netif Expire default 192.168.0.1 UGSc 1 0 rl0 127.0.0.1 127.0.0.1 UH 0 27 lo0 192.168.0 link#1 UC 1 0 rl0 192.168.0.1 00:08:c7:08:16:d8 UHLW 2 42 rl0 1064 shinjuku# ----------------------------------------------------------------------- I create the sink for the paquet data and this time I let it stay open. I aslo configure 10.0.0.1 and 10.0.0.2 on tap0 ----------------------------------------------------------------------- shinjuku# dd if=/dev/tap0 of=/dev/null bs=1500 & [1] 601 shinjuku# ifconfig rl0: flags=8843 mtu 1500 inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:08:02:05:51:e0 media: Ethernet autoselect (100baseTX) status: active lp0: flags=8810 mtu 1500 lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 tap0: flags=8802 mtu 1500 ether 00:bd:99:13:01:00 Opened by PID 601 shinjuku# ifconfig tap0 up shinjuku# ifconfig tap0 inet 10.0.0.1/24 add shinjuku# ifconfig tap0 inet 10.0.0.2/32 add shinjuku# ifconfig rl0: flags=8843 mtu 1500 inet 192.168.0.2 netmask 0xffffff00 broadcast 192.168.0.255 ether 00:08:02:05:51:e0 media: Ethernet autoselect (100baseTX) status: active lp0: flags=8810 mtu 1500 lo0: flags=8049 mtu 16384 inet 127.0.0.1 netmask 0xff000000 tap0: flags=8843 mtu 1500 inet 10.0.0.1 netmask 0xffffff00 broadcast 10.0.0.255 inet 10.0.0.2 netmask 0xffffffff broadcast 10.0.0.2 ether 00:bd:99:13:01:00 Opened by PID 601 shinjuku# ----------------------------------------------------------------------- Below are the routes for 10.0.0.1 and 10.0.0.2, but once 10.0.0.1 is pinged, its interface is changed into lo0. According to what Mr Watson said before (please, correct me if I'm wrong), this is because 10.0.0.1 is considered as a local IP. So evenif the route for reaching the address was trough tap0, a local IP is re-routed on lo0... The same effect can also be seen with 10.0.0.2. A tcpdump on lo0 will dump all requests/replies from 10.0.01 and 10.0.0.2. Nothing is going to be seen trough tap0. ----------------------------------------------------------------------- shinjuku# route get 10.0.0.1 route to: 10.0.0.1 destination: 10.0.0.0 mask: 255.255.255.0 interface: tap0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 500 -55 shinjuku# shinjuku# route get 10.0.0.2 route to: 10.0.0.2 destination: 10.0.0.2 mask: 255.255.255.255 interface: tap0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 500 -75 shinjuku# shinjuku# ping 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=0.106 ms 64 bytes from 10.0.0.1: icmp_seq=1 ttl=64 time=0.079 ms ^C --- 10.0.0.1 ping statistics --- 2 packets transmitted, 2 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.079/0.092/0.106/0.014 ms shinjuku# ping 10.0.0.2 PING 10.0.0.2 (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmp_seq=0 ttl=64 time=0.155 ms 64 bytes from 10.0.0.2: icmp_seq=1 ttl=64 time=0.088 ms 64 bytes from 10.0.0.2: icmp_seq=2 ttl=64 time=0.085 ms ^C --- 10.0.0.2 ping statistics --- 3 packets transmitted, 3 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.085/0.109/0.155/0.032 ms shinjuku# shinjuku# route get 10.0.0.1 route to: 10.0.0.1 destination: 10.0.0.1 interface: lo0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 shinjuku# shinjuku# route get 10.0.0.2 route to: 10.0.0.2 destination: 10.0.0.2 interface: lo0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 shinjuku# ----------------------------------------------------------------------- What happens if I explicitly say to 10.0.0.1 to pass through tap0... Nothing! ... the default route (lo0) always come back.. ----------------------------------------------------------------------- shinjuku# route add 10.0.0.1 -interface tap0 add host 10.0.0.1: gateway tap0 shinjuku# route get 10.0.0.1 route to: 10.0.0.1 destination: 10.0.0.1 interface: lo0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 shinjuku# ----------------------------------------------------------------------- And if I explicitly say to 10.0.0.2 to be reached through tap0... It works in the sens that icmp request will be dumped trough tap0 but icmp replies will go trough lo0. ----------------------------------------------------------------------- shinjuku# route add 10.0.0.2 -interface tap0 add host 10.0.0.2: gateway tap0 shinjuku# route get 10.0.0.2 route to: 10.0.0.2 destination: 10.0.0.2 interface: tap0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 shinjuku# shinjuku# ping -c 1 10.0.0.2 PING 10.0.0.2 (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmp_seq=0 ttl=64 time=0.115 ms --- 10.0.0.2 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.115/0.115/0.115/0.000 ms shinjuku# shinjuku# tcpdump -i lo0 tcpdump: listening on lo0 10:18:55.505577 10.0.0.2 > 10.0.0.1: icmp: echo reply shinjuku# tcpdump -i tap0 tcpdump: listening on tap0 10:18:55.505523 10.0.0.1 > 10.0.0.2: icmp: echo request ----------------------------------------------------------------------- Now if I explicitly say to 10.0.0.1 to use 10.0.0.2 as a gateway, the gateway will be inserted and the interface will be changes into tap0. ----------------------------------------------------------------------- shinjuku# route add 10.0.0.1 10.0.0.2 add host 10.0.0.1: gateway 10.0.0.2 shinjuku# route get 10.0.0.1 route to: 10.0.0.1 destination: 10.0.0.1 gateway: 10.0.0.2 interface: tap0 flags: recvpipe sendpipe ssthresh rtt,msec rttvar hopcount mtu expire 0 0 0 0 0 0 1500 0 shinjuku# ----------------------------------------------------------------------- And all paquets (requests and replies) will go trough tap0. Nothing pass trough lo0. ----------------------------------------------------------------------- shinjuku# ping -c 1 10.0.0.1 PING 10.0.0.1 (10.0.0.1): 56 data bytes 64 bytes from 10.0.0.1: icmp_seq=0 ttl=64 time=0.137 ms --- 10.0.0.1 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.137/0.137/0.137/0.000 ms shinjuku# shinjuku# tcpdump -i lo0 tcpdump: listening on lo0 shinjuku# tcpdump -i tap0 tcpdump: listening on tap0 10:29:10.540622 10.0.0.2 > 10.0.0.1: icmp: echo request 10:29:10.540680 10.0.0.1 > 10.0.0.2: icmp: echo reply ----------------------------------------------------------------------- We can also see that all requests/replies from 10.0.0.2 will be seen trough tap0 and not lo0. ----------------------------------------------------------------------- shinjuku# ping -c 1 10.0.0.2 PING 10.0.0.2 (10.0.0.2): 56 data bytes 64 bytes from 10.0.0.2: icmp_seq=0 ttl=64 time=0.145 ms --- 10.0.0.2 ping statistics --- 1 packets transmitted, 1 packets received, 0% packet loss round-trip min/avg/max/stddev = 0.145/0.145/0.145/0.000 ms shinjuku# shinjuku# tcpdump -i lo0 tcpdump: listening on lo0 shinjuku# tcpdump -i tap0 tcpdump: listening on tap0 10:30:30.488313 10.0.0.1 > 10.0.0.2: icmp: echo request 10:30:30.488381 10.0.0.2 > 10.0.0.1: icmp: echo reply ----------------------------------------------------------------------- One big problem that I could have asked you before finishing all that tests is that I lose the intial address, I mean if I ping 10.0.0.2, the icmp requests are dumped as if they came from 10.0.0.1 and not from localhost or 127.0.0.1. You will tell me that it is normal because I use a gateway... But the thing I'll do further with the use of virtual interfaces will be dumping data going to and coming from a web server inside a jail and dumping data going to and coming from something else inside another jail. All that personnal research is not for beeing in a production environment as if it was I will have used more than one box. On my box with it's 2 jails, each virtual interface will have a snort listening for whatever could pass trough theim and I want to study that as if I could have enaugh money and as many box as I would... Well, any information is welcome. Sorry if this mail was a little bit too long Yannick