Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 29 Mar 2007 09:34:37 +0200 (CEST)
From:      User Patrice <patrice@idea.dnsalias.net>
To:        freebsd-ipfw@freebsd.org
Subject:   ipfw drive me crazy
Message-ID:  <200703290734.l2T7YbSO003126@servidea.dvp.idea>

next in thread | raw e-mail | index | archive | help
Hello

I observe a strange behavior with ipfw/natd and fwd command.
the same packet , fwd to a same address, use a different outgoing interface
if it is nated.

Here is the sample:

freebsd (5.3release), 3 interfaces :
- bge1: 10.10.21.2 connected to a local LAN 10.10.21/0
- bge0: switch: 10.10.20.1 for DMZ, and 192.168.0.101 for a new internet
router at 192.168.0.254
- tun0: internet public address for a PPP adsl modem

the freebsd is the default gateway for 10.10.21/24 network
the tun0 is the default gateway interface inside the freebsd.


i want to catch (based on tcp out port) outgoing packet to default gateway tun0, and send them to
192.168.0.254 

the test i run is :
- i have a valid system, routing every external traffic throught tun0
- i want every outgoing tcp 8080 connection will use 192.168.0.254 router



10.10.21.1 request a http connection on port 8080
without additionnal config, the request will come in bge1, go out tun0

i try to trap the request to use 192.168.0.254 gateway
the target ip 192.168.0.254 is on a lan connected to bge0.

ipfw add 00150 fwd 192.168.0.254 log ip from any to any dst-port 8080

the test was successfull, tcpdump show that :
- incoming packet from 10.10.21.1 to external ip, 8080 on bge1
- outgoing packet to 192.168.0.254 via bge0
just a little strange behavior in ifpw log which show outgoing packet on
tun0
i think it's strange because 192.168.0.254 is on lan connected to bge0 wich
have ip 192.168.0.101

so now, just missing to nat the incoming packet
ipfw add 00149 divert 3617 log ip from 10.10.0.0/16 to not 10.10.0.0/16
dst-port 8080

the test now give me headache.
The log show than the packet is well catched & diverted, with same strange
behavior: out via tun0
(strange because the target ip 192.168.0.254 is on a lan connected to bge0)
and tcpdump show:
- incoming packet from 10.10.21.1 to external ip, 8080 on bge1
- outgoing natd packet to 192.168.0.254 via tun0 (instead of bge0???)

so for resume:
if i do nothing, a packet in bge1 is going out on tun9
i want to catch packet in bge1, fwd to gateway on bge0
- if i just fwd, tcpdump say : ok it work, ipfw say: fwd is ok but on wrong
interface
- if i fwd&nat, tcpdump say : wrong interface, ipfw say: fwd is ok but on
wrong interface

and now ive got headache


log from my test follow:
=============================================================
interface:
bge0: flags=8943<UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST> mtu 1500
        options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
        inet 10.10.20.1 netmask 0xffffff00 broadcast 10.10.20.255
        inet 192.168.0.101 netmask 0xffffff00 broadcast 192.168.0.255
        ether 00:30:48:88:5f:f2
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
bge1: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> mtu 1500
        options=1a<TXCSUM,VLAN_MTU,VLAN_HWTAGGING>
        inet 10.10.21.2 netmask 0xffffff00 broadcast 10.10.21.255
        ether 00:30:48:88:5f:f3
        media: Ethernet autoselect (100baseTX <full-duplex>)
        status: active
tun0: flags=8151<UP,POINTOPOINT,RUNNING,PROMISC,MULTICAST> mtu 1492
        inet 80.11.76.251 --> 80.11.76.129 netmask 0xffffffff


Route:
Destination,Gateway, interface
default, AMarseille-111-1-3, tun0
10.10.1/24, link#3,em0
10.10.20/24,link#1,bge0
10.10.21/24, link#2,bge1
AMarseille-111-1-3,AMarseille-111-1-3,tun0
192.168.0,link#1,bge0

natd:
natd -o 3617 -alias_address 192.168.0.101
(i used -o port because i m going to nat packet on input interface, and the
option -reverse of natd cause core dump)

====================================================================
test #1: 10.10.21.1 request a tcp to 8080 on a external web, forward every
incoming packet for 8080
ipfw add 00150 fwd 192.168.0.254 log ip from any to any dst-port 8080

log:
Mar 23 09:11:50 servidea kernel: ipfw: 150 Forward to 192.168.0.254 TCP
10.10.21
.1:3798 194.167.78.73:8080 in via bge1
Mar 23 09:11:50 servidea kernel: ipfw: 150 Forward to 192.168.0.254 TCP
10.10.21
.1:3798 194.167.78.73:8080 out via tun0

i do not understand why there is 2 lines for a single packet, and why is
show tun0  as out interface
but trace with tcpdump show that the packet :
- is coming in bge1, out bge0, nothing as tun0
(192.168.0.254 is connected on bge0)

tcppdump bge1:
09:11:50.131590 00:e0:18:53:3b:a1 > 00:30:48:88:5f:f3, ethertype IPv4
(0x0800),
length 60: IP 10.10.21.1.3798 > 194.167.78.73.8080: S
1223470518:1223470518(0) w
in 8192 <mss 1460>
09:11:53.391659 00:e0:18:53:3b:a1 > 00:30:48:88:5f:f3, ethertype IPv4
(0x0800),
length 60: IP 10.10.21.1.3798 > 194.167.78.73.8080: S
1223470518:1223470518(0) w
in 8192 <mss 1460>

tcpdump bge0
09:11:50.132040 00:30:48:88:5f:f2 > 00:07:cb:24:2b:c8, ethertype IPv4
(0x0800),
length 58: IP 10.10.21.1.3798 > 194.167.78.73.8080: S
1223470518:1223470518(0) w
in 8192 <mss 1460>
09:11:53.391740 00:30:48:88:5f:f2 > 00:07:cb:24:2b:c8, ethertype IPv4
(0x0800),
length 58: IP 10.10.21.1.3798 > 194.167.78.73.8080: S
1223470518:1223470518(0) w
in 8192 <mss 1460>

tcpdump tun0: nothing

====================================================================
test #2: same as test#1
ipfw add 00150 fwd 192.168.0.254 log ip from any to any dst-port 8080

and added natd before forwarding
ipfw 00149 divert 3617 log ip from 10.10.0.0/16 to not 10.10.0.0/16 dst-port
8080

log:
Mar 23 09:09:14 servidea kernel: ipfw: 149 Divert 3617 TCP 10.10.21.1:3757
194.1
67.78.73:8080 in via bge1
Mar 23 09:09:14 servidea kernel: ipfw: 150 Forward to 192.168.0.254 TCP
192.168.
0.101:3757 194.167.78.73:8080 in via bge1
Mar 23 09:09:14 servidea kernel: ipfw: 150 Forward to 192.168.0.254 TCP
192.168.
0.101:3757 194.167.78.73:8080 out via tun0
Mar 23 09:09:17 servidea kernel: ipfw: 149 Divert 3617 TCP 10.10.21.1:3757
194.1
67.78.73:8080 in via bge1
Mar 23 09:09:17 servidea kernel: ipfw: 150 Forward to 192.168.0.254 TCP
192.168.
0.101:3757 194.167.78.73:8080 in via bge1
Mar 23 09:09:17 servidea kernel: ipfw: 150 Forward to 192.168.0.254 TCP
192.168.
0.101:3757 194.167.78.73:8080 out via tun0

still same strange thing: 1 packet in bge1 cause 2 line forward

tcpdump: bge1:
09:09:14.346305 00:e0:18:53:3b:a1 > 00:30:48:88:5f:f3, ethertype IPv4
(0x0800),
length 60: IP 10.10.21.1.3757 > 194.167.78.73.8080: S
1223469999:1223469999(0) w
in 8192 <mss 1460>
09:09:17.528385 00:e0:18:53:3b:a1 > 00:30:48:88:5f:f3, ethertype IPv4
(0x0800),
length 60: IP 10.10.21.1.3757 > 194.167.78.73.8080: S
1223469999:1223469999(0) w
in 8192 <mss 1460>

tcpdump bge0: nothing

tcpdump tun0:
09:09:14.346459 AF 2 44: IP 192.168.0.101.3757 > 194.167.78.73.8080: S
122346999
9:1223469999(0) win 8192 <mss 1460>
09:09:17.528522 AF 2 44: IP 192.168.0.101.3757 > 194.167.78.73.8080: S
122346999
9:1223469999(0) win 8192 <mss 1460>








Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200703290734.l2T7YbSO003126>