From owner-freebsd-net@FreeBSD.ORG Mon May 1 19:21:41 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0215A16A445 for ; Mon, 1 May 2006 19:21:41 +0000 (UTC) (envelope-from vulture@netvulture.com) Received: from rackman.netvulture.com (adsl-63-197-17-60.dsl.snfc21.pacbell.net [63.197.17.60]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6D5D843D46 for ; Mon, 1 May 2006 19:21:38 +0000 (GMT) (envelope-from vulture@netvulture.com) Received: from [208.201.244.73] (host73.netvulture.com [208.201.244.73]) (authenticated bits=0) by rackman.netvulture.com (8.13.5/8.13.5) with ESMTP id k41JFLHG073948 for ; Mon, 1 May 2006 12:15:22 -0700 (PDT) Message-ID: <44565E41.2080905@netvulture.com> Date: Mon, 01 May 2006 12:15:13 -0700 From: Jonathan Feally User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7.2) Gecko/20040804 Netscape/7.2 (ax) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-net@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-MailScanner-Information: Please contact your system administrator for more information X-MailScanner: Found to be clean X-MailScanner-SpamCheck: not spam (whitelisted), SpamAssassin (score=-0.89, required 2.5, autolearn=spam, AWL -1.48, HOT_NASTY 0.59) Subject: Having a problem with getting ipfw fwd to work with vlans and bge - 6.1-RC1 amd64 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 May 2006 19:21:41 -0000 Hello, I have setup a new firewall and I'm having trouble with it. Perhaps the bge is to blame, perhaps its something else. I'll explain my setup, problem and the workaround to get it going. Box connects to 2 Internal Lans and 2 External Wans. Vlans are mixed untagged and tagged on a single bge0 Vlan Network Desc 1 10.255.1.0/24 Admin Lan - No Vlan Tagging 2 10.255.2.0/24 VoIP Lan 900 67.xxx.xxx.128/27 Internet A - Default Route - Going to be pure VoIP only - thus 10.255.2 boxes get 1:1 NAT to 67.xxx.xxx 902 208.xxx.xxx.48/28 Internet B - Web Services 1st problem I ran into was pings from vlan 2 through natd to vlan 900 were not coming back. I could see the packet enter vlan2 - leave and return on vlan900 - but go nowhere. I tried a tcpdump on bge0 and the pings started coming back. Leading me to putting promisc on my ifconfig bge0 Now I'm trying to setup up a simple web server on an IP from vlan 902 in combination with fwd rule # 999 to route packets from a vlan902 address back to the router on that internet connection. I try to ping from the outside and can see the icmp echo request. But the replies keep getting sent out vlan900 to the other internet router. Hopefully somebody can point me in the right direction. If its the bge, then I can replace it with some em. If its an issue with mixing native vlan and tagged, I can tag everything, If its not me, then who can help getting the code fixed? I have put my ifconfig, ipfw rules and natd.conf's below. Thanks -Jon --------------------------------------------------------- [root@t3031fw ~]# ifconfig -a bge0: flags=28943 mtu 1500 options=18 inet6 fe80::215:f2ff:fed0:d898%bge0 prefixlen 64 scopeid 0x1 inet 10.255.1.254 netmask 0xffffff00 broadcast 10.255.1.255 ether 00:15:f2:d0:d8:98 media: Ethernet autoselect (100baseTX ) status: active bge1: flags=8802 mtu 1500 options=1b ether 00:15:f2:40:d8:35 media: Ethernet autoselect (none) status: no carrier plip0: flags=108810 mtu 1500 lo0: flags=8049 mtu 16384 inet6 ::1 prefixlen 128 inet6 fe80::1%lo0 prefixlen 64 scopeid 0x4 inet 127.0.0.1 netmask 0xff000000 vlan2: flags=8843 mtu 1500 inet6 fe80::215:f2ff:fed0:d898%vlan2 prefixlen 64 scopeid 0x5 inet 10.255.2.1 netmask 0xffffff00 broadcast 10.255.2.255 ether 00:15:f2:d0:d8:98 media: Ethernet autoselect (100baseTX ) status: active vlan: 2 parent interface: bge0 vlan900: flags=8843 mtu 1500 inet6 fe80::215:f2ff:fed0:d898%vlan900 prefixlen 64 scopeid 0x6 inet 67.xxx.xxx.158 netmask 0xffffffe0 broadcast 67.xxx.xxx.159 inet 67.xxx.xxx.130 netmask 0xffffffff broadcast 67.xxx.xxx.130 inet 67.xxx.xxx.131 netmask 0xffffffff broadcast 67.xxx.xxx.131 inet 67.xxx.xxx.132 netmask 0xffffffff broadcast 67.xxx.xxx.132 inet 67.xxx.xxx.133 netmask 0xffffffff broadcast 67.xxx.xxx.133 inet 67.xxx.xxx.134 netmask 0xffffffff broadcast 67.xxx.xxx.134 inet 67.xxx.xxx.135 netmask 0xffffffff broadcast 67.xxx.xxx.135 inet 67.xxx.xxx.136 netmask 0xffffffff broadcast 67.xxx.xxx.136 inet 67.xxx.xxx.137 netmask 0xffffffff broadcast 67.xxx.xxx.137 inet 67.xxx.xxx.138 netmask 0xffffffff broadcast 67.xxx.xxx.138 inet 67.xxx.xxx.139 netmask 0xffffffff broadcast 67.xxx.xxx.139 inet 67.xxx.xxx.140 netmask 0xffffffff broadcast 67.xxx.xxx.140 inet 67.xxx.xxx.141 netmask 0xffffffff broadcast 67.xxx.xxx.141 inet 67.xxx.xxx.142 netmask 0xffffffff broadcast 67.xxx.xxx.142 inet 67.xxx.xxx.143 netmask 0xffffffff broadcast 67.xxx.xxx.143 inet 67.xxx.xxx.144 netmask 0xffffffff broadcast 67.xxx.xxx.144 inet 67.xxx.xxx.145 netmask 0xffffffff broadcast 67.xxx.xxx.145 inet 67.xxx.xxx.146 netmask 0xffffffff broadcast 67.xxx.xxx.146 inet 67.xxx.xxx.147 netmask 0xffffffff broadcast 67.xxx.xxx.147 inet 67.xxx.xxx.148 netmask 0xffffffff broadcast 67.xxx.xxx.148 inet 67.xxx.xxx.149 netmask 0xffffffff broadcast 67.xxx.xxx.149 inet 67.xxx.xxx.150 netmask 0xffffffff broadcast 67.xxx.xxx.150 inet 67.xxx.xxx.151 netmask 0xffffffff broadcast 67.xxx.xxx.151 inet 67.xxx.xxx.152 netmask 0xffffffff broadcast 67.xxx.xxx.152 inet 67.xxx.xxx.153 netmask 0xffffffff broadcast 67.xxx.xxx.153 inet 67.xxx.xxx.154 netmask 0xffffffff broadcast 67.xxx.xxx.154 inet 67.xxx.xxx.155 netmask 0xffffffff broadcast 67.xxx.xxx.155 inet 67.xxx.xxx.156 netmask 0xffffffff broadcast 67.xxx.xxx.156 inet 67.xxx.xxx.157 netmask 0xffffffff broadcast 67.xxx.xxx.157 ether 00:15:f2:d0:d8:98 media: Ethernet autoselect (100baseTX ) status: active vlan: 900 parent interface: bge0 vlan902: flags=8843 mtu 1500 inet6 fe80::215:f2ff:fed0:d898%vlan902 prefixlen 64 scopeid 0x7 inet 208.xxx.xxx.48 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.49 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.50 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.51 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.52 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.53 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.54 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.55 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.56 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.57 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.58 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.59 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.60 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.61 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.62 netmask 0xffffff00 broadcast 208.xxx.xxx.255 inet 208.xxx.xxx.63 netmask 0xffffff00 broadcast 208.xxx.xxx.255 ether 00:15:f2:d0:d8:98 media: Ethernet autoselect (100baseTX ) status: active vlan: 902 parent interface: bge0 [root@t3031fw ~]# ipfw show 00100 612 297138 allow ip from any to any via lo0 00200 0 0 deny ip from any to 127.0.0.0/8 00300 0 0 deny ip from 127.0.0.0/8 to any 00401 507 46266 allow ip from 63.197.17.60 to any 00402 434 71914 allow ip from any to 63.197.17.60 00999 1256 75280 fwd 208.xxx.xxx.1 ip from 208.xxx.xxx.48/28 to any 01000 51349830 10346449386 divert 8668 ip from any to any via vlan900 01100 25290 6692181 divert 8669 ip from any to any via vlan902 01999 0 0 check-state 02999 5393 444962 allow icmp from any to any 03000 5290 847646 allow tcp from 10.255.2.0/24 to any keep-state 03001 0 0 allow udp from any to 10.255.2.100 dst-port 4569 keep-state 03001 26469 3267888 allow tcp from any to 10.255.2.100 dst-port 22 keep-state 03002 0 0 allow udp from any to 10.255.2.200 dst-port 4569 keep-state 03002 22003 2652985 allow tcp from any to 10.255.2.200 dst-port 22 keep-state 03300 10313 1223322 allow ip from 10.255.1.0/24 to 10.255.1.0/24 keep-state 03999 0 0 allow ip from 208.xxx.xxx.48/28 to any keep-state 04000 25701603 5174357258 allow ip from 67.xxx.xxx.128/27 to any keep-state 04001 0 0 allow tcp from any to 67.xxx.xxx.130 dst-port 22 keep-state 04002 0 0 allow tcp from any to 67.xxx.xxx.140 dst-port 22 keep-state 04058 32848 4351775 allow tcp from any to 67.xxx.xxx.158 dst-port 22 keep-state 04080 4596 3101277 allow tcp from any to 67.xxx.xxx.158 dst-port 80 keep-state 04080 4349 2856224 allow tcp from any to 208.xxx.xxx.48 dst-port 80 keep-state 10011 0 0 allow ip from 208.201.244.72/29 to 67.xxx.xxx.128/27 keep-state 10012 120462 68409347 allow ip from 208.201.244.72/29 to 10.255.2.0/24 keep-state 10013 0 0 allow ip from 67.xxx.xxx.128/27 to 208.201.244.72/29 keep-state 10014 223046 54830393 allow ip from 10.255.2.0/24 to 208.201.244.72/29 keep-state 11111 13137 6722265 allow ip from 10.255.2.0/24 to 207.174.202.2 keep-state 11112 0 0 allow ip from 67.xxx.xxx.128/27 to 207.174.202.2 keep-state 11113 0 0 allow ip from 207.174.202.2 to 67.xxx.xxx.128/27 keep-state 11114 22806 11460460 allow ip from 207.174.202.2 to 10.255.2.0/24 keep-state 11201 39017 19450498 allow ip from 10.255.2.0/24 to 207.174.202.3 keep-state 11202 0 0 allow ip from 67.xxx.xxx.128/27 to 207.174.202.3 keep-state 11203 0 0 allow ip from 207.174.202.3 to 67.xxx.xxx.128/27 keep-state 11204 17986 9036892 allow ip from 207.174.202.3 to 10.255.2.0/24 keep-state 11301 72141 10621231 allow ip from 10.255.2.0/24 to 207.174.202.4 keep-state 11302 0 0 allow ip from 67.xxx.xxx.128/27 to 207.174.202.4 keep-state 11303 0 0 allow ip from 207.174.202.4 to 67.xxx.xxx.128/27 keep-state 11304 22625 11368053 allow ip from 207.174.202.4 to 10.255.2.0/24 keep-state 11401 43193817 8659831738 allow ip from 10.255.2.0/24 to 216.241.188.54 keep-state 11402 0 0 allow ip from 67.xxx.xxx.128/27 to 216.241.188.54 keep-state 11403 0 0 allow ip from 216.241.188.54 to 67.xxx.xxx.128/27 keep-state 11404 611137 131292121 allow ip from 216.241.188.54 to 10.255.2.0/24 keep-state 12101 31804010 6372136314 allow ip from 10.255.2.0/24 to 207.174.111.12 keep-state 12102 0 0 allow ip from 67.xxx.xxx.128/27 to 207.174.111.12 keep-state 12103 0 0 allow ip from 207.174.111.12 to 67.xxx.xxx.128/27 keep-state 12104 441864 96541650 allow ip from 207.174.111.12 to 10.255.2.0/24 keep-state 13101 98120 11157261 allow ip from 10.255.2.0/24 to 66.246.246.52 keep-state 13102 0 0 allow ip from 67.xxx.xxx.128/27 to 66.246.246.52 keep-state 13103 0 0 allow ip from 66.246.246.52 to 67.xxx.xxx.128/27 keep-state 13104 0 0 allow ip from 66.246.246.52 to 10.255.2.0/24 keep-state 64000 49199 5396398 allow udp from 10.255.2.0/24 to any dst-port 53 keep-state 65000 213362 84312193 deny ip from any to any 65535 1 72 allow ip from any to any [root@t3031fw ~]# cat /etc/natd900.conf log_facility security use_sockets same_ports port natd interface vlan900 unregistered_only redirect_address 10.255.2.100 67.xxx.xxx.130 redirect_address 10.255.2.101 67.xxx.xxx.131 redirect_address 10.255.2.102 67.xxx.xxx.132 redirect_address 10.255.2.103 67.xxx.xxx.133 redirect_address 10.255.2.104 67.xxx.xxx.134 redirect_address 10.255.2.105 67.xxx.xxx.135 redirect_address 10.255.2.106 67.xxx.xxx.136 redirect_address 10.255.2.107 67.xxx.xxx.137 redirect_address 10.255.2.108 67.xxx.xxx.138 redirect_address 10.255.2.109 67.xxx.xxx.139 redirect_address 10.255.2.200 67.xxx.xxx.140 [root@t3031fw ~]# cat /etc/natd902.conf log_facility security use_sockets same_ports port natd2 alias_address 208.xxx.xxx.48 unregistered_only redirect_address 10.255.2.100 208.xxx.xxx.50 redirect_address 10.255.2.101 208.xxx.xxx.51 redirect_address 10.255.2.102 208.xxx.xxx.52 redirect_address 10.255.2.103 208.xxx.xxx.53 redirect_address 10.255.2.104 208.xxx.xxx.54 redirect_address 10.255.2.105 208.xxx.xxx.55 redirect_address 10.255.2.106 208.xxx.xxx.56 redirect_address 10.255.2.107 208.xxx.xxx.57 redirect_address 10.255.2.108 208.xxx.xxx.58 redirect_address 10.255.2.109 208.xxx.xxx.59 redirect_address 10.255.2.200 208.xxx.xxx.60