From owner-freebsd-net@FreeBSD.ORG Sat Jun 16 00:35:36 2007 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 82D8C16A468 for ; Sat, 16 Jun 2007 00:35:36 +0000 (UTC) (envelope-from julian@elischer.org) Received: from outW.internet-mail-service.net (outW.internet-mail-service.net [216.240.47.246]) by mx1.freebsd.org (Postfix) with ESMTP id 6D0B413C483 for ; Sat, 16 Jun 2007 00:35:36 +0000 (UTC) (envelope-from julian@elischer.org) Received: from mx0.idiom.com (HELO idiom.com) (216.240.32.160) by out.internet-mail-service.net (qpsmtpd/0.32) with ESMTP; Fri, 15 Jun 2007 17:35:36 -0700 Received: from julian-mac.elischer.org (nat.ironport.com [63.251.108.100]) by idiom.com (Postfix) with ESMTP id 91455125A2A; Fri, 15 Jun 2007 17:35:35 -0700 (PDT) Message-ID: <46733055.4080502@elischer.org> Date: Fri, 15 Jun 2007 17:35:33 -0700 From: Julian Elischer User-Agent: Thunderbird 2.0.0.4 (Macintosh/20070604) MIME-Version: 1.0 To: Boris Kochergin , freebsd-net@freebsd.org, sysadmin@rescomp.berkeley.edu References: <20070615213454.GE2335@rescomp.berkeley.edu> <467312FF.5020506@acm.poly.edu> <20070615231255.GG2335@rescomp.berkeley.edu> In-Reply-To: <20070615231255.GG2335@rescomp.berkeley.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Routing outbound IP packets on multihomed box X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 16 Jun 2007 00:35:36 -0000 Christopher Cowart wrote: > On Fri, Jun 15, 2007 at 06:30:23PM -0400, Boris Kochergin wrote: >> Christopher Cowart wrote: >>> I have a server with two NICs: >>> >>> em0: 169.229.79.139/25 >>> vlan526: 169.229.126.9/24 >>> >>> The default gateway is 169.229.79.129. The router for the 126 subnet is >>> 169.229.126.1. >>> >>> netstat -rn: >>> | Destination Gateway Flags Refs Use Netif >>> Expire >>> | default 169.229.79.129 UGS 0 102537 em0 >>> | 127.0.0.1 127.0.0.1 UH 0 217 lo0 >>> | 169.229.79.128/25 link#1 UC 0 0 em0 >>> | 169.229.79.129 00:15:c7:b9:f4:80 UHLW 2 4 em0 >>> 1193 >>> | 169.229.79.139 00:11:25:ab:42:70 UHLW 1 589 lo0 >>> | 169.229.126/24 link#9 UC 0 0 vlan52 >>> | 169.229.126.1 00:15:c7:b9:f4:80 UHLW 1 34 vlan52 >>> 1200 >>> | 169.229.126.9 00:18:f8:09:d3:a5 UHLW 1 8 lo0 >>> >>> The IP address on em0 works exactly as one would expect. I have full IP >>> connectivity to it from other subnets. >>> >>> The problem is I can't get 2-way connectivity with the IP address on >>> vlan526. >>> >>> Using my workstation on a third subnet (169.229.127.38/24), I cannot >>> ping 169.229.126.9. I leave the ping running and do some tcpdumps on >>> the server. >>> >>> $ sudo tcpdump -ni vlan526 host 169.229.127.38 >>> | 14:14:37.002920 IP 169.229.127.38 > 169.229.126.9: ICMP echo >>> | request, id 15733, seq 35, length 64 >>> | 14:14:38.003037 IP 169.229.127.38 > 169.229.126.9: ICMP echo >>> | request, id 15733, seq 36, length 64 >>> >>> Notice there are no echo replies. That's because they're being sent >>> here: >>> >>> $ sudo tcpdump -ni em0 host 169.229.127.38 >>> | 14:15:42.006997 IP 169.229.126.9 > 169.229.127.38: ICMP echo reply, >>> | id 15733, seq 100, length 64 >>> | 14:15:43.007118 IP 169.229.126.9 > 169.229.127.38: ICMP echo reply, >>> | id 15733, seq 101, length 64 >>> >>> I repeated this last snoop with a -w and loaded it into ethereal. The >>> echo replies being sent out on em0 indeed have a source address of >>> 169.229.126.9. The router (169.229.79.139) drops these packets on the >>> floor, because their source address isn't routable on that interface. >>> >>> Because routing is based on destination, not source address, I'm not >>> sure how to get packets sourced from the 126 subnet to the router on the >>> 126 subnet. I tried the following ipfw rule right after allow loopback >>> traffic (my second rule): >>> >>> fwd 169.229.126.1 ip from 169.229.126.9 to not 169.229.126.0/24 >>> >>> Still no luck. Has anyone set up a multihomed box on *different* subnets >>> before without routing them through the FreeBSD box? Does anyone have >>> any pointers or things I should be looking at? >> Hi. I've come across this problem but solved it with a PF rule of this >> form, if that's an option for you: >> >> pass out route-to (vlan256 169.229.126.1) from 169.229.126.9 to any >> >> This tells PF to send all packets sent from 169.229.126.9 through the >> vlan256 interface with a next-hop address of 169.229.126.1. > > Unfortunately, I don't think we can use pf. The rest of our > infrastructure is ipfw and we don't particularly want this to be a > one-off. I was under the impression that my ipfw rule did exactly this, > by sending the packets to the 126 router as their next hop. > > Anyone have any ideas on whether an ipfw fwd rule can be used in a > similar way to this pf rule? > > Thanks again, > he ipfw rule should work, assuming you have the IPFIREWALL_FORWARDING option (and it's not the couple of versions of the OS where you also needed the IPFIREWALL_FORWARD_EXTENDED option as well.. also, you need to make it an 'out' rule..i.e. fwd 169.229.126.1 ip from 169.229.126.9 to not 169.229.126.0/24 out