From owner-freebsd-net@FreeBSD.ORG Fri Feb 22 07:44:20 2008 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5C5BB16A400 for ; Fri, 22 Feb 2008 07:44:20 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from tokyo01.jp.mail.your.org (tokyo01.jp.mail.your.org [204.9.54.5]) by mx1.freebsd.org (Postfix) with ESMTP id 23CD013C465 for ; Fri, 22 Feb 2008 07:44:19 +0000 (UTC) (envelope-from toasty@dragondata.com) Received: from mail.your.org (server3-a.your.org [64.202.112.67]) by tokyo01.jp.mail.your.org (Postfix) with ESMTP id A5B6D2AD5532; Fri, 22 Feb 2008 07:44:18 +0000 (UTC) Received: from pool014.dhcp.your.org (pool014.dhcp.your.org [69.31.99.14]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mail.your.org (Postfix) with ESMTP id DCC42A0A44E; Fri, 22 Feb 2008 07:44:17 +0000 (UTC) Message-Id: From: Kevin Day To: Wes Peters In-Reply-To: <1C828D1A-192A-40ED-8391-DA316611E6E2@opensail.org> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes Content-Transfer-Encoding: 7bit Mime-Version: 1.0 (Apple Message framework v919.2) Date: Fri, 22 Feb 2008 01:44:17 -0600 References: <20080219021012.95B1116A4CB@hub.freebsd.org> <8E87DC1A-6EC2-4E53-9FA3-17E694BE7846@opensail.org> <47BCA1AA.7060800@FreeBSD.org> <1C828D1A-192A-40ED-8391-DA316611E6E2@opensail.org> X-Mailer: Apple Mail (2.919.2) Cc: freebsd-net@freebsd.org Subject: Re: Multiple default routes on multihome host X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Feb 2008 07:44:20 -0000 On Feb 21, 2008, at 9:51 PM, Wes Peters wrote: > > As much as anything I just object to the semantic dissonance in > "multiple" "default". Think about it. > > I still haven't decided what it means at the packet level to have > multiple default routes. Does that mean that, not having found a > "better" route, I send the packets out both routes? Choose between > them? Doesn't that tend to flap packets in a TCP "connection" back > and forth? Does my router have to remember which route it chose for > a TCP connection and reuse that one? > > I know people want to be able to plug in a pair of itty bitty > routers and just have their computers be smart enough to use the > "best" one, but it's not clear the implementations they are pushing > us towards -- Linux and Windows -- actually accomplish that. In > fact, what they usually do is screw it up badly and the people only > THINK they're getting any enhanced reliability. > I know I'm not who you were asking, but I can give you an example of where we've used this successfully. Our branch office has a T1 to our main office. The branch office has a /26 of public IPs routed over the T1. The T1 has extremely low latency, and plenty of bandwidth for the business side of things. The problem is that it didn't have enough bandwidth to handle a bunch of people watching videos on YouTube, downloading OS updates and everything else. I played with QoS and traffic shaping, but the solution for us was more bandwidth. Adding additional T1s was impossible, but we could get a very fast business DSL line to the office. They obviously wouldn't run BGP with us over it, so some trickery was required to make use of both connections at once. On our firewall/router box at the branch office, we've got 3 ethernet interfaces. em0 goes to our LAN(1.2.3.4/26). em1 goes to the T1 router. em2 goes to the DSL line(5.6.7.8/24). The system's default route is through em1 to the T1. I want to send some traffic over the DSL line, em2. This is complicated by the fact that the DSL provider has only given us one IP and won't route our corporate IPs. So, I started up a natd instance: natd -interface em2 -same_ports -dynamic Now, with ipfw I can select which traffic goes through the DSL line: ipfw add 100 divert 8668 ip from 1.2.3.0/26 to any 80 # Send all HTTP traffic through natd, which will go through the DSL line Next, I need to force all traffic sourced on the DSL line's IP to actually go out the DSL interface. Without this, the kernel tries sending packets sourced with the DSL line's IP over the T1. ipfw add 200 fwd $dsl_line_gateway ip from 5.6.7.8 to not 1.2.3.0/26 # If it's not trying to talk to a local IP, force it to go down the DSL line if it's using the DSL source IP. Now, like magic, web traffic goes over the DSL line. Everything else goes over the T1. In reality the configuration is much more complex, but it's easy enough with ipfw rules to specify what I want to go down the DSL line (divert it) and what I want to go down the T1. If I didn't have to deal with the lack of routing cooperation from our DSL provider, I could skip the natd step completely and just fwd traffic as appropriate. This isn't truly multiple default routes, but it's as close as I can get. As-is it adds no redundancy at all, but it was very easy to script something up that checked the liveliness of both interfaces and completely redirect everything to go down one or the other if one goes down. Make sense? -- Kevin