From owner-freebsd-net@FreeBSD.ORG Sat Apr 1 05:31:18 2006 Return-Path: X-Original-To: freebsd-net@freebsd.org Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4CCFD16D450; Sat, 1 Apr 2006 05:29:08 +0000 (UTC) (envelope-from michael@staff.openaccess.org) Received: from smtp.openaccess.org (smtp.openaccess.org [66.165.52.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2C0FB43D4C; Sat, 1 Apr 2006 05:29:05 +0000 (GMT) (envelope-from michael@staff.openaccess.org) Received: from [10.0.1.2] (unknown [216.57.214.93]) by smtp.openaccess.org (Postfix) with ESMTP id 83DEE6D4977; Fri, 31 Mar 2006 21:28:19 -0800 (PST) In-Reply-To: <20060331071115.GC884@trit.org> References: <014e01c64928$6107abd0$020b000a@bartwrkstxp> <20060316193740.GE11850@spc.org> <20060325092123.GB5468@trit.org> <20060331071115.GC884@trit.org> Mime-Version: 1.0 (Apple Message framework v746.3) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <21DCE1FA-4A7E-443F-8EFA-9E3CC7CE1C30@staff.openaccess.org> Content-Transfer-Encoding: 7bit From: Michael DeMan Date: Fri, 31 Mar 2006 21:29:09 -0800 To: Dima Dorfman X-Mailer: Apple Mail (2.746.3) Cc: Bart Van Kerckhove , "freebsd-net@FreeBSD.org" Subject: Re: OT - Quagga/CARP X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 01 Apr 2006 05:31:18 -0000 Hi, See inline... On Mar 30, 2006, at 11:11 PM, Dima Dorfman wrote: > Michael DeMan wrote: >> So, if you already have a route to 10.100.100.0/24 via OSPF to >> another machine, then try to... >> >> ip address 10.100.100.55/24 >> >> You get an error. > > Is that the only problem? Someone was talking about funding > development to fix something--surely there must be something more > severe than the inability to use the "ip address" interface command? I > thought the problem was about encoutering broken ingress paths if one > of the routers loses connectivity to the destination network. > My understanding is that my issue is just one symptom of a general limitation in the kernel routing tables or something, and that fixing this problem would also allow multi-path routing for FreeBSD, which is probably a bigger 'win' for the community overall. > Does the combination of CARP and quagga OSPF work once it's configured > using system tools? Yes, it will work then. However, I still have to kill and restart the zebra and ospf processes entirely for them to pick things up correctly. > >> It is possible to force the interface configuration via 'ifconfig' on >> the UNIX command line, but for this equipment I want all interface >> configuration and routing driven out of Quagga. > > It would be cool if that was possible, but it's not really practical. > My systems tend to have a lot of very custom configuration that quagga > will never be able to express. If I had a cookie-cutter configuration, > I'd probably be using a C or J box. > > While I've found bgpd and ospfd to be very stable, the zebra part that > interacts with the kernel has had various problems over time--routes > not being installed correctly, or going away, or having incorrect > flags. I wouldn't trust it to configure the entire network subsystem. > I've noticed some oddities with zebra too, but never anything that is a show-stopper. There was some kind of bug with notifications of interface 'up/down' not getting propogated correctly between zebra/ kernel, but that seems to be fixed. We do some scripting for automation of firewall rules for the routers to protect themselves, but at this point I have no need of the UNIX command line on these machines on a regular basis. The idea of using ifconfig, rc.conf and Quagga.conf to manage multiple machines, especially with automated management tools, is just impossible. Long term manageability is the goal. If everything is just in zebra/ quagga, then I just have one file to manage - Quagga.conf - for all backup, change control and managing lots of boxes in the field means I want much of the management driven straight out of our customer management application. Basically, fast/easy to turn up/down an office suite, colocation box, microwave circuit, for a customer right off our internal management system. > Dima. > >> On Mar 25, 2006, at 1:21 AM, Dima Dorfman wrote: >> >>> Michael DeMan wrote: >>>> Anyway, thanks very much for the information. I'm going to have to >>>> figure out some kind of workaround on my architecture. In the >>>> worst >>>> case, I can shut off OSPF on the edge routers and use static routes >>>> upstream and OSPF from there, but that is going to be a real >>>> nightmare for network maintenance over the long haul. >>> >>> You're talking about using CARP and OSPF on the edge routers, right? >>> >>> Can you explain a little more why CARP and zebra/ospfd don't play >>> well >>> together? I understand the problem about having two copies of the >>> same >>> route in the FIB, but I don't think it should prevent redundancy >>> from >>> working. I am planning to deploy FreeBSD-based access routers in the >>> near future, and I'd like to have an idea of what issues I'll be >>> facing. >>> >>> The scenario I have in mind is two FreeBSD boxes connected to the >>> rest >>> of the network on one side and clients (using carp) on the other. >>> CARP >>> is supposed to protect the client against one of the routers >>> failing. >>> I tried this on some test boxes today, and it looks like it should >>> work. Both boxes are configured as OSPF neighbors and share a CARP >>> vhid. When both links are up, each router has a route through the >>> physical interface (it also sees the OSPF route, but the connected >>> route is better). If one of the links fails (any condition that >>> causes >>> the physical interface to be down), the routes are withdrawn, the >>> other box takes over the VIP, and the first box installs the OSPF >>> route. Everything is still reachable. >>> >>> Am I missing an obvious problem or a case where this doesn't work? >>