From owner-freebsd-net@FreeBSD.ORG Sat Feb 12 12:50:13 2011 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 607B9106564A for ; Sat, 12 Feb 2011 12:50:13 +0000 (UTC) (envelope-from nvass@gmx.com) Received: from mailout-eu.gmx.com (mailout-eu.gmx.com [213.165.64.42]) by mx1.freebsd.org (Postfix) with SMTP id C18EC8FC1D for ; Sat, 12 Feb 2011 12:50:12 +0000 (UTC) Received: (qmail invoked by alias); 12 Feb 2011 12:50:10 -0000 Received: from adsl-15.109.242.166.tellas.gr (EHLO [192.168.73.192]) [109.242.166.15] by mail.gmx.com (mp-eu004) with SMTP; 12 Feb 2011 13:50:10 +0100 X-Authenticated: #46156728 X-Provags-ID: V01U2FsdGVkX1+hQ+8PSYuhV3XY7hRmerr3jtNnEz+A8kTq6tW/z+ YgfYNfCKrF7mJ1 Message-ID: <4D5681DE.5010908@gmx.com> Date: Sat, 12 Feb 2011 14:49:34 +0200 From: Nikos Vassiliadis User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.2.13) Gecko/20101207 Thunderbird/3.1.7 MIME-Version: 1.0 To: Nikolay Denev References: <7C507431-3902-4B7A-B33F-51ECCFFF8306@gmail.com> In-Reply-To: <7C507431-3902-4B7A-B33F-51ECCFFF8306@gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: freebsd-net@freebsd.org Subject: Re: option RADIX_MPATH, RT_LINK_IS_UP() and interface routes. X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 12 Feb 2011 12:50:13 -0000 On 2/12/2011 11:36 AM, Nikolay Denev wrote: > Hello, > > A quick glance through sys/netinet/ip_output.c shows that interface routes are short-circuited and > not checked for RT_LINK_IS_UP as gateway routes are. > > Consider the following scenario : > > A pair of redundant routers : RTR1 and RTR2. > Each having dedicated uplink to some ISP and both run BGP, and they also have a dedicated cross-connection. > On the LAN side, they share a IP using CARP. > > Uplink1 Uplink2 > | | > | | > +------+ +------+ > | RTR1 |---| RTR2 | > +------+ +------+ > | | > | | > +---+----------+---+ > | LAN | > +------------------+ > > Now, if the cable on RTR1 connecting it to the LAN is disconnected, > RTR2 will become carp master and will start receiving packets from clients on LAN and they will be routed ok. > But form the ISP point of view the best path to the network is via RTR1, so the incoming traffic > will still be routed thru RTR1 because it's Uplink1 interface is UP and the BGP session established. > This will cause the packets destined to the LAN to be effectively blackholed, because of the interface route on RTR1. > When using kernel with RADIX_MPATH and ospf on both routers RTR1 will have two routes to the LAN, one interface and one via > the crossconnect to RTR2 but still, the interface route will be consen, regardless of link state up or down. > > I'm thinking about checking for RT_LINK_IS_UP on interface routes, or clear the RTF_UP flag on those routes when interface goes link down. Any other solutions/ideas? You could try sysutils/heartbeat which is similar in concept with CARP, but runs in userspace and gives you the ability to run scripts in case of a change. It can also ping any IP address to monitor a link's health, that is, "if I cannot get replies from hosta in my LAN, something must be wrong with me, I'll notify the backup host and change my status to backup". Heartbeat wikipedia article: http://en.wikipedia.org/wiki/Linux-HA HTH, Nikos