From owner-freebsd-net@FreeBSD.ORG Thu Mar 7 11:56:00 2013 Return-Path: Delivered-To: net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 68849149 for ; Thu, 7 Mar 2013 11:56:00 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id DE9543A6 for ; Thu, 7 Mar 2013 11:55:59 +0000 (UTC) Received: (qmail 92765 invoked from network); 7 Mar 2013 13:09:25 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 7 Mar 2013 13:09:25 -0000 Message-ID: <51388046.7040408@freebsd.org> Date: Thu, 07 Mar 2013 12:55:50 +0100 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130107 Thunderbird/17.0.2 MIME-Version: 1.0 To: "Alexander V. Chernikov" Subject: Re: [patch] interface routes References: <513834E4.7050203@FreeBSD.org> <51384443.5070209@freebsd.org> <51387D4A.9030408@FreeBSD.org> In-Reply-To: <51387D4A.9030408@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Mar 2013 11:56:00 -0000 On 07.03.2013 12:43, Alexander V. Chernikov wrote: > On 07.03.2013 11:39, Andre Oppermann wrote: >> On 07.03.2013 07:34, Alexander V. Chernikov wrote: >>> Hello list! >>> >>> There is a known long-lived issue with interface routes >>> addition/deletion: >>> >>> ifconfig iface inet 1.2.3.4/24 can fail if given prefix is already in >>> kernel route table (for >>> example, advertised by IGP like OSPF). >>> >>> Interface route can be deleted via route(8) or any route socket user >>> (sometimes this happens with >>> popular opensource daemons like bird/quagga). >>> >>> Problem is reported at least in kern/106722 and kern/155772. >> >> You patch is a welcome addition. >> >>> This can be fixed the following way: >>> Immutable route flag (RTM_PINNED, added in 19995 with 'for future use' >>> comment) is utilised to mark >>> route 'immutable'. >>> rtrequest1_fib refuses to delete routes with given flag unless >>> RTM_PINNED is set in rti_flags. >> >> How do the routing daemons react to being unable to change/delete >> such a route? > routing daemons live long with the fact that there route socket cmds can > fail (and the is route(8) utility which can do anything), so typically > bird/quagga yells like > 'bird: KRT: Error sending route 11.0.0.0/24 to kernel: File exists' > and marks given route as not installed in internal RIB. Additionally, > daemon will probably re-try to insert such routes on every periodic KRT > rescan (tens of minutes). OK. No problem then. > Given that such sutiations usually happens for a very short time (e.g. > physical link flaps) everything should become to normal state quickly. > >> >> EADDRINUSE would likely be a more descriptive error instead of EPERM? > Well, not sure if EADDRINUSE is very descriptive for _deleting_ route. > "Yes, I know that it is in use so that's the reason I'm trying to delete > it". I'm thinking of distinguishing it from a permission denial, because of insufficient rights (jail or something like that) vs. an explicitly pinned route. With EPERM you may look for the problem in the wrong place. E*INUSE is a common error for something can't be removed due to it still being used by or for something else. Which is the case here and may be more appropriate. >>> Every interface address manupulation is done via rtinit[1], so >>> rtinit1() sets this flag (and behavior does not change here). >>> >>> Adding interface address is handled via atomically deleting old prefix >>> and adding interface one. >> >> This brings up a long standing sore point of our routing code >> which this patch makes more pronounced. When an interface link >> state is down I don't want the route to it to persist but to >> become inactive so another path can be chosen. This the very >> point of running a routing daemon. So on the link-down event >> the installed interface routes should be removed from the routing >> table. The configured addresses though should persist and the >> interface routes re-installed on a link-up event. What's your >> opinion on it? > > This is exactly what is done in current code for IPv4: > if_down calls if_unroute(), it cals prctlinput() for every interface > address, and domain-dependent function like rip_ctlinput calls > in_ifscrub() cleaning given interface route. > However, address route (/32) still remains (but route daemons, at least > bird, tends to ignore it since it is not listed as valid interface > address/mask). IF_DOWN and link state down are not the same thing. When the cable is unplugged the link state goes down but not the interface. > This is not done for IPv6 and we should probably do the same. Yes, they should be synchronized. >> Other than these points I think your code is fine and can go >> into the tree. -- Andre