From owner-freebsd-net@freebsd.org Tue Aug 18 19:27:27 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 24E893C52C7 for ; Tue, 18 Aug 2020 19:27:27 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from forward500p.mail.yandex.net (forward500p.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:110]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4BWLXF1VdRz4Y3R for ; Tue, 18 Aug 2020 19:27:25 +0000 (UTC) (envelope-from melifaro@ipfw.ru) Received: from mxback14g.mail.yandex.net (mxback14g.mail.yandex.net [IPv6:2a02:6b8:0:1472:2741:0:8b7:93]) by forward500p.mail.yandex.net (Yandex) with ESMTP id A8CC494011D; Tue, 18 Aug 2020 22:27:20 +0300 (MSK) Received: from localhost (localhost [::1]) by mxback14g.mail.yandex.net (mxback/Yandex) with ESMTP id w381f0sYOa-RJeWQAZ8; Tue, 18 Aug 2020 22:27:20 +0300 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=ipfw.ru; s=mail; t=1597778840; bh=Q4UMbHQWfsODTTkasgqno3FhSRI2NHmeFf9z7BpShZ0=; h=References:Date:Message-Id:Subject:In-Reply-To:To:From; b=TC2mBiqjn8zgXJos9s1fgWBWRW5tNLgKnNWX/eh6fdmJAFBSo/Q9JzjHceq3I0UTl O0mp3RNQUeoyK3xsDdlr6kmFzQYPX9MgeAI0X8er6EXafCVCL/XuBJ01LjVNJVPrDs RSKicTtbcBaehF+tATwNU4u85ID3NawK0aBm5s8k= Received: by sas8-da6d7485e0c7.qloud-c.yandex.net with HTTP; Tue, 18 Aug 2020 22:27:19 +0300 From: Alexander V. Chernikov To: Grzegorz Junka , "freebsd-net@freebsd.org" In-Reply-To: References: <236161595078191@mail.yandex.ru> <348771597489519@mail.yandex.ru> <0f321bdc-240e-0f85-915b-c9bdd7daa01b@freebsd.org> Subject: Re: net.add_addr_allfibs=1 behaviour deprecation MIME-Version: 1.0 X-Mailer: Yamail [ http://yandex.ru ] 5.0 Date: Tue, 18 Aug 2020 20:27:19 +0100 Message-Id: <837391597778482@mail.yandex.ru> Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=utf-8 X-Rspamd-Queue-Id: 4BWLXF1VdRz4Y3R X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ipfw.ru header.s=mail header.b=TC2mBiqj; dmarc=none; spf=pass (mx1.freebsd.org: domain of melifaro@ipfw.ru designates 2a02:6b8:0:1472:2741:0:8b7:110 as permitted sender) smtp.mailfrom=melifaro@ipfw.ru X-Spamd-Result: default: False [-1.27 / 15.00]; ARC_NA(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_ALLOW(-0.20)[ipfw.ru:s=mail]; FREEFALL_USER(0.00)[melifaro]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a02:6b8:0:1000::/52:c]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[ipfw.ru]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[ipfw.ru:+]; RCPT_COUNT_TWO(0.00)[2]; NEURAL_HAM_SHORT(-0.67)[-0.666]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:13238, ipnet:2a02:6b8::/32, country:RU]; MAILMAN_DEST(0.00)[freebsd-net]; RCVD_IN_DNSWL_LOW(-0.10)[2a02:6b8:0:1472:2741:0:8b7:110:from] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 18 Aug 2020 19:27:27 -0000 18.08.2020, 09:17, "Grzegorz Junka" : > On 18/08/2020 07:54, Julian Elischer wrote: >>  The reason for the two behaviours is that there are two ways that the >>  previous behaviour of  "add addresses to the only FIB" could be >>  interpreted and extended once multiple fibs became available. The >>  single fib case could be interpreted as either of: >> >>  "Add to All N fibs where N == 1"    or     "add to the first (of 1) >>  fibs". >>  I decided to do both :-) >> >>  At Ironport where I wrote it we had a scenario where I didn't want to >>  have wrong entries in all the fibs when a new interface was brought >>  up. Even for a moment. An other scenarios where  for example a tunnel >>  uses fib 1 but the rest of the system uses fib0 (which points through >>  the tunnel) The addition of new routes into the tunnel's route when a >>  new virtual interface is brought up pointing through the tunnel to the >>  same address, leads in the tunnel immediately redirecting packets >>  through itself which ends in tears.     SO the obvious thing to do was >>  to make it possible to only add the entry in the fib that was the >>  default fib or in the case of Ironport, the fib that was the default >>  fib of the process adding the interface. >> >>  If you had to make a choice I think the '0' choice is the way to go. >>  All other fibs need to be populated deliberately.. >> >>  On 8/15/20 4:24 AM, Alexander V. Chernikov wrote: >>>  18.07.2020, 14:22, "Alexander V. Chernikov" : >>>>  Dear FreeBSD users, >>>> >>>>  I would like to make net.add_addr_allfibs=0 as the default system >>>>  behaviour and remove net.add_addr_allfibs. >>>>  To do so, I would like to collect use cases with >>>>  net.add_addr_allfibs=1 and multiple fibs, to ensure they can still >>>>  be supported after removal. >>>> >>>>  Background: >>>> >>>>  Multi-fib support was added in r178888 [1], 12 years ago. Addition >>>>  of interface addresses to all fibs was a feature from day 1. >>>>  The `net.add_addr_allfibs` sysctl  was added in r180840 [2], 12 >>>>  years ago. >>>> >>>>  Problem: >>>>  The goal of the fib support is to provide multiple independent >>>>  routing tables, isolated from each other. >>>>  `net.add_addr_allfibs` default tries to shift gears in the opposite >>>>  direction, unconditionally inserting all addresses to all of the fibs. >>>> >>>>  It complicates the logic, kernel code and makes control plane >>>>  performance decrease with the number of fibs. >>>>  It make impossible to use the same prefixes in multiple fibs, which >>>>  may be desired given shortage of IPv4 address space. >>>> >>>>  I do understand that there are some cases where such behaviour is >>>>  desired. >>>>  For example, it can be used to achieve VRF route leaking or binding >>>>  on address from different fibs. >>>>  I would like to collect such cases to consider supporting them in a >>>>  different way. >>>> >>>>  The goal is to make net.add_addr_allfibs=0 default behaviour and >>>>  remove net.add_addr_allfibs. >>>>  It will simplify kernel fib-related code and allow bringing more >>>>  fib-related features. It will also improve fib scaling. >>>  No objections has been received. >>>  Next steps: >>>  * Switch net.add_addr_allfibs to 0 ( >>>  https://reviews.freebsd.org/D26076 ) >>>  * Provide an ability to use nexthops from different fibs >>>  * Remove net.add_addr_allfibs >>>>  Timeline: >>>>  Aug 1: summarising feedback and the usecases, decision on proceeding >>>>  further >>>>  Aug 20 (tentative):  patches for supported usecases >>>>  Sep 15 (tentative):  net.add_addr_allfibs removal. >>>> >>>>  [1]: [base Contents of >>>>  /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=178888&view=markup) >>>>  [2]: [base Diff of >>>>  /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839&r2=180840&) >>>> >>>>  /Alexander > > Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing > installations, which goes against the least surprise principle so many > times advocated on FreeBSD lists. > > This is just one example: > https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/ > > Now, changing the default again might again break existing > installations, which shouldn't be a reason for not doing it, but might > be a reason to better communicate it this time around. I plan to communicate it the following way: 1) this thread 2) GONE_IN13 in the sysctl (which will print console message when set to 1) 3) Release notes Do you think there are other communication channels one should try to use? > > GrzegorzJ > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"