Date: Tue, 18 Aug 2020 20:27:19 +0100 From: Alexander V. Chernikov <melifaro@ipfw.ru> To: Grzegorz Junka <list1@gjunka.com>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: net.add_addr_allfibs=1 behaviour deprecation Message-ID: <837391597778482@mail.yandex.ru> In-Reply-To: <c0995001-d4dc-cb34-17a1-45d650410da6@gjunka.com> References: <236161595078191@mail.yandex.ru> <348771597489519@mail.yandex.ru> <0f321bdc-240e-0f85-915b-c9bdd7daa01b@freebsd.org> <c0995001-d4dc-cb34-17a1-45d650410da6@gjunka.com>
next in thread | previous in thread | raw e-mail | index | archive | help
18.08.2020, 09:17, "Grzegorz Junka" <list1@gjunka.com>: > On 18/08/2020 07:54, Julian Elischer wrote: >> The reason for the two behaviours is that there are two ways that the >> previous behaviour of "add addresses to the only FIB" could be >> interpreted and extended once multiple fibs became available. The >> single fib case could be interpreted as either of: >> >> "Add to All N fibs where N == 1" or "add to the first (of 1) >> fibs". >> I decided to do both :-) >> >> At Ironport where I wrote it we had a scenario where I didn't want to >> have wrong entries in all the fibs when a new interface was brought >> up. Even for a moment. An other scenarios where for example a tunnel >> uses fib 1 but the rest of the system uses fib0 (which points through >> the tunnel) The addition of new routes into the tunnel's route when a >> new virtual interface is brought up pointing through the tunnel to the >> same address, leads in the tunnel immediately redirecting packets >> through itself which ends in tears. SO the obvious thing to do was >> to make it possible to only add the entry in the fib that was the >> default fib or in the case of Ironport, the fib that was the default >> fib of the process adding the interface. >> >> If you had to make a choice I think the '0' choice is the way to go. >> All other fibs need to be populated deliberately.. >> >> On 8/15/20 4:24 AM, Alexander V. Chernikov wrote: >>> 18.07.2020, 14:22, "Alexander V. Chernikov" <melifaro@freebsd.org>: >>>> Dear FreeBSD users, >>>> >>>> I would like to make net.add_addr_allfibs=0 as the default system >>>> behaviour and remove net.add_addr_allfibs. >>>> To do so, I would like to collect use cases with >>>> net.add_addr_allfibs=1 and multiple fibs, to ensure they can still >>>> be supported after removal. >>>> >>>> Background: >>>> >>>> Multi-fib support was added in r178888 [1], 12 years ago. Addition >>>> of interface addresses to all fibs was a feature from day 1. >>>> The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 >>>> years ago. >>>> >>>> Problem: >>>> The goal of the fib support is to provide multiple independent >>>> routing tables, isolated from each other. >>>> `net.add_addr_allfibs` default tries to shift gears in the opposite >>>> direction, unconditionally inserting all addresses to all of the fibs. >>>> >>>> It complicates the logic, kernel code and makes control plane >>>> performance decrease with the number of fibs. >>>> It make impossible to use the same prefixes in multiple fibs, which >>>> may be desired given shortage of IPv4 address space. >>>> >>>> I do understand that there are some cases where such behaviour is >>>> desired. >>>> For example, it can be used to achieve VRF route leaking or binding >>>> on address from different fibs. >>>> I would like to collect such cases to consider supporting them in a >>>> different way. >>>> >>>> The goal is to make net.add_addr_allfibs=0 default behaviour and >>>> remove net.add_addr_allfibs. >>>> It will simplify kernel fib-related code and allow bringing more >>>> fib-related features. It will also improve fib scaling. >>> No objections has been received. >>> Next steps: >>> * Switch net.add_addr_allfibs to 0 ( >>> https://reviews.freebsd.org/D26076 ) >>> * Provide an ability to use nexthops from different fibs >>> * Remove net.add_addr_allfibs >>>> Timeline: >>>> Aug 1: summarising feedback and the usecases, decision on proceeding >>>> further >>>> Aug 20 (tentative): patches for supported usecases >>>> Sep 15 (tentative): net.add_addr_allfibs removal. >>>> >>>> [1]: [base Contents of >>>> /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=178888&view=markup) >>>> [2]: [base Diff of >>>> /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839&r2=180840&) >>>> >>>> /Alexander > > Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing > installations, which goes against the least surprise principle so many > times advocated on FreeBSD lists. > > This is just one example: > https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/ > > Now, changing the default again might again break existing > installations, which shouldn't be a reason for not doing it, but might > be a reason to better communicate it this time around. I plan to communicate it the following way: 1) this thread 2) GONE_IN13 in the sysctl (which will print console message when set to 1) 3) Release notes Do you think there are other communication channels one should try to use? > > GrzegorzJ > > _______________________________________________ > freebsd-net@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?837391597778482>