Date: Tue, 18 Aug 2020 08:16:55 +0000 From: Grzegorz Junka <list1@gjunka.com> To: freebsd-net@freebsd.org Subject: Re: net.add_addr_allfibs=1 behaviour deprecation Message-ID: <c0995001-d4dc-cb34-17a1-45d650410da6@gjunka.com> In-Reply-To: <0f321bdc-240e-0f85-915b-c9bdd7daa01b@freebsd.org> References: <236161595078191@mail.yandex.ru> <348771597489519@mail.yandex.ru> <0f321bdc-240e-0f85-915b-c9bdd7daa01b@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 18/08/2020 07:54, Julian Elischer wrote: > The reason for the two behaviours is that there are two ways that the > previous behaviour of "add addresses to the only FIB" could be > interpreted and extended once multiple fibs became available. The > single fib case could be interpreted as either of: > > "Add to All N fibs where N == 1" or "add to the first (of 1) > fibs". > I decided to do both :-) > > At Ironport where I wrote it we had a scenario where I didn't want to > have wrong entries in all the fibs when a new interface was brought > up. Even for a moment. An other scenarios where for example a tunnel > uses fib 1 but the rest of the system uses fib0 (which points through > the tunnel) The addition of new routes into the tunnel's route when a > new virtual interface is brought up pointing through the tunnel to the > same address, leads in the tunnel immediately redirecting packets > through itself which ends in tears. SO the obvious thing to do was > to make it possible to only add the entry in the fib that was the > default fib or in the case of Ironport, the fib that was the default > fib of the process adding the interface. > > If you had to make a choice I think the '0' choice is the way to go. > All other fibs need to be populated deliberately.. > > On 8/15/20 4:24 AM, Alexander V. Chernikov wrote: >> 18.07.2020, 14:22, "Alexander V. Chernikov" <melifaro@freebsd.org>: >>> Dear FreeBSD users, >>> >>> I would like to make net.add_addr_allfibs=0 as the default system >>> behaviour and remove net.add_addr_allfibs. >>> To do so, I would like to collect use cases with >>> net.add_addr_allfibs=1 and multiple fibs, to ensure they can still >>> be supported after removal. >>> >>> Background: >>> >>> Multi-fib support was added in r178888 [1], 12 years ago. Addition >>> of interface addresses to all fibs was a feature from day 1. >>> The `net.add_addr_allfibs` sysctl was added in r180840 [2], 12 >>> years ago. >>> >>> Problem: >>> The goal of the fib support is to provide multiple independent >>> routing tables, isolated from each other. >>> `net.add_addr_allfibs` default tries to shift gears in the opposite >>> direction, unconditionally inserting all addresses to all of the fibs. >>> >>> It complicates the logic, kernel code and makes control plane >>> performance decrease with the number of fibs. >>> It make impossible to use the same prefixes in multiple fibs, which >>> may be desired given shortage of IPv4 address space. >>> >>> I do understand that there are some cases where such behaviour is >>> desired. >>> For example, it can be used to achieve VRF route leaking or binding >>> on address from different fibs. >>> I would like to collect such cases to consider supporting them in a >>> different way. >>> >>> The goal is to make net.add_addr_allfibs=0 default behaviour and >>> remove net.add_addr_allfibs. >>> It will simplify kernel fib-related code and allow bringing more >>> fib-related features. It will also improve fib scaling. >> No objections has been received. >> Next steps: >> * Switch net.add_addr_allfibs to 0 ( >> https://reviews.freebsd.org/D26076 ) >> * Provide an ability to use nexthops from different fibs >> * Remove net.add_addr_allfibs >>> Timeline: >>> Aug 1: summarising feedback and the usecases, decision on proceeding >>> further >>> Aug 20 (tentative): patches for supported usecases >>> Sep 15 (tentative): net.add_addr_allfibs removal. >>> >>> [1]: [base Contents of >>> /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?revision=178888&view=markup) >>> [2]: [base Diff of >>> /head/sys/net/route.c](https://svnweb.freebsd.org/base/head/sys/net/route.c?r1=180839&r2=180840&) >>> >>> /Alexander >> Agree completely, defaulting "add_addr_allfibs" to 1 broke many existing installations, which goes against the least surprise principle so many times advocated on FreeBSD lists. This is just one example: https://forums.freebsd.org/threads/strange-behavior-of-setfib-since-freebsd-12-0.73348/ Now, changing the default again might again break existing installations, which shouldn't be a reason for not doing it, but might be a reason to better communicate it this time around. GrzegorzJ
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?c0995001-d4dc-cb34-17a1-45d650410da6>