Date: Wed, 24 Feb 2021 01:22:24 +0100 From: Marek Zarychta <zarychtam@plan-b.pwste.edu.pl> To: "Alexander V. Chernikov" <melifaro@ipfw.ru>, freebsd-net@freebsd.org Subject: Re: option FIB_ALGO and dpdk_lpm4 Message-ID: <7056db64-2e7f-6ad8-54a3-59ff406061cf@plan-b.pwste.edu.pl> In-Reply-To: <7121614026888@mail.yandex.ru> References: <5670cd9a-cd10-2b89-1347-97a6c817c50f@sentex.net> <8696072a-dc25-8eff-04fa-4d1db13bf5cc@plan-b.pwste.edu.pl> <3588111612809020@mail.yandex.ru> <53ef5715-42ce-aad0-9a8b-11d91a9891e3@plan-b.pwste.edu.pl> <301391612827263@mail.yandex.ru> <ecb75bc5-72fe-4583-8fe6-e0cb5347ae02@plan-b.pwste.edu.pl> <2642181612914660@mail.yandex.ru> <2951613086198@mail.yandex.ru> <b51320bb-d401-3d96-64a6-139f2c980ffa@plan-b.pwste.edu.pl> <7121614026888@mail.yandex.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
> =C2=A0 > Btw, have you seen any notable dataplane/control plane performance > difference w/ FIB_ALGO ? Dear Alexander, valued subscribers, at last, I have upgraded our BGP router running two instances of net/bird (IP and IPv6) from 11.4-STABLE=C2=A0 to 13.0-STABLE just on time= to catch the "Fix nd6 rib_action() handling". The performance boost is really impressive, but probably not only FIB_ALGO is responsible for it since there were also improvements in PF, lagg(4), aggregated=C2=A0 NICs utilize now iflib and so on. Getting to the point: now when the links are saturated with normal traffic the load on this 8 core ATOM dropped from ~3.5 to ~0.9. The pmc(3) has changed a bit over time and now with PMC: [cpu_clk_unhalted.ref] I get max 1.5% for=C2=A0 rn_match function, b= ut the caller is always pfr_match_addr, so it comes likely from PF. In 11.4=C2=A0 rn_match was peaking up to=C2=A0 9% with links saturated with = normal traffic but there were more callers: fib6_lookup_nh_basic:4.2 pfr_match_addr:1.4 fib4_lookup_nh_basic:1.4. Now only occasionally I observe function=C2=A0 rn_walktree peaking up to ~20% called by rib_walk = what probably comes from net/bird adding/removing routes. At this stage (updating routes) on 11.x-STABLE bird seemed to block/stale the ability to interact with the system for a fraction of second. Now all updates go smooth. So congrats, an excellent job was done, thank you very much for this effort and time spent on making this all happen. Please let me ask only one final question. Does adding: net.route.algo.inet.algo=3Ddpdk_lpm4 net.route.algo.inet6.algo=3Ddpdk_lpm6 to /etc/sysctl.conf make any sense?=C2=A0 I see that right FIB_ALGO is automatically picked up when the module is available. After first reboot into 13-STABLE: Feb 23 22:42:10 rtr kernel: [fib_algo] fib_module_register: attaching radix6_lockless to inet6 Feb 23 22:42:10 rtr kernel: [fib_algo] fib_module_register: attaching radix6 to inet6 Feb 23 22:42:10 rtr kernel: [fib_algo] fib_module_register: attaching bsearch4 to inet Feb 23 22:42:10 rtr kernel: [fib_algo] fib_module_register: attaching radix4_lockless to inet Feb 23 22:42:10 rtr kernel: [fib_algo] fib_module_register: attaching radix4 to inet Feb 23 22:42:10 rtr kernel: [33] [fib_algo] fib_module_register: attaching dpdk_lpm6 to inet6 Feb 23 22:42:10 rtr kernel: [33] [fib_algo] fib_module_register: attaching dpdk_lpm4 to inet Feb 23 22:42:10 rtr kernel: [43] [fib_algo] inet.0 (bsearch4#66) rebuild_fd: switching algo to radix4_lockless Feb 23 22:42:10 rtr kernel: [49] [fib_algo] inet.1 (bsearch4#67) rebuild_fd: switching algo to radix4_lockless Feb 23 22:42:39 rtr kernel: [80] [fib_algo] inet.0 (radix4_lockless#650) rebuild_fd: switching algo to dpdk_lpm4 Feb 23 22:42:42 rtr kernel: [83] [fib_algo] inet6.0 (radix6_lockless#705) rebuild_fd: switching algo to dpdk_lpm6 Though after second reboot with "Fix nd6 rib_action() handling" applied there is no "switching algo to dpdk_lpm6": Feb 23 23:55:24 rtr kernel: [fib_algo] fib_module_register: attaching radix6_lockless to inet6 Feb 23 23:55:24 rtr kernel: [fib_algo] fib_module_register: attaching radix6 to inet6 Feb 23 23:55:24 rtr kernel: [fib_algo] fib_module_register: attaching bsearch4 to inet Feb 23 23:55:24 rtr kernel: [fib_algo] fib_module_register: attaching radix4_lockless to inet Feb 23 23:55:24 rtr kernel: [fib_algo] fib_module_register: attaching radix4 to inet Feb 23 23:55:24 rtr kernel: [10] [fib_algo] fib_module_register: attaching dpdk_lpm6 to inet6 Feb 23 23:55:24 rtr kernel: [10] [fib_algo] fib_module_register: attaching dpdk_lpm4 to inet Feb 23 23:55:24 rtr kernel: [20] [fib_algo] inet.0 (bsearch4#66) rebuild_fd: switching algo to radix4_lockless Feb 23 23:55:24 rtr kernel: [26] [fib_algo] inet.1 (bsearch4#67) rebuild_fd: switching algo to radix4_lockless Feb 23 23:55:57 rtr kernel: [61] [fib_algo] inet.0 (radix4_lockless#571) rebuild_fd: switching algo to dpdk_lpm4 Should I be bothered about it? With kind regards, --=20 Marek Zarychta
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?7056db64-2e7f-6ad8-54a3-59ff406061cf>