Date: Fri, 24 Apr 2020 14:12:02 -0700 From: Xin Li <delphij@delphij.net> To: Kristof Provost <kp@FreeBSD.org>, d@delphij.net Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Subject: Re: CFT: if_bridge performance improvements Message-ID: <5246fd44-b856-5402-092e-9065d4d4f7ae@delphij.net> In-Reply-To: <544E27A6-D799-4AF3-B4B7-1E68D5D50698@FreeBSD.org> References: <5377E42E-4C01-4BCC-B934-011AC3448B54@FreeBSD.org> <8e0e2bf1-27cd-1a99-b266-c7223255942f@delphij.net> <BF81FE6C-D4F4-43BA-9DE1-2C6A28A65AF3@FreeBSD.org> <8634ec5c-a509-d2dd-8f5c-31efcbd50340@delphij.net> <544E27A6-D799-4AF3-B4B7-1E68D5D50698@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --OL3HcfXl96oGJZBGGYPqcYw9drokEEWjC Content-Type: multipart/mixed; boundary="bBbUDl9mUraOpX4O489N3sYhzSlAUBRFO"; protected-headers="v1" From: Xin Li <delphij@delphij.net> Reply-To: d@delphij.net To: Kristof Provost <kp@FreeBSD.org>, d@delphij.net Cc: freebsd-current@freebsd.org, freebsd-stable@freebsd.org Message-ID: <5246fd44-b856-5402-092e-9065d4d4f7ae@delphij.net> Subject: Re: CFT: if_bridge performance improvements References: <5377E42E-4C01-4BCC-B934-011AC3448B54@FreeBSD.org> <8e0e2bf1-27cd-1a99-b266-c7223255942f@delphij.net> <BF81FE6C-D4F4-43BA-9DE1-2C6A28A65AF3@FreeBSD.org> <8634ec5c-a509-d2dd-8f5c-31efcbd50340@delphij.net> <544E27A6-D799-4AF3-B4B7-1E68D5D50698@FreeBSD.org> In-Reply-To: <544E27A6-D799-4AF3-B4B7-1E68D5D50698@FreeBSD.org> --bBbUDl9mUraOpX4O489N3sYhzSlAUBRFO Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 4/24/20 06:42, Kristof Provost wrote: > On 22 Apr 2020, at 18:15, Xin Li wrote: >> On 4/22/20 01:45, Kristof Provost wrote: >>> On 22 Apr 2020, at 10:20, Xin Li wrote: >>>> Hi, >>>> >>>> On 4/14/20 02:51, Kristof Provost wrote: >>>>> Hi, >>>>> >>>>> Thanks to support from The FreeBSD Foundation I=E2=80=99ve been abl= e to >>>>> work on >>>>> improving the throughput of if_bridge. >>>>> It changes the (data path) locking to use the NET_EPOCH >>>>> infrastructure. >>>>> Benchmarking shows substantial improvements (x5 in test setups). >>>>> >>>>> This work is ready for wider testing now. >>>>> >>>>> It=E2=80=99s under review here: https://reviews.freebsd.org/D24250 >>>>> >>>>> Patch for CURRENT: https://reviews.freebsd.org/D24250?download=3Dtr= ue >>>>> Patches for stable/12: >>>>> https://people.freebsd.org/~kp/if_bridge/stable_12/ >>>>> >>>>> I=E2=80=99m not currently aware of any panics or issues resulting f= rom these >>>>> patches. >>>> >>>> I have observed the following panic with latest stable/12 after >>>> applying >>>> the stable_12 patchset, it appears like a race condition related NUL= L >>>> pointer deference, but I haven't took a deeper look yet. >>>> >>>> The box have 7 igb(4) NICs, with several bridge and VLAN configured >>>> acting as a router.=C2=A0 Please let me know if you need additional >>>> information; I can try -CURRENT as well, but it would take some time= as >>>> the box is relatively slow (it's a ZFS based system so I can create = a >>>> separate boot environment for -CURRENT if needed, but that would tak= e >>>> some time as I might have to upgrade the packages, should there be a= ny >>>> ABI breakages). >>>> >>> Thanks for the report. I don=E2=80=99t immediately see how this could= happen. >>> >>> Are you running an L2 firewall on that bridge by any chance? An earli= er >>> version of the patch had issues with a stray unlock in that code path= =2E >> >> I don't think I have a L2 firewall (I assume means filtering based on >> MAC address like what can be done with e.g. ipfw?=C2=A0 The bridges we= re >> created on vlan interfaces though, do they count as L2 firewall?), the= >> system is using pf with a few NAT rules: >> >=20 > That backtrace looks identical to the one Peter reported, up to and > including the offset in the bridge_input() function. > Given that there=E2=80=99s no likely way to end up with a NULL mutex ei= ther I > have to assume that it=E2=80=99s a case of trying to unlock a locked mu= tex, and > the most likely reason is that you ran into the same problem Peter ran > into. >=20 > The current version of the patch should resolve it. Thanks, I'd like to report that after applying the patch from Peter the system seems to survive without problem. Cheers, --bBbUDl9mUraOpX4O489N3sYhzSlAUBRFO-- --OL3HcfXl96oGJZBGGYPqcYw9drokEEWjC Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.2.20 (Darwin) iQIzBAEBCgAdFiEEceNg5NEMZIki80nQQHl/fJX0g08FAl6jViUACgkQQHl/fJX0 g08YSQ//ehsN8qEaDAOVOCamsRCb4l5MiW/0i9gulOLsRbEN6Rm4ZRWDa+5yCTyq p0bs1RlOhspYOZhVkwtw7gITCrJ29LdHM7VIxVzaHOskuh6zTGcnUG6PZMHKw62H fA54Yzt20pRpYYS+L/LrKsQUaaJRsZNzMnASchANunuEQcaAuGZJIBG+nZXkM1rQ TRKJv6F9sxAkE+CtWB73AB1isV7IREDrcLgi/C7CIrehqD/nvBdpF35+JflFIe0f s2NHZZNyjoFm6Fxbs7tWfbit+hLDY5pVMHyB/ItsIyFToB0gvWWuNAGasLP4Pimn b5yKyGKb1KXPzIrtkvcLFhRHwqPBqiGeOrm/657MHJ8IxvaXzs6/UGuy2uqZU2iP NfPWVkvU6oP3TZSVkedysyk5/UCWhslCfdAnUP/tT8QqKI5lpbmDsvnCBXN0rNbi uzKnlbMmatqNgLLgw8D9h/MvXj9ZdzUKm2Y1wYF/Jzzag0BBTR3IsOV4BxKmvpX2 e5GwHaa6y/fwHdjIO4K9KWg7y1b6XUFTgp1sxm/0c5GnrmSj29wHUdDntse8yAqm PdWpyU/vwfV2Uc1FIJBpWYnKiWZPx4GgULSKLwliZPCaKeoTqStmE00rfLmtnPPd o2n7dv+oH4ybM2qOkVt55N1TGpGT3HlIne5SWqsNG7aaMWk46p4= =Kjcw -----END PGP SIGNATURE----- --OL3HcfXl96oGJZBGGYPqcYw9drokEEWjC--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5246fd44-b856-5402-092e-9065d4d4f7ae>