Date: Fri, 14 Feb 2020 15:21:11 +0300 From: "Andrey V. Elsukov" <bu7cher@yandex.ru> To: Rudy <crapsh@monkeybrains.net>, net@FreeBSD.org Subject: Re: Issue with BGP router / high interrupt / Chelsio / FreeBSD 12.1 Message-ID: <d3d4de8b-879a-f485-69a5-0567d97ac49b@yandex.ru> In-Reply-To: <bb6c3997-c369-28c3-9d85-c9cca526e093@monkeybrains.net> References: <1aa78c6e-e640-623c-73d3-473df132eb72@monkeybrains.net> <c921825a-3a9c-cc15-78e6-c7e3776ab12a@monkeybrains.net> <bb6c3997-c369-28c3-9d85-c9cca526e093@monkeybrains.net>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --0rituGGqk0svUhhHJDarfuF0ECFBrtLrM Content-Type: multipart/mixed; boundary="qUTRyfeZIqvBMwWHtnsWDC0Y95CDVZfca"; protected-headers="v1" From: "Andrey V. Elsukov" <bu7cher@yandex.ru> To: Rudy <crapsh@monkeybrains.net>, net@FreeBSD.org Message-ID: <d3d4de8b-879a-f485-69a5-0567d97ac49b@yandex.ru> Subject: Re: Issue with BGP router / high interrupt / Chelsio / FreeBSD 12.1 References: <1aa78c6e-e640-623c-73d3-473df132eb72@monkeybrains.net> <c921825a-3a9c-cc15-78e6-c7e3776ab12a@monkeybrains.net> <bb6c3997-c369-28c3-9d85-c9cca526e093@monkeybrains.net> In-Reply-To: <bb6c3997-c369-28c3-9d85-c9cca526e093@monkeybrains.net> --qUTRyfeZIqvBMwWHtnsWDC0Y95CDVZfca Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 13.02.2020 06:21, Rudy wrote: >=20 >=20 > I'm having issues with a box that is acting as a BGP router for my > network.=C2=A0 3 Chelsio cards, two T5 and one T6.=C2=A0 It was working= great > until I turned up our first port on the T6.=C2=A0 It seems like traffic= > passing in from a T5 card and out the T6 causes a really high load (and= > high interrupts). >=20 > Traffic (not that much, right?) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0 Dev=C2=A0 RX bps=C2=A0=C2=A0=C2=A0 TX bps=C2=A0= =C2=A0=C2=A0 RX PPS=C2=A0=C2=A0=C2=A0 TX PPS Error > =C2=A0=C2=A0=C2=A0=C2=A0 cc0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0=C2=A0 cc1=C2=A0=C2=A0=C2=A0 2212 M=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 7 M=C2=A0=C2=A0=C2=A0=C2=A0 250 k=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 6 k 0 (100Gbps uplink, > filtering inbound routes to keep TX low) > =C2=A0=C2=A0=C2=A0 cxl0=C2=A0=C2=A0=C2=A0=C2=A0 287 k=C2=A0=C2=A0=C2=A0= 2015 M=C2=A0=C2=A0=C2=A0=C2=A0 353=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 2= 44 k 0=C2=A0=C2=A0 (our network) > =C2=A0=C2=A0=C2=A0 cxl1=C2=A0=C2=A0=C2=A0=C2=A0 940 M=C2=A0=C2=A0=C2=A0= 3115 M=C2=A0=C2=A0=C2=A0=C2=A0 176 k=C2=A0=C2=A0=C2=A0=C2=A0 360 k 0 (ou= r network) > =C2=A0=C2=A0=C2=A0 cxl2=C2=A0=C2=A0=C2=A0=C2=A0 634 M=C2=A0=C2=A0=C2=A0= 1014 M=C2=A0=C2=A0=C2=A0=C2=A0 103 k=C2=A0=C2=A0=C2=A0=C2=A0 128 k 0 (ou= r network) > =C2=A0=C2=A0=C2=A0 cxl3=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1 k=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 16 M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4 k=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 0 > =C2=A0=C2=A0=C2=A0 cxl4=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0 cxl5=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0=C2=A0 cxl6=C2=A0=C2=A0=C2=A0 2343 M=C2=A0=C2=A0=C2=A0=C2=A0= 791 M=C2=A0=C2=A0=C2=A0=C2=A0 275 k=C2=A0=C2=A0=C2=A0=C2=A0 137 k 0 (IX = , part of lagg0) > =C2=A0=C2=A0=C2=A0 cxl7=C2=A0=C2=A0=C2=A0 1675 M=C2=A0=C2=A0=C2=A0=C2=A0= 762 M=C2=A0=C2=A0=C2=A0=C2=A0 215 k=C2=A0=C2=A0=C2=A0=C2=A0 133 k 0 (IX = , part of lagg0) > =C2=A0=C2=A0=C2=A0 ixl0=C2=A0=C2=A0=C2=A0=C2=A0 913 k=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 18 M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0 0 > =C2=A0=C2=A0=C2=A0 ixl1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1 M=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 30 M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0 lagg0=C2=A0=C2=A0=C2=A0 4019 M=C2=A0=C2=A0=C2=A0 1554 M=C2= =A0=C2=A0=C2=A0=C2=A0 491 k=C2=A0=C2=A0=C2=A0=C2=A0 271 k=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0 0 > =C2=A0=C2=A0 lagg1=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1 M=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 48 M=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0 0 > FreeBSD 12.1-STABLE orange=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 976 Bytes/Packetavg > =C2=A01:42PM=C2=A0 up 13:25, 5 users, load averages: 9.38, 10.43, 9.827= Hi, did you try to use pmcstat to determine what is the heaviest task for your system? # kldload hwpmc # pmcstat -S inst_retired.any -Tw1 Then capture several first lines from the output and quit using 'q'. Do you use some firewall? Also, can you show the snapshot from the `top -HPSIzts1` output. --=20 WBR, Andrey V. Elsukov --qUTRyfeZIqvBMwWHtnsWDC0Y95CDVZfca-- --0rituGGqk0svUhhHJDarfuF0ECFBrtLrM Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Comment: Using GnuPG with Thunderbird - https://www.enigmail.net/ iQEzBAEBCAAdFiEE5lkeG0HaFRbwybwAAcXqBBDIoXoFAl5GkLcACgkQAcXqBBDI oXoCpwf+NJlY3EQg1o3KiWc9Qvff1dUjsSbuX5KZWLILWcKuwPg1XBqcV/OIUYOf ItlLZa2RkdMNCNGue6Oi3PPgZ0Js69CCPYMoyE+WQfoUQhfG+fmgwXLli5GHZP5m vL1zL4ArhSKB/jHZL5aIPWisBJQ4TEBdyZC5l1UgrbrcKe3dHKVtwqfG8LDMsB+F ZhVhi+zDrWga8ioufa6lQRarnjzNJiirhH7qrocLfAOvUcw7atOg5MaFIVvxTFfe JcXEaMD44TgNaHuLdBIRbiSQSGN8037+9zP0rU9HUMHy7yHO/fJtQlIz0T0JgYr5 rJVdNm5/W8WGfXyNRFFcrnnvMTgPAQ== =FAKb -----END PGP SIGNATURE----- --0rituGGqk0svUhhHJDarfuF0ECFBrtLrM--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?d3d4de8b-879a-f485-69a5-0567d97ac49b>