Date: Wed, 24 Aug 2022 19:52:12 -0400 From: Mike Jakubik <mike.jakubik@swiftsmsgateway.com> To: "mike tancsa" <mike@sentex.net> Cc: "pieper, jeffrey e" <jeffrey.e.pieper@intel.com>, "jim king" <jim@jimking.net>, "stable@freebsd.org" <stable@freebsd.org>, "kbowling@freebsd.org" <kbowling@freebsd.org> Subject: Re: igc problems with heavy traffic Message-ID: <182d24547e1.d8e201341286492.219437989220339304@swiftsmsgateway.com> In-Reply-To: <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net> References: <fc256428-3ff1-68ba-cfcc-a00ca427e85b@jimking.net> <59b9cec0-d8c2-ce72-b5e9-99d1a1e807f8@sentex.net> <e714cd76-0aaa-3ea0-3c31-5e61badffa18@sentex.net> <86995d10-af63-d053-972e-dd233029f3bf@jimking.net> <3d874f65-8ce2-8f06-f19a-14cd550166e3@sentex.net> <a8192d60-2970-edb5-ce1a-c17ea875bf07@jimking.net> <fd1e825b-c306-64b1-f9ef-fec0344a9c95@sentex.net> <a4ddc96a-3dd5-4fee-8003-05f228d10858@jimking.net> <MW4PR11MB5890493674ADD1757BB47075D0659@MW4PR11MB5890.namprd11.prod.outlook.com> <a9935ba0-9cb2-5a41-ca73-b6962fef5e4d@sentex.net> <879b9239-2b9a-f0ae-4173-4a226c84cd85@sentex.net> <182d22a6c6d.1119560c11283607.2998737705092721009@swiftsmsgateway.com> <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net>
next in thread | previous in thread | raw e-mail | index | archive | help
------=_Part_4003905_1296152876.1661385132001
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Ah, sorry about the noise then. I cant remember if I ran them connected dir=
ectly, I may have briefly for testing but I certainly did not notice any dr=
op issues. I am just using a built in switch with the TP Link Archer-ax1100=
0 WiFi router which connects to my Internet at 2.5Gb and the local wired co=
nnections are at 1Gb. No drop issues with that. My initial thought would be=
that the problem is with your connection media, but I'm guessing you alrea=
dy troubleshooted that.
---- On Wed, 24 Aug 2022 19:37:43 -0400 mike tancsa <mike@sentex.net> wrote=
---
On 8/24/2022 7:22 PM, Mike Jakubik
wrote:
What kind of HW are you running on? Im assuming some sort
of fairly modern x86 CPU with at least 4 cores.. Is it
multiple CPUs with Numa nodes perhaps? In any case, if you are
testing with iperf3, try using cpuset on iperf3 to bind it to
specific cores. I had a performance issue on a modern Epyc
server with a Mellanox 25Gb card. It turns out the issue was
with the scheduler and how it was bouncing the processes
around diff cores/CPU caches. See "Poor performance with
stable/13 and Mellanox ConnectX-6 (mlx5)" on the freebsd-net
mailing list for details.
P.S. I also use a number of igc (Intel i225 @ 2.5Gb) cards
at home and have had no issues with them.
Hi,
=C2=A0=C2=A0=C2=A0 Performance is excellent. Its just the random link drops
thats at issue.=C2=A0 With default settings, running iperf3 on back t=
o
back NICs via xover takes a good 20-45min before the link drop.=C2=A0
If anything, I am surprised at how much traffic these small
devices can forward.=C2=A0 IPSEC especially is super fast on RELENG_1=
3.
The link drops seem to be always on the sender.=C2=A0 With fc disable=
d,
reducing the link speed to 1G seems to make the issue go away, or
at least its not happening in overnight testing. Its a Celeron
N5105. https :// http://www.aliexpress.com/item/1005003990581434.html
Also, if you hook a couple back to back via xover cable, are you
able to manually set the speed to 1G and pass traffic ? It doesnt
work for me.
=C2=A0=C2=A0=C2=A0 ---Mike
Mike Jakubik
https://www.swiftsmsgateway.com/
Disclaimer: This e-mail and any attachments are intended only for the use o=
f the addressee(s) and may contain information that is privileged or confid=
ential. If you are not the intended recipient, or responsible for deliverin=
g the information to the intended recipient, you are hereby notified that a=
ny dissemination, distribution, printing or copying of this e-mail and any =
attachments is strictly prohibited. If this e-mail and any attachments were=
received in error, please notify the sender by reply e-mail and delete the=
original message.
------=_Part_4003905_1296152876.1661385132001
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"><html><head>=
<meta content=3D"text/html;charset=3DUTF-8" http-equiv=3D"Content-Type"></h=
ead><body ><div style=3D"font-family: Verdana, Arial, Helvetica, sans-serif=
; font-size: 10pt;"><div>Ah, sorry about the noise then. I cant remember if=
I ran them connected directly, I may have briefly for testing but I certai=
nly did not notice any drop issues. I am just using a built in switch with =
the TP Link Archer-ax11000 WiFi router which connects to my Internet at 2.5=
Gb and the local wired connections are at 1Gb. No drop issues with that. My=
initial thought would be that the problem is with your connection media, b=
ut I'm guessing you already troubleshooted that.</div><div><br></div><div c=
lass=3D"zmail_extra_hr" style=3D"border-top: 1px solid rgb(204, 204, 204); =
height: 0px; margin-top: 10px; margin-bottom: 10px; line-height: 0px;"><br>=
</div><div class=3D"zmail_extra" data-zbluepencil-ignore=3D"true"><div><br>=
</div><div id=3D"Zm-_Id_-Sgn1">---- On Wed, 24 Aug 2022 19:37:43 -0400 <b>m=
ike tancsa <mike@sentex.net></b> wrote ---<br></div><div><br></div><b=
lockquote style=3D"margin: 0px;"><div><div class=3D"x_-788095332moz-cite-pr=
efix">On 8/24/2022 7:22 PM, Mike Jakubik
wrote:<br></div><blockquote><div style=3D"font-family: Verdana, Arial=
, Helvetica, sans-serif;font-size: 10.0pt;"><div>What kind of HW are you ru=
nning on? Im assuming some sort
of fairly modern x86 CPU with at least 4 cores.. Is it
multiple CPUs with Numa nodes perhaps? In any case, if you are
testing with iperf3, try using cpuset on iperf3 to bind it to
specific cores. I had a performance issue on a modern Epyc
server with a Mellanox 25Gb card. It turns out the issue was
with the scheduler and how it was bouncing the processes
around diff cores/CPU caches. See "Poor performance with
stable/13 and Mellanox ConnectX-6 (mlx5)" on the freebsd-net
mailing list for details.<br></div><div><br></div><div>P.S. I als=
o use a number of igc (Intel i225 @ 2.5Gb) cards
at home and have had no issues with them.<br></div><br></div><br>=
</blockquote><p>Hi,<br></p><p> Performance is excellent. =
Its just the random link drops
thats at issue. With default settings, running iperf3 on back t=
o
back NICs via xover takes a good 20-45min before the link drop.
If anything, I am surprised at how much traffic these small
devices can forward. IPSEC especially is super fast on RELENG_1=
3.
The link drops seem to be always on the sender. With fc disable=
d,
reducing the link speed to 1G seems to make the issue go away, or
at least its not happening in overnight testing. Its a Celeron
N5105. https :// <a class=3D"x_-788095332moz-txt-link-abbreviated" hr=
ef=3D"http://www.aliexpress.com/item/1005003990581434.html" target=3D"_blan=
k">www.aliexpress.com/item/1005003990581434.html</a><br></p><p>Also, if you=
hook a couple back to back via xover cable, are you
able to manually set the speed to 1G and pass traffic ? It doesnt
work for me.<br></p><p> ---Mike<br></p></div></bloc=
kquote></div><div><br></div><div class=3D"zmail_signature_below"><div id=3D=
"Zm-_Id_-Sgn" data-zbluepencil-ignore=3D"true" data-sigid=3D"78860770000000=
96004"><div>Mike Jakubik<br></div><div><a target=3D"_blank" href=3D"https:/=
/www.swiftsmsgateway.com/">https://www.swiftsmsgateway.com/</a><br></div><d=
iv><br></div><div><span class=3D"size" style=3D"font-size:10px">Disclaimer:=
This e-mail and any attachments are intended only for the use of the addre=
ssee(s) and may contain information that is privileged or confidential. If =
you are not the intended recipient, or responsible for delivering the infor=
mation to the intended recipient, you are hereby notified that any dissemin=
ation, distribution, printing or copying of this e-mail and any attachments=
is strictly prohibited. If this e-mail and any attachments were received i=
n error, please notify the sender by reply e-mail and delete the original m=
essage.</span><br></div></div></div><div><br></div></div><br></body></html>
------=_Part_4003905_1296152876.1661385132001--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?182d24547e1.d8e201341286492.219437989220339304>