From nobody Wed Aug 24 23:52:12 2022 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MCjZD1Tbhz4Z9vn for ; Wed, 24 Aug 2022 23:52:20 +0000 (UTC) (envelope-from michal.jakubik@zoho.com) Received: from sender4-op-o13.zoho.com (sender4-op-o13.zoho.com [136.143.188.13]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MCjZC113Wz3mMt; Wed, 24 Aug 2022 23:52:19 +0000 (UTC) (envelope-from michal.jakubik@zoho.com) ARC-Seal: i=1; a=rsa-sha256; t=1661385134; cv=none; d=zohomail.com; s=zohoarc; b=au+JpilOOLgDLlpPUqUHrzCwRq4Ao7cmZMYhuKbSMk8nizn5QyZMtEawr20qn2JltQ19yX2UtaBhDRF3Dw+etZ704jU3VEsxmIVpXh+PmDv70NLfyj3xDq21zZhUdVJHiQZSqIUTZND1oGSsPK89UcgRONvAJGqXPMCkNF2JPYc= ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=zohomail.com; s=zohoarc; t=1661385134; h=Content-Type:Cc:Date:From:In-Reply-To:MIME-Version:Message-ID:Reply-To:References:Subject:To; bh=txR4IBZkzBgcQp7qm1q4J9nw/t2aQWQ1z39cqiiOD2E=; b=lStKlNgznqKidw/OUPU8KeMcMo07OhqX/TUB87HQaqWTHDCOYFgTb83dEDm4qxEutsQ5QpyXLNB0BxwR8fDMb6NqV3hM5a/jziwGHg9M4VJbR8Ow+BDRCACqG0h5OX5Lhpxd6AV7qL+4n4mdMcTgrqIyqpcfzB+JMI5JJaoeujU= ARC-Authentication-Results: i=1; mx.zohomail.com; dkim=pass header.i=swiftsmsgateway.com; spf=pass smtp.mailfrom=michal.jakubik@zoho.com; dmarc=pass header.from= DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=zapps768; d=zoho.com; h=date:from:reply-to:to:cc:message-id:in-reply-to:references:subject:mime-version:content-type:user-agent; b=syQcpTiaesv/ch333XE12fD9GmlLgdjQJEFaTaDgdVda2zwnTv98L6n/+k0JSVwkvfGMctUyDN7K YqsB4fy5XD8xXHoUh0i3j/Ifck9t+22jFxoOq4VoVQq0uyk3PwLz DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1661385134; s=zoho; d=swiftsmsgateway.com; i=mike.jakubik@swiftsmsgateway.com; h=Date:Date:From:From:Reply-To:Reply-To:To:To:Cc:Cc:Message-Id:Message-Id:In-Reply-To:References:Subject:Subject:MIME-Version:Content-Type; bh=txR4IBZkzBgcQp7qm1q4J9nw/t2aQWQ1z39cqiiOD2E=; b=uwuR5lGqr90cGb7XOJt50cZMABoqg3Rv+ISayQgq5JHK2cUNBfClsnJ8b7VFqgIA fGR2BXcilZTohiqJl813xr1ydt1uzRaJMB8vTm3TZSWgZrWrvN/8vDXeKZfHnKvhl0b /yElIJ7FIoaej6APi+o0vffbUt84r0VZ3sNr9YWg= Received: from mail.zoho.com by mx.zohomail.com with SMTP id 1661385132032791.9117103050088; Wed, 24 Aug 2022 16:52:12 -0700 (PDT) Date: Wed, 24 Aug 2022 19:52:12 -0400 From: Mike Jakubik Reply-To: mike.jakubik@swiftsmsgateway.com To: "mike tancsa" Cc: "pieper, jeffrey e" , "jim king" , "stable@freebsd.org" , "kbowling@freebsd.org" Message-Id: <182d24547e1.d8e201341286492.219437989220339304@swiftsmsgateway.com> In-Reply-To: <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net> References: <59b9cec0-d8c2-ce72-b5e9-99d1a1e807f8@sentex.net> <86995d10-af63-d053-972e-dd233029f3bf@jimking.net> <3d874f65-8ce2-8f06-f19a-14cd550166e3@sentex.net> <879b9239-2b9a-f0ae-4173-4a226c84cd85@sentex.net> <182d22a6c6d.1119560c11283607.2998737705092721009@swiftsmsgateway.com> <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net> Subject: Re: igc problems with heavy traffic List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: multipart/alternative; boundary="----=_Part_4003905_1296152876.1661385132001" Importance: Medium User-Agent: Zoho Mail X-Mailer: Zoho Mail X-Rspamd-Queue-Id: 4MCjZC113Wz3mMt X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=swiftsmsgateway.com header.s=zoho header.b=uwuR5lGq; arc=pass ("zohomail.com:s=zohoarc:i=1"); dmarc=none; spf=pass (mx1.freebsd.org: domain of michal.jakubik@zoho.com designates 136.143.188.13 as permitted sender) smtp.mailfrom=michal.jakubik@zoho.com X-Spamd-Result: default: False [-4.08 / 15.00]; ARC_ALLOW(-1.00)[zohomail.com:s=zohoarc:i=1]; NEURAL_HAM_MEDIUM(-1.00)[-0.999]; NEURAL_HAM_SHORT(-0.99)[-0.995]; NEURAL_HAM_LONG(-0.90)[-0.897]; FORGED_SENDER(0.30)[mike.jakubik@swiftsmsgateway.com,michal.jakubik@zoho.com]; R_SPF_ALLOW(-0.20)[+ip4:136.143.188.0/24]; R_DKIM_ALLOW(-0.20)[swiftsmsgateway.com:s=zoho]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; XM_UA_NO_VERSION(0.01)[]; FROM_HAS_DN(0.00)[]; HAS_REPLYTO(0.00)[mike.jakubik@swiftsmsgateway.com]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DMARC_NA(0.00)[swiftsmsgateway.com]; RCVD_IN_DNSWL_NONE(0.00)[136.143.188.13:from]; ASN(0.00)[asn:2639, ipnet:136.143.188.0/23, country:US]; MID_RHS_MATCH_FROM(0.00)[]; MLMMJ_DEST(0.00)[stable@freebsd.org]; REPLYTO_ADDR_EQ_FROM(0.00)[]; TO_DN_SOME(0.00)[]; TO_DN_EQ_ADDR_SOME(0.00)[]; FROM_NEQ_ENVFROM(0.00)[mike.jakubik@swiftsmsgateway.com,michal.jakubik@zoho.com]; RCPT_COUNT_FIVE(0.00)[5]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[swiftsmsgateway.com:+]; FREEMAIL_ENVFROM(0.00)[zoho.com]; MIME_TRACE(0.00)[0:+,1:+,2:~]; RCVD_COUNT_TWO(0.00)[2] X-ThisMailContainsUnwantedMimeParts: N ------=_Part_4003905_1296152876.1661385132001 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Ah, sorry about the noise then. I cant remember if I ran them connected dir= ectly, I may have briefly for testing but I certainly did not notice any dr= op issues. I am just using a built in switch with the TP Link Archer-ax1100= 0 WiFi router which connects to my Internet at 2.5Gb and the local wired co= nnections are at 1Gb. No drop issues with that. My initial thought would be= that the problem is with your connection media, but I'm guessing you alrea= dy troubleshooted that. ---- On Wed, 24 Aug 2022 19:37:43 -0400 mike tancsa wrote= --- On 8/24/2022 7:22 PM, Mike Jakubik wrote: What kind of HW are you running on? Im assuming some sort of fairly modern x86 CPU with at least 4 cores.. Is it multiple CPUs with Numa nodes perhaps? In any case, if you are testing with iperf3, try using cpuset on iperf3 to bind it to specific cores. I had a performance issue on a modern Epyc server with a Mellanox 25Gb card. It turns out the issue was with the scheduler and how it was bouncing the processes around diff cores/CPU caches. See "Poor performance with stable/13 and Mellanox ConnectX-6 (mlx5)" on the freebsd-net mailing list for details. P.S. I also use a number of igc (Intel i225 @ 2.5Gb) cards at home and have had no issues with them. Hi, =C2=A0=C2=A0=C2=A0 Performance is excellent. Its just the random link drops thats at issue.=C2=A0 With default settings, running iperf3 on back t= o back NICs via xover takes a good 20-45min before the link drop.=C2=A0 If anything, I am surprised at how much traffic these small devices can forward.=C2=A0 IPSEC especially is super fast on RELENG_1= 3. The link drops seem to be always on the sender.=C2=A0 With fc disable= d, reducing the link speed to 1G seems to make the issue go away, or at least its not happening in overnight testing. Its a Celeron N5105. https :// http://www.aliexpress.com/item/1005003990581434.html Also, if you hook a couple back to back via xover cable, are you able to manually set the speed to 1G and pass traffic ? It doesnt work for me. =C2=A0=C2=A0=C2=A0 ---Mike Mike Jakubik https://www.swiftsmsgateway.com/ Disclaimer: This e-mail and any attachments are intended only for the use o= f the addressee(s) and may contain information that is privileged or confid= ential. If you are not the intended recipient, or responsible for deliverin= g the information to the intended recipient, you are hereby notified that a= ny dissemination, distribution, printing or copying of this e-mail and any = attachments is strictly prohibited. If this e-mail and any attachments were= received in error, please notify the sender by reply e-mail and delete the= original message. ------=_Part_4003905_1296152876.1661385132001 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable =
Ah, sorry about the noise then. I cant remember if= I ran them connected directly, I may have briefly for testing but I certai= nly did not notice any drop issues. I am just using a built in switch with = the TP Link Archer-ax11000 WiFi router which connects to my Internet at 2.5= Gb and the local wired connections are at 1Gb. No drop issues with that. My= initial thought would be that the problem is with your connection media, b= ut I'm guessing you already troubleshooted that.


=

=
---- On Wed, 24 Aug 2022 19:37:43 -0400 m= ike tancsa <mike@sentex.net> wrote ---

On 8/24/2022 7:22 PM, Mike Jakubik wrote:
What kind of HW are you ru= nning on? Im assuming some sort of fairly modern x86 CPU with at least 4 cores.. Is it multiple CPUs with Numa nodes perhaps? In any case, if you are testing with iperf3, try using cpuset on iperf3 to bind it to specific cores. I had a performance issue on a modern Epyc server with a Mellanox 25Gb card. It turns out the issue was with the scheduler and how it was bouncing the processes around diff cores/CPU caches. See "Poor performance with stable/13 and Mellanox ConnectX-6 (mlx5)" on the freebsd-net mailing list for details.

P.S. I als= o use a number of igc (Intel i225 @ 2.5Gb) cards at home and have had no issues with them.


=

Hi,

    Performance is excellent. = Its just the random link drops thats at issue.  With default settings, running iperf3 on back t= o back NICs via xover takes a good 20-45min before the link drop.  If anything, I am surprised at how much traffic these small devices can forward.  IPSEC especially is super fast on RELENG_1= 3. The link drops seem to be always on the sender.  With fc disable= d, reducing the link speed to 1G seems to make the issue go away, or at least its not happening in overnight testing. Its a Celeron N5105. https :// www.aliexpress.com/item/1005003990581434.html

Also, if you= hook a couple back to back via xover cable, are you able to manually set the speed to 1G and pass traffic ? It doesnt work for me.

    ---Mike


Disclaimer:= This e-mail and any attachments are intended only for the use of the addre= ssee(s) and may contain information that is privileged or confidential. If = you are not the intended recipient, or responsible for delivering the infor= mation to the intended recipient, you are hereby notified that any dissemin= ation, distribution, printing or copying of this e-mail and any attachments= is strictly prohibited. If this e-mail and any attachments were received i= n error, please notify the sender by reply e-mail and delete the original m= essage.


------=_Part_4003905_1296152876.1661385132001--