From nobody Thu Aug 25 00:36:35 2022 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MCkYp4X1bz4ZGjs for ; Thu, 25 Aug 2022 00:37:02 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1j2.sentex.ca (smarthost1j2.sentex.ca [IPv6:2607:f3e0:0:c::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "smarthost1j2.sentex.ca", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MCkYn3xKgz3s0W; Thu, 25 Aug 2022 00:37:01 +0000 (UTC) (envelope-from mike@sentex.net) Received: from vinyl4.sentex.ca (vinyl4.sentex.ca [64.7.153.17]) by smarthost1j2.sentex.ca (8.16.1/8.15.2) with ESMTPS id 27P0apXr059036 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Thu, 25 Aug 2022 00:36:51 GMT (envelope-from mike@sentex.net) Received: from [192.168.251.134] (cage.simianscience.com [64.7.134.1]) (authenticated bits=0) by vinyl4.sentex.ca (8.15.2/8.15.2) with ESMTPA id 27P0aZeF056580; Wed, 24 Aug 2022 20:36:35 -0400 (EDT) (envelope-from mike@sentex.net) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.101.1 at vinyl4.sentex.ca From: mike tancsa To: Tomoaki AOKI CC: , "pieper, jeffrey e" , jim king , , Date: Wed, 24 Aug 2022 20:36:35 -0400 Message-ID: <182d26dea38.27dc.e68d32c7521a042b3773fe36a0156dc7@sentex.net> In-Reply-To: <20220825093024.60cf0c6d026644bb83036665@dec.sakura.ne.jp> References: <59b9cec0-d8c2-ce72-b5e9-99d1a1e807f8@sentex.net> <86995d10-af63-d053-972e-dd233029f3bf@jimking.net> <3d874f65-8ce2-8f06-f19a-14cd550166e3@sentex.net> <879b9239-2b9a-f0ae-4173-4a226c84cd85@sentex.net> <182d22a6c6d.1119560c11283607.2998737705092721009@swiftsmsgateway.com> <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net> <20220825093024.60cf0c6d026644bb83036665@dec.sakura.ne.jp> User-Agent: AquaMail/1.38.0 (build: 103800177) Subject: Re: igc problems with heavy traffic List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="us-ascii" Content-Transfer-Encoding: 8bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (vinyl4.sentex.ca [64.7.153.17]); Wed, 24 Aug 2022 20:36:35 -0400 (EDT) X-Scanned-By: MIMEDefang 2.83 on 64.7.153.17 X-Rspamd-Queue-Id: 4MCkYn3xKgz3s0W X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of mike@sentex.net designates 2607:f3e0:0:c::2 as permitted sender) smtp.mailfrom=mike@sentex.net X-Spamd-Result: default: False [-3.30 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-0.999]; R_SPF_ALLOW(-0.20)[+ip6:2607:f3e0::/32]; MIME_GOOD(-0.10)[text/plain]; MLMMJ_DEST(0.00)[stable@freebsd.org]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:11647, ipnet:2607:f3e0::/32, country:CA]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_TLS_LAST(0.00)[]; FREEFALL_USER(0.00)[mike]; ARC_NA(0.00)[]; RCVD_COUNT_THREE(0.00)[3]; FROM_HAS_DN(0.00)[]; DMARC_NA(0.00)[sentex.net]; TO_DN_SOME(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N Thanks, it was a straight through cat6. It works (with the reported occasional link drops) at autoneg, but if I specify 1g it fails to pass traffic On August 24, 2022 8:31:41 p.m. Tomoaki AOKI wrote: > On Wed, 24 Aug 2022 19:37:43 -0400 > mike tancsa wrote: > >> On 8/24/2022 7:22 PM, Mike Jakubik wrote: >> > What kind of HW are you running on? Im assuming some sort of fairly >> > modern x86 CPU with at least 4 cores.. Is it multiple CPUs with Numa >> > nodes perhaps? In any case, if you are testing with iperf3, try using >> > cpuset on iperf3 to bind it to specific cores. I had a performance >> > issue on a modern Epyc server with a Mellanox 25Gb card. It turns out >> > the issue was with the scheduler and how it was bouncing the processes >> > around diff cores/CPU caches. See "Poor performance with stable/13 and >> > Mellanox ConnectX-6 (mlx5)" on the freebsd-net mailing list for details. >> > >> > P.S. I also use a number of igc (Intel i225 @ 2.5Gb) cards at home and >> > have had no issues with them. >> > >> > >> Hi, >> >> Performance is excellent. Its just the random link drops thats at >> issue. With default settings, running iperf3 on back to back NICs via >> xover takes a good 20-45min before the link drop. If anything, I am >> surprised at how much traffic these small devices can forward. IPSEC >> especially is super fast on RELENG_13. The link drops seem to be always >> on the sender. With fc disabled, reducing the link speed to 1G seems to >> make the issue go away, or at least its not happening in overnight >> testing. Its a Celeron N5105. https :// >> www.aliexpress.com/item/1005003990581434.html >> >> Also, if you hook a couple back to back via xover cable, are you able to >> manually set the speed to 1G and pass traffic ? It doesnt work for me. >> >> ---Mike > > FYI: > https://en.wikipedia.org/wiki/Medium-dependent_interface > > Maybe you should use straight-through cable for 1G or faster. > > > -- > Tomoaki AOKI