From nobody Thu Aug 25 00:30:24 2022 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MCkQP4d4Fz4ZF9l for ; Thu, 25 Aug 2022 00:30:37 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from www121.sakura.ne.jp (www121.sakura.ne.jp [153.125.133.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4MCkQN1CNQz3qg5; Thu, 25 Aug 2022 00:30:35 +0000 (UTC) (envelope-from junchoon@dec.sakura.ne.jp) Received: from kalamity.joker.local (123-1-85-147.area1b.commufa.jp [123.1.85.147]) (authenticated bits=0) by www121.sakura.ne.jp (8.16.1/8.16.1/[SAKURA-WEB]/20201212) with ESMTPA id 27P0UORv025450; Thu, 25 Aug 2022 09:30:25 +0900 (JST) (envelope-from junchoon@dec.sakura.ne.jp) Date: Thu, 25 Aug 2022 09:30:24 +0900 From: Tomoaki AOKI To: mike tancsa Cc: mike.jakubik@swiftsmsgateway.com, "pieper, jeffrey e" , jim king , "stable@freebsd.org" , "kbowling@freebsd.org" Subject: Re: igc problems with heavy traffic Message-Id: <20220825093024.60cf0c6d026644bb83036665@dec.sakura.ne.jp> In-Reply-To: <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net> References: <59b9cec0-d8c2-ce72-b5e9-99d1a1e807f8@sentex.net> <86995d10-af63-d053-972e-dd233029f3bf@jimking.net> <3d874f65-8ce2-8f06-f19a-14cd550166e3@sentex.net> <879b9239-2b9a-f0ae-4173-4a226c84cd85@sentex.net> <182d22a6c6d.1119560c11283607.2998737705092721009@swiftsmsgateway.com> <2fa9c9d7-1eb9-e7ad-1c19-a6202ac7082b@sentex.net> Organization: Junchoon corps X-Mailer: Sylpheed 3.7.0 (GTK+ 2.24.33; amd64-portbld-freebsd13.0) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4MCkQN1CNQz3qg5 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of junchoon@dec.sakura.ne.jp has no SPF policy when checking 153.125.133.21) smtp.mailfrom=junchoon@dec.sakura.ne.jp X-Spamd-Result: default: False [-1.60 / 15.00]; NEURAL_HAM_LONG(-1.00)[-1.000]; AUTH_NA(1.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; NEURAL_HAM_SHORT(-1.00)[-1.000]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[text/plain]; R_SPF_NA(0.00)[no SPF record]; MLMMJ_DEST(0.00)[stable@freebsd.org]; TO_DN_EQ_ADDR_SOME(0.00)[]; R_DKIM_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; ASN(0.00)[asn:7684, ipnet:153.125.128.0/18, country:JP]; MIME_TRACE(0.00)[0:+]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCPT_COUNT_FIVE(0.00)[6]; RCVD_COUNT_TWO(0.00)[2]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; HAS_ORG_HEADER(0.00)[]; FROM_HAS_DN(0.00)[]; RCVD_TLS_LAST(0.00)[]; TO_DN_SOME(0.00)[]; DMARC_NA(0.00)[sakura.ne.jp]; MID_RHS_MATCH_FROM(0.00)[] X-ThisMailContainsUnwantedMimeParts: N On Wed, 24 Aug 2022 19:37:43 -0400 mike tancsa wrote: > On 8/24/2022 7:22 PM, Mike Jakubik wrote: > > What kind of HW are you running on? Im assuming some sort of fairly > > modern x86 CPU with at least 4 cores.. Is it multiple CPUs with Numa > > nodes perhaps? In any case, if you are testing with iperf3, try using > > cpuset on iperf3 to bind it to specific cores. I had a performance > > issue on a modern Epyc server with a Mellanox 25Gb card. It turns out > > the issue was with the scheduler and how it was bouncing the processes > > around diff cores/CPU caches. See "Poor performance with stable/13 and > > Mellanox ConnectX-6 (mlx5)" on the freebsd-net mailing list for details. > > > > P.S. I also use a number of igc (Intel i225 @ 2.5Gb) cards at home and > > have had no issues with them. > > > > > Hi, > >     Performance is excellent. Its just the random link drops thats at > issue.  With default settings, running iperf3 on back to back NICs via > xover takes a good 20-45min before the link drop. If anything, I am > surprised at how much traffic these small devices can forward.  IPSEC > especially is super fast on RELENG_13. The link drops seem to be always > on the sender.  With fc disabled, reducing the link speed to 1G seems to > make the issue go away, or at least its not happening in overnight > testing. Its a Celeron N5105. https :// > www.aliexpress.com/item/1005003990581434.html > > Also, if you hook a couple back to back via xover cable, are you able to > manually set the speed to 1G and pass traffic ? It doesnt work for me. > >     ---Mike FYI: https://en.wikipedia.org/wiki/Medium-dependent_interface Maybe you should use straight-through cable for 1G or faster. -- Tomoaki AOKI