From owner-freebsd-net@freebsd.org Thu Oct 1 16:57:22 2020 Return-Path: Delivered-To: freebsd-net@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D9E6542DD86 for ; Thu, 1 Oct 2020 16:57:22 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4C2K6n691tz4PgK for ; Thu, 1 Oct 2020 16:57:21 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1kO1tH-000858-FL; Thu, 01 Oct 2020 19:57:11 +0300 Date: Thu, 1 Oct 2020 19:57:11 +0300 From: Slawa Olhovchenkov To: Hans Petter Selasky Cc: Michal =?utf-8?B?VmFuxI1v?= , freebsd-net@freebsd.org Subject: Re: mlx5 irq Message-ID: <20201001165708.GE2033@zxy.spb.ru> References: <0aa09fcc-dfcc-005e-8834-2a758ba6a03f@microwave.sk> <94978a05-94c6-cc55-229c-5a3c5352b29a@selasky.org> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <94978a05-94c6-cc55-229c-5a3c5352b29a@selasky.org> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-Rspamd-Queue-Id: 4C2K6n691tz4PgK X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=none (mx1.freebsd.org: domain of slw@zxy.spb.ru has no SPF policy when checking 195.70.199.98) smtp.mailfrom=slw@zxy.spb.ru X-Spamd-Result: default: False [-0.78 / 15.00]; MID_RHS_MATCH_FROM(0.00)[]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.41)[-0.413]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-0.82)[-0.823]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[zxy.spb.ru]; AUTH_NA(1.00)[]; TO_MATCH_ENVRCPT_SOME(0.00)[]; NEURAL_HAM_SHORT(-0.44)[-0.443]; R_SPF_NA(0.00)[no SPF record]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:5495, ipnet:195.70.192.0/19, country:RU]; MIME_TRACE(0.00)[0:+]; MAILMAN_DEST(0.00)[freebsd-net]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 01 Oct 2020 16:57:22 -0000 On Thu, Oct 01, 2020 at 10:10:42AM +0200, Hans Petter Selasky wrote: > On 2020-10-01 09:39, Michal Vančo via freebsd-net wrote: > > Hi > > Hi Michal, > > > I have a server with one Mellanox ConnectX-4 adapter and the following > > CPU configuration (SMT disabled): > > > > # dmesg | grep SMP > > FreeBSD/SMP: Multiprocessor System Detected: 16 CPUs > > FreeBSD/SMP: 2 package(s) x 8 core(s) x 2 hardware threads > > FreeBSD/SMP Online: 2 package(s) x 8 core(s) > > > > What I don't understand is the number of IRQs allocated for each mlx5_core: > > > > # vmstat -i | grep mlx5_core > > irq320: mlx5_core0                     1          0 > > irq321: mlx5_core0              18646775         84 > > irq322: mlx5_core0                    21          0 > > irq323: mlx5_core0                 97793          0 > > irq324: mlx5_core0                 84685          0 > > irq325: mlx5_core0                 89288          0 > > irq326: mlx5_core0                 93564          0 > > irq327: mlx5_core0                 86892          0 > > irq328: mlx5_core0                 99141          0 > > irq329: mlx5_core0                 86695          0 > > irq330: mlx5_core0                104023          0 > > irq331: mlx5_core0                 85238          0 > > irq332: mlx5_core0                 88387          0 > > irq333: mlx5_core0              93310221        420 > > ^^^ it appears you have some application which is using a single TCP > connection heavily. Then the traffic doesn't get distributed. Do you planed to use more describe irq's name?