From owner-freebsd-performance@FreeBSD.ORG Mon Jan 23 20:20:32 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B3C051065677 for ; Mon, 23 Jan 2012 20:20:32 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 49F6A8FC14 for ; Mon, 23 Jan 2012 20:20:32 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id D4503358C5B; Mon, 23 Jan 2012 21:20:29 +0100 (CET) To: X-PHP-Script: mail.leon.pl/index.php for 188.137.111.246 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Mon, 23 Jan 2012 21:20:29 +0100 From: Marcin Markowski Message-ID: X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Subject: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 20:20:32 -0000 Hello, We use FreeBSD as sniffer (libpcap programs) and we experience performance problems when incoming traffic is greater than 7.5Gbps/s. If we check 'top' we see that first irq from network card is using 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), and both systems behave the same. In logs we see also: interrupt storm detected on "irq268:"; throttling interrupt source Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and NIC Intel X520-DA2. I'm not sure if problem is with NIC or motherboard in SR2600URBRP, because everything is fine when we use other server configuration: Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 My /boot/loader.conf: kern.ipc.nmbclusters=262144 hw.ixgbe.rxd=2048 hw.ixgbe.txd=2048 hw.ixgbe.num_queues=16 /etc/sysctl.conf hw.intr_storm_threshold=10000 -- Marcin Markowski From owner-freebsd-performance@FreeBSD.ORG Mon Jan 23 20:50:29 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 995FA10656D5 for ; Mon, 23 Jan 2012 20:50:29 +0000 (UTC) (envelope-from jfvogel@gmail.com) Received: from mail-we0-f182.google.com (mail-we0-f182.google.com [74.125.82.182]) by mx1.freebsd.org (Postfix) with ESMTP id 3207A8FC14 for ; Mon, 23 Jan 2012 20:50:28 +0000 (UTC) Received: by werg1 with SMTP id g1so3884210wer.13 for ; Mon, 23 Jan 2012 12:50:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=xey8vCnk+GV2NYTsnW0ggSvnSZSYIVfbf7xoavMMXj0=; b=b7tGVyL0/7Rk6LkAIBBJu04xWS/KT4N8rDfIDF0mC5Wfdghr4gJ/j629cftPlHHaeM vd0DQkYeNlmGeUohnumN8nwI5Eu++iVsycv/q56gLi0wgjeGiujOAzpjHTGEWnE7OFp1 5KrYI1eJ1iaDdj6v+Dk1iHuF7i/1FQmI9sYbk= MIME-Version: 1.0 Received: by 10.216.136.168 with SMTP id w40mr4401963wei.37.1327350309606; Mon, 23 Jan 2012 12:25:09 -0800 (PST) Received: by 10.180.84.66 with HTTP; Mon, 23 Jan 2012 12:25:09 -0800 (PST) In-Reply-To: References: Date: Mon, 23 Jan 2012 12:25:09 -0800 Message-ID: From: Jack Vogel To: Marcin Markowski Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Cc: freebsd-performance@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 20:50:29 -0000 There may be some interrupt handling changes coming, but at least for the moment you need to increase the storm threshold, or set it to 0 to disable it. Jack On Mon, Jan 23, 2012 at 12:20 PM, Marcin Markowski wrote: > Hello, > > We use FreeBSD as sniffer (libpcap programs) and we experience > performance problems when incoming traffic is greater than 7.5Gbps/s. > If we check 'top' we see that first irq from network card is using > 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE > (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), > and both systems behave the same. In logs we see also: > interrupt storm detected on "irq268:"; throttling interrupt source > > Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and > NIC Intel X520-DA2. > > I'm not sure if problem is with NIC or motherboard in SR2600URBRP, > because everything is fine when we use other server configuration: > Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 > > My /boot/loader.conf: > kern.ipc.nmbclusters=262144 > hw.ixgbe.rxd=2048 > hw.ixgbe.txd=2048 > hw.ixgbe.num_queues=16 > > /etc/sysctl.conf > hw.intr_storm_threshold=10000 > > -- > Marcin Markowski > > ______________________________**_________________ > freebsd-performance@freebsd.**org mailing list > http://lists.freebsd.org/**mailman/listinfo/freebsd-**performance > To unsubscribe, send any mail to "freebsd-performance-** > unsubscribe@freebsd.org " > From owner-freebsd-performance@FreeBSD.ORG Mon Jan 23 21:23:42 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B5E0E1065673 for ; Mon, 23 Jan 2012 21:23:42 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 486388FC1E for ; Mon, 23 Jan 2012 21:23:42 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id BB928358C5B; Mon, 23 Jan 2012 22:23:40 +0100 (CET) To: Jack Vogel X-PHP-Script: mail.leon.pl/index.php for 188.137.111.246 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Date: Mon, 23 Jan 2012 22:23:40 +0100 From: Marcin Markowski In-Reply-To: References: Message-ID: <0bcfb39775599ec4be04a710040dab80@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Cc: freebsd-performance@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 23 Jan 2012 21:23:42 -0000 Hi Jack, Thank You for Your answer. I forgot to mention that if the program will run for more than 30-40 sec. server loses connection to the internet. I'm not sure if problem isnt with MSI-X, because NIC (82575EB - igb driver) used for the internet is also using MSI-X. Disabling MSI-X for ixgbe or switching on old NIC (82571EB - em driver) fixes issue (I'm unable to disable MSI-X on igb because the watchdog resets the card if using only MSI). -- Marcin Markowski On 23.01.2012 21:25, Jack Vogel wrote: > There may be some interrupt handling changes coming, but at least for > the moment you need to increase the storm threshold, or set it to 0 > to > disable it. > > Jack > > On Mon, Jan 23, 2012 at 12:20 PM, Marcin Markowski wrote: > >>  Hello, >> >> We use FreeBSD as sniffer (libpcap programs) and we experience >> performance problems when incoming traffic is greater than >> 7.5Gbps/s. >> If we check top we see that first irq from network card is using >> 100% CPU. Ive tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >> CPU), >> and both systems behave the same. In logs we see also: >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM >> and >> NIC Intel X520-DA2. >> >> Im not sure if problem is with NIC or motherboard in SR2600URBRP, >> because everything is fine when we use other server configuration: >> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >> >> My /boot/loader.conf: >> kern.ipc.nmbclusters=262144 >> hw.ixgbe.rxd=2048 >> hw.ixgbe.txd=2048 >> hw.ixgbe.num_queues=16 >> >> /etc/sysctl.conf >> hw.intr_storm_threshold=10000 >> >> -- >> Marcin Markowski From owner-freebsd-performance@FreeBSD.ORG Tue Jan 24 00:03:37 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0036C106566C for ; Tue, 24 Jan 2012 00:03:36 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-wi0-f182.google.com (mail-wi0-f182.google.com [209.85.212.182]) by mx1.freebsd.org (Postfix) with ESMTP id 843A78FC0A for ; Tue, 24 Jan 2012 00:03:36 +0000 (UTC) Received: by wibhn14 with SMTP id hn14so3973008wib.13 for ; Mon, 23 Jan 2012 16:03:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dudu.ro; s=google; h=date:from:to:cc:message-id:in-reply-to:references:subject:x-mailer :mime-version:x-gm-message-state:content-type :content-transfer-encoding:content-disposition; bh=+6ugUauuYsGghapMuEsSuzOwLQ2JXd3Gab5eeZlyGTU=; b=fhE7FzIvlwALCRBSKTwqrfmzYil486LMWRA+BN9Vo6BFyJcDvMVtigELP2IJYGob+e tIESCDKPANoD84fQOtkxEVQgqYNT0KZqffhQr3wBT10rqHX2czECl36z5R2J8ZzWoUxH Bm7Qc6ieek5eRmoPSPB+Rr8gHeHZ0sTcyNps8= Received: by 10.180.81.35 with SMTP id w3mr16745609wix.10.1327361665434; Mon, 23 Jan 2012 15:34:25 -0800 (PST) Received: from lespaul (5ad02814.bb.sky.com. [90.208.40.20]) by mx.google.com with ESMTPS id ex2sm46295339wib.1.2012.01.23.15.34.23 (version=TLSv1/SSLv3 cipher=OTHER); Mon, 23 Jan 2012 15:34:24 -0800 (PST) Date: Mon, 23 Jan 2012 23:34:21 +0000 From: Vlad Galu To: Marcin Markowski Message-ID: <52C581907BBB4B4AAF55D14F46BAAF2F@dudu.ro> In-Reply-To: References: X-Mailer: sparrow 1.5 (build 1043.1) MIME-Version: 1.0 X-Gm-Message-State: ALoCoQm6lPlnBjuAsfzqSCsaT3VbcAlDaJ1C1+7i7flW72lA8zsME7zJQ6PJk+mj4K97yl8X0ynP Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit Content-Disposition: inline Cc: freebsd-performance@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 00:03:37 -0000 -- Good, fast and cheap: pick any two. On Monday, January 23, 2012 at 8:20 PM, Marcin Markowski wrote: > Hello, > > We use FreeBSD as sniffer (libpcap programs) and we experience > performance problems when incoming traffic is greater than 7.5Gbps/s. > If we check 'top' we see that first irq from network card is using > 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE > (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), > and both systems behave the same. In logs we see also: > interrupt storm detected on "irq268:"; throttling interrupt source > > Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and > NIC Intel X520-DA2. > > I'm not sure if problem is with NIC or motherboard in SR2600URBRP, > because everything is fine when we use other server configuration: > Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 > > My /boot/loader.conf: > kern.ipc.nmbclusters=262144 > hw.ixgbe.rxd=2048 > hw.ixgbe.txd=2048 > hw.ixgbe.num_queues=16 > > /etc/sysctl.conf > hw.intr_storm_threshold=10000 > Hi Marcin, Have a look at Luigi's excellent netmap framework [1]. [1] http://info.iet.unipi.it/~luigi/netmap/ From owner-freebsd-performance@FreeBSD.ORG Tue Jan 24 11:28:01 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8EDA61065672 for ; Tue, 24 Jan 2012 11:28:01 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 4F3D28FC19 for ; Tue, 24 Jan 2012 11:28:01 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id CECCF358C5C; Tue, 24 Jan 2012 12:27:59 +0100 (CET) To: Vlad Galu X-PHP-Script: mail.leon.pl/index.php for 91.195.135.101 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 24 Jan 2012 12:27:59 +0100 From: Marcin Markowski In-Reply-To: <52C581907BBB4B4AAF55D14F46BAAF2F@dudu.ro> References: <52C581907BBB4B4AAF55D14F46BAAF2F@dudu.ro> Message-ID: X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Cc: freebsd-performance@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:28:01 -0000 On 24.01.2012 00:34, Vlad Galu wrote: > -- > Good, fast and cheap: pick any two. > > > On Monday, January 23, 2012 at 8:20 PM, Marcin Markowski wrote: > >> Hello, >> >> We use FreeBSD as sniffer (libpcap programs) and we experience >> performance problems when incoming traffic is greater than >> 7.5Gbps/s. >> If we check 'top' we see that first irq from network card is using >> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >> CPU), >> and both systems behave the same. In logs we see also: >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and >> NIC Intel X520-DA2. >> >> I'm not sure if problem is with NIC or motherboard in SR2600URBRP, >> because everything is fine when we use other server configuration: >> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >> >> My /boot/loader.conf: >> kern.ipc.nmbclusters=262144 >> hw.ixgbe.rxd=2048 >> hw.ixgbe.txd=2048 >> hw.ixgbe.num_queues=16 >> >> /etc/sysctl.conf >> hw.intr_storm_threshold=10000 >> > > Hi Marcin, > > Have a look at Luigi's excellent netmap framework [1]. > > [1] http://info.iet.unipi.it/~luigi/netmap/ Hi Vlad, I tried to compile the kernel with NETMAP on FreeBSD 8 and 9, but I get warnings and the compilation ends. cc1: warnings being treated as errors ../../../dev/netmap/netmap.c: In function 'netmap_memory_init': ../../../dev/netmap/netmap.c:1557: warning: format '%d' expects type 'int', but argument 7 has type 'size_t' ../../../dev/netmap/netmap.c:1564: warning: format '%d' expects type 'int', but argument 7 has type 'size_t' ../../../dev/netmap/netmap.c: In function 'netmap_memory_fini': ../../../dev/netmap/netmap.c:1607: warning: format '%d' expects type 'int', but argument 2 has type 'size_t' ../../../dev/netmap/netmap.c: In function 'netmap_init': ../../../dev/netmap/netmap.c:1636: warning: format '%d' expects type 'int', but argument 2 has type 'size_t' *** Error code 1 I'll try HEAD and see if it will be the same. -- Marcin Markowski From owner-freebsd-performance@FreeBSD.ORG Tue Jan 24 11:34:17 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 26BD81065674 for ; Tue, 24 Jan 2012 11:34:17 +0000 (UTC) (envelope-from dudu@dudu.ro) Received: from mail-iy0-f182.google.com (mail-iy0-f182.google.com [209.85.210.182]) by mx1.freebsd.org (Postfix) with ESMTP id E4D518FC18 for ; Tue, 24 Jan 2012 11:34:16 +0000 (UTC) Received: by iagz16 with SMTP id z16so9335802iag.13 for ; Tue, 24 Jan 2012 03:34:16 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dudu.ro; s=google; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:x-gm-message-state:content-type:content-transfer-encoding; bh=XPwzUW1ley8A6M/NJGNInWgLyfxVSsb0SdqwvE39tW8=; b=OJpZftSomft+cwhlBv0xNj8gZN21rpnwe5AB7XzplehWs78ExMrRhIg+eIlUIaGGD3 +sYpZZTkx5K43JiD100DBbA4eGkJohiBQFP89N0YCngUfGXTZoY4u7Sw88hjEMDzCb6R j0qKifbtlFmcCkscDqvxINdedi7QmCfy7TpRE= Received: by 10.50.216.201 with SMTP id os9mr3177744igc.22.1327404856209; Tue, 24 Jan 2012 03:34:16 -0800 (PST) MIME-Version: 1.0 Received: by 10.42.226.68 with HTTP; Tue, 24 Jan 2012 03:33:35 -0800 (PST) In-Reply-To: References: <52C581907BBB4B4AAF55D14F46BAAF2F@dudu.ro> From: Vlad Galu Date: Tue, 24 Jan 2012 11:33:35 +0000 Message-ID: To: Marcin Markowski X-Gm-Message-State: ALoCoQnns6PdIyWouoJYT2VTH8gsbvUa3Jh08AjGFxwX63279Xwd0aUuOkKAoGaOVxp9zaqhbVrY Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-performance@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:34:17 -0000 On Tue, Jan 24, 2012 at 11:27 AM, Marcin Markowski wro= te: > On 24.01.2012 00:34, Vlad Galu wrote: >> >> -- >> Good, fast and cheap: pick any two. >> >> >> On Monday, January 23, 2012 at 8:20 PM, Marcin Markowski wrote: >> >>> Hello, >>> >>> We use FreeBSD as sniffer (libpcap programs) and we experience >>> performance problems when incoming traffic is greater than 7.5Gbps/s. >>> If we check 'top' we see that first irq from network card is using >>> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >>> (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), >>> and both systems behave the same. In logs we see also: >>> interrupt storm detected on "irq268:"; throttling interrupt source >>> >>> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and >>> NIC Intel X520-DA2. >>> >>> I'm not sure if problem is with NIC or motherboard in SR2600URBRP, >>> because everything is fine when we use other server configuration: >>> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >>> >>> My /boot/loader.conf: >>> kern.ipc.nmbclusters=3D262144 >>> hw.ixgbe.rxd=3D2048 >>> hw.ixgbe.txd=3D2048 >>> hw.ixgbe.num_queues=3D16 >>> >>> /etc/sysctl.conf >>> hw.intr_storm_threshold=3D10000 >>> >> >> Hi Marcin, >> >> Have a look at Luigi's excellent netmap framework [1]. >> >> [1] http://info.iet.unipi.it/~luigi/netmap/ > > > =A0Hi Vlad, > > > I tried to compile the kernel with NETMAP on FreeBSD 8 and 9, but I get > warnings and > the compilation ends. > > cc1: warnings being treated as errors > ../../../dev/netmap/netmap.c: In function 'netmap_memory_init': > ../../../dev/netmap/netmap.c:1557: warning: format '%d' expects type 'int= ', > but argument 7 has type 'size_t' > ../../../dev/netmap/netmap.c:1564: warning: format '%d' expects type 'int= ', > but argument 7 has type 'size_t' > ../../../dev/netmap/netmap.c: In function 'netmap_memory_fini': > ../../../dev/netmap/netmap.c:1607: warning: format '%d' expects type 'int= ', > but argument 2 has type 'size_t' > ../../../dev/netmap/netmap.c: In function 'netmap_init': > ../../../dev/netmap/netmap.c:1636: warning: format '%d' expects type 'int= ', > but argument 2 has type 'size_t' > *** Error code 1 > > I'll try HEAD and see if it will be the same. > Hi Marcin, Yes, I manually patched the sources. The format string should be %zd. --=20 Good, fast & cheap. Pick any two. From owner-freebsd-performance@FreeBSD.ORG Tue Jan 24 13:37:47 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0C27C106564A for ; Tue, 24 Jan 2012 13:37:47 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id BC05F8FC15 for ; Tue, 24 Jan 2012 13:37:46 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1RpgKi-00064E-As for freebsd-performance@freebsd.org; Tue, 24 Jan 2012 14:22:44 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Jan 2012 14:22:44 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Tue, 24 Jan 2012 14:22:44 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Tue, 24 Jan 2012 14:22:30 +0100 Lines: 10 Message-ID: References: Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120110 Thunderbird/9.0 In-Reply-To: Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 13:37:47 -0000 > On Mon, Jan 23, 2012 at 12:20 PM, Marcin Markowskiwrote: >> (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), >> hw.ixgbe.num_queues=16 If there really are 16 hardware queues, shouldn't there be 16 kernel threads for queue processing? From owner-freebsd-performance@FreeBSD.ORG Tue Jan 24 16:53:25 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D8DA8106564A for ; Tue, 24 Jan 2012 16:53:25 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 6BEB68FC0A for ; Tue, 24 Jan 2012 16:53:25 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id C6D23358C5C; Tue, 24 Jan 2012 17:53:23 +0100 (CET) To: X-PHP-Script: mail.leon.pl/index.php for 188.137.111.246 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 24 Jan 2012 17:53:23 +0100 From: Marcin Markowski In-Reply-To: References: Message-ID: <909994c3cdd84cb2c47ff8037c23e142@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 16:53:25 -0000 On 24.01.2012 14:22, Ivan Voras wrote: >> On Mon, Jan 23, 2012 at 12:20 PM, Marcin >> Markowskiwrote: > >>> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >>> CPU), > >>> hw.ixgbe.num_queues=16 > > If there really are 16 hardware queues, shouldn't there be 16 kernel > threads for queue processing? There are 16 threads, but only one of them consumes 100% CPU and the others do not use more than 5% CPU: http://pastebin.com/BWDWh8kW The same test but on the Intel SR1630GP, 1 CPU Xeon X3450 and FreeBSD 8.2: http://pastebin.com/vLib1kqE -- Marcin Markowski From owner-freebsd-performance@FreeBSD.ORG Wed Jan 25 11:59:53 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3FFC2106566C for ; Wed, 25 Jan 2012 11:59:53 +0000 (UTC) (envelope-from gofp-freebsd-performance@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id EB6758FC1A for ; Wed, 25 Jan 2012 11:59:52 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1Rq1W1-0004FG-2B for freebsd-performance@freebsd.org; Wed, 25 Jan 2012 12:59:49 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 25 Jan 2012 12:59:48 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 25 Jan 2012 12:59:48 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-performance@freebsd.org From: Ivan Voras Date: Wed, 25 Jan 2012 12:59:36 +0100 Lines: 21 Message-ID: References: <909994c3cdd84cb2c47ff8037c23e142@leon.pl> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:9.0) Gecko/20120110 Thunderbird/9.0 In-Reply-To: <909994c3cdd84cb2c47ff8037c23e142@leon.pl> Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 11:59:53 -0000 On 24/01/2012 17:53, Marcin Markowski wrote: > On 24.01.2012 14:22, Ivan Voras wrote: >>> On Mon, Jan 23, 2012 at 12:20 PM, Marcin >>> Markowskiwrote: >> >>>> (on 9.0 we can see also kernel thread named {ix0 que} using 100% CPU), >> >>>> hw.ixgbe.num_queues=16 >> >> If there really are 16 hardware queues, shouldn't there be 16 kernel >> threads for queue processing? > > There are 16 threads, but only one of them consumes 100% CPU and the others > do not use more than 5% CPU: > > http://pastebin.com/BWDWh8kW You need Jack to confirm it but this looks like a serious problem / bottleneck. It just shouldn't be like that (if the test is exactly the same). From owner-freebsd-performance@FreeBSD.ORG Wed Jan 25 13:40:35 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 201271065673; Wed, 25 Jan 2012 13:40:35 +0000 (UTC) (envelope-from ndenev@gmail.com) Received: from mail-ee0-f54.google.com (mail-ee0-f54.google.com [74.125.83.54]) by mx1.freebsd.org (Postfix) with ESMTP id 7902D8FC17; Wed, 25 Jan 2012 13:40:34 +0000 (UTC) Received: by eekb47 with SMTP id b47so2174602eek.13 for ; Wed, 25 Jan 2012 05:40:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=subject:mime-version:content-type:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to:x-mailer; bh=xX3cYHcAEgRqaJ8TBlWBzmux7i+LUz/yEoaH++N4/g0=; b=t9NuF0XAjAt1BazMb/0XNvqTLgCqLEVrUFC7c27Yiw8nuBIn7DowEPAk5ujka042do +gyiB9Ro9zhNHASOOkSDd5s5YMykHnetp6hWKFvYVF9f/9A2I71eVRcce5NzE4jEcEWT qPESJ6op5e8zKnYYymcPXyKiDL0wwvmsCTrWM= Received: by 10.14.10.35 with SMTP id 35mr6100549eeu.121.1327497514345; Wed, 25 Jan 2012 05:18:34 -0800 (PST) Received: from ndenevsa.sf.moneybookers.net (g1.moneybookers.com. [217.18.249.148]) by mx.google.com with ESMTPS id w46sm1765534eeb.0.2012.01.25.05.18.32 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 25 Jan 2012 05:18:32 -0800 (PST) Mime-Version: 1.0 (Apple Message framework v1251.1) Content-Type: text/plain; charset=us-ascii From: Nikolay Denev In-Reply-To: Date: Wed, 25 Jan 2012 15:18:31 +0200 Content-Transfer-Encoding: quoted-printable Message-Id: References: <909994c3cdd84cb2c47ff8037c23e142@leon.pl> To: Ivan Voras X-Mailer: Apple Mail (2.1251.1) X-Mailman-Approved-At: Wed, 25 Jan 2012 14:46:12 +0000 Cc: freebsd-performance@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 13:40:35 -0000 On Jan 25, 2012, at 1:59 PM, Ivan Voras wrote: > On 24/01/2012 17:53, Marcin Markowski wrote: >> On 24.01.2012 14:22, Ivan Voras wrote: >>>> On Mon, Jan 23, 2012 at 12:20 PM, Marcin >>>> Markowskiwrote: >>>=20 >>>>> (on 9.0 we can see also kernel thread named {ix0 que} using 100% = CPU), >>>=20 >>>>> hw.ixgbe.num_queues=3D16 >>>=20 >>> If there really are 16 hardware queues, shouldn't there be 16 kernel >>> threads for queue processing? >>=20 >> There are 16 threads, but only one of them consumes 100% CPU and the = others >> do not use more than 5% CPU: >>=20 >> http://pastebin.com/BWDWh8kW >=20 > You need Jack to confirm it but this looks like a serious problem / = bottleneck. It just shouldn't be like that (if the test is exactly the = same). >=20 Is this by any chance non-IP traffic (or encapsulated in some way). I remember similar thread : = http://freebsd.1045724.n5.nabble.com/igb-ixgbe-RSS-RX-queues-for-non-IP-tr= affic-td4778961.html From owner-freebsd-performance@FreeBSD.ORG Wed Jan 25 20:50:51 2012 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D33701065675 for ; Wed, 25 Jan 2012 20:50:51 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 63C208FC0A for ; Wed, 25 Jan 2012 20:50:51 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id A9557358C5D; Wed, 25 Jan 2012 21:50:48 +0100 (CET) To: X-PHP-Script: mail.leon.pl/index.php for 188.137.111.246 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Wed, 25 Jan 2012 21:50:48 +0100 From: Marcin Markowski In-Reply-To: References: <909994c3cdd84cb2c47ff8037c23e142@leon.pl> Message-ID: <94bb160090e45c85083176023a278f85@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 25 Jan 2012 20:50:51 -0000 On 25.01.2012 14:18, Nikolay Denev wrote: > On Jan 25, 2012, at 1:59 PM, Ivan Voras wrote: > >> On 24/01/2012 17:53, Marcin Markowski wrote: >>> On 24.01.2012 14:22, Ivan Voras wrote: >>>>> On Mon, Jan 23, 2012 at 12:20 PM, Marcin >>>>> Markowskiwrote: >>>> >>>>>> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >>>>>> CPU), >>>> >>>>>> hw.ixgbe.num_queues=16 >>>> >>>> If there really are 16 hardware queues, shouldn't there be 16 >>>> kernel >>>> threads for queue processing? >>> >>> There are 16 threads, but only one of them consumes 100% CPU and >>> the others >>> do not use more than 5% CPU: >>> >>> http://pastebin.com/BWDWh8kW >> >> You need Jack to confirm it but this looks like a serious problem / >> bottleneck. It just shouldn't be like that (if the test is exactly the >> same). >> > > Is this by any chance non-IP traffic (or encapsulated in some way). > I remember similar thread : > > http://freebsd.1045724.n5.nabble.com/igb-ixgbe-RSS-RX-queues-for-non-IP-traffic-td4778961.html Most of the traffic received by the interface is a TCP/UDP, but there is also a PPPoE (~2.5Gbps/s). The whole is divided into a large number of VLANs (>500) -- Marcin Markowski