From owner-freebsd-net@FreeBSD.ORG Tue Jan 24 11:15:50 2012 Return-Path: Delivered-To: freebsd-net@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7717F106566C for ; Tue, 24 Jan 2012 11:15:50 +0000 (UTC) (envelope-from mmarkowski@leon.pl) Received: from kameleon.leon.pl (kameleon.leon.pl [IPv6:2a02:c40:0:10::5bc3:8704]) by mx1.freebsd.org (Postfix) with ESMTP id 0A0528FC08 for ; Tue, 24 Jan 2012 11:15:50 +0000 (UTC) Received: by kameleon.leon.pl (Postfix, from userid 1080) id 91070358C5C; Tue, 24 Jan 2012 12:15:47 +0100 (CET) To: Nikolay Denev X-PHP-Script: mail.leon.pl/index.php for 91.195.135.101 X-PHP-Originating-Script: 1080:main.inc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Date: Tue, 24 Jan 2012 12:15:47 +0100 From: Marcin Markowski In-Reply-To: <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> References: <159d4fbce722663a84f3cea12da828a5@leon.pl> <6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> Message-ID: <62333c6068c0df493b2e4e9ec86f34d2@leon.pl> X-Sender: mmarkowski@leon.pl User-Agent: Roundcube Webmail/0.6 Cc: freebsd-net@freebsd.org Subject: Re: Performance problem using Intel X520-DA2 X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Jan 2012 11:15:50 -0000 On 24.01.2012 09:18, Nikolay Denev wrote: > On Jan 23, 2012, at 11:39 PM, Marcin Markowski wrote: > >> Hello, >> >> This message has been sent to freebsd-performance@ but got >> the information that should contact also with freebsd-net@. >> >> We use FreeBSD as sniffer (libpcap programs) and we experience >> performance problems when incoming traffic is greater than >> 7.5Gbps/s. >> If we check 'top' we see that first irq from network card is using >> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE >> (on 9.0 we can see also kernel thread named {ix0 que} using 100% >> CPU), >> and both systems behave the same. In logs we see also: >> interrupt storm detected on "irq268:"; throttling interrupt source >> >> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM and >> NIC Intel X520-DA2. >> >> I'm not sure if problem is with NIC or motherboard in SR2600URBRP, >> because everything is fine when we use other server configuration: >> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2 >> >> My /boot/loader.conf: >> kern.ipc.nmbclusters=262144 >> hw.ixgbe.rxd=2048 >> hw.ixgbe.txd=2048 >> hw.ixgbe.num_queues=16 >> >> /etc/sysctl.conf >> hw.intr_storm_threshold=10000 >> >> -- >> Marcin Markowski > > > Hi, > > Maybe you want to take a loot at NETMAP : > http://info.iet.unipi.it/~luigi/netmap/ > There is a libpcap wrapper library, so you can use it with unchanged > pcap consumers, > and get great performance increase. > I'm not sure that the patches are updated for 8 and 9 though, since > the initial commit to HEAD > there were several related changes. > > P.S.: It is important also what is you packet rate, since 7.5Gbps > with jumbo packets or 64 byte packets > are very different things :) Hi Nikolay, I tried to compile the kernel with NETMAP on FreeBSD 8 and 9, but I get warnings and the compilation ends. cc1: warnings being treated as errors ../../../dev/netmap/netmap.c: In function 'netmap_memory_init': ../../../dev/netmap/netmap.c:1557: warning: format '%d' expects type 'int', but argument 7 has type 'size_t' ../../../dev/netmap/netmap.c:1564: warning: format '%d' expects type 'int', but argument 7 has type 'size_t' ../../../dev/netmap/netmap.c: In function 'netmap_memory_fini': ../../../dev/netmap/netmap.c:1607: warning: format '%d' expects type 'int', but argument 2 has type 'size_t' ../../../dev/netmap/netmap.c: In function 'netmap_init': ../../../dev/netmap/netmap.c:1636: warning: format '%d' expects type 'int', but argument 2 has type 'size_t' *** Error code 1 I'll try HEAD and see if it will be the same. -- Marcin Markowski