Date: Tue, 24 Jan 2012 18:17:13 +0100 From: Marcin Markowski <mmarkowski@leon.pl> To: <freebsd-net@freebsd.org> Subject: RE: Performance problem using Intel X520-DA2 Message-ID: <e4a398438131e527aa1fdc6f9d403cc1@leon.pl> In-Reply-To: <529374128DC1B04D9D037911B8E8F0531095BC48@Exchange26.EDU.epsb.ca> References: <159d4fbce722663a84f3cea12da828a5@leon.pl><6A254A40-7DA5-4EFE-93C5-4E084F33B78A@gmail.com> <86d5416f7f55a71fb01fd86e9051d678@leon.pl> <529374128DC1B04D9D037911B8E8F0531095BC48@Exchange26.EDU.epsb.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On 24.01.2012 16:07, Kirk Davis wrote:
>>-----Original Message-----
>>On 24.01.2012 09:18, Nikolay Denev wrote:
>>> On Jan 23, 2012, at 11:39 PM, Marcin Markowski wrote:
>>>
>>>> Hello,
>>>>
>>>> This message has been sent to freebsd-performance@ but got the
>>>> information that should contact also with freebsd-net@.
>>>>
>>>> We use FreeBSD as sniffer (libpcap programs) and we experience
>>>> performance problems when incoming traffic is greater than
>>>> 7.5Gbps/s.
>>>> If we check 'top' we see that first irq from network card is using
>>>> 100% CPU. I've tested this on FreeBSD 8.2-RELEASE and 9.0-RELEASE
>>>> (on
>>>> 9.0 we can see also kernel thread named {ix0 que} using 100% CPU),
>>>> and both systems behave the same. In logs we see also:
>>>> interrupt storm detected on "irq268:"; throttling interrupt source
>>>>
>>>> Our server platform is Intel SR2600URBRP, 2x Xeon X5650, 6GB RAM
>>>> and
>>>> NIC Intel X520-DA2.
>>>>
>>>> I'm not sure if problem is with NIC or motherboard in SR2600URBRP,
>>>> because everything is fine when we use other server configuration:
>>>> Intel SR1630GP, 1x Xeon X3450, 8GB RAM, NIC X520-DA2
>>>>
>>>> My /boot/loader.conf:
>>>> kern.ipc.nmbclusters=262144
>>>> hw.ixgbe.rxd=2048
>>>> hw.ixgbe.txd=2048
>>>> hw.ixgbe.num_queues=16
>>>>
>>>> /etc/sysctl.conf
>>>> hw.intr_storm_threshold=10000
>>>>
>
> I just finished a bunch of performance tests on this card. In my
> case I was trying to get as close to a full 10Gb/s as possible on a
> Dell R710 but I haven't yet tried any sniffing.
>
> Have you tried turning on LRO on the interface ( ifconfig lro ). In
> my case this made a big difference and I can now get 9.41Gb/s without
> high CPU or interrupt storms. I am also using Jack's latest driver
> downloaded from intel (version 2.4.4). Even the driver in 9.0 was
> older.
>
> Here is what I have
> /etc/sysctl.conf
> # Increase the network buffers
> kern.ipc.nmbclusters=262144
> kern.ipc.maxsockbuf=4194304
> hw.intr_storm_threshold=9000
> kern.ipc.nmbjumbop=262144
>
> /boot/loader.conf
> ixgbe_load="YES"
> hw.ixgbe.txd=4096
> hw.ixgbe.rxd=4096
>
> ---- Kirk
Hi Kirk,
I did not notice any change after turning on LRO. When checking stats
I see that LRO is not used (perhaps because the interface is receiving
traffic from port mirror on the switch).
sysctl output from dev.ix.0:
http://pastebin.com/fkRp7Py5
--
Marcin Markowski
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e4a398438131e527aa1fdc6f9d403cc1>