Date: Thu, 11 Mar 2010 09:42:05 -0800 From: Jack Vogel <jfvogel@gmail.com> To: rihad <rihad@mail.ru> Cc: freebsd-net@freebsd.org Subject: Re: ixgbe input errors at high data rates Message-ID: <2a41acea1003110942u717e2222hd984bd2859c3e477@mail.gmail.com> In-Reply-To: <4B99114E.7060909@mail.ru> References: <4B99114E.7060909@mail.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
The 1.3.3 driver is two years old, and your OS is older. I would respectfully suggest that you update to 8.0 where lots of effort was put to make 10G hardware perform up to its capabilities. Similarly, I have done lots of work in two years to the ixgbe driver, I would even suggest that once you have 8 installed you get the driver from HEAD. Regards, Jack On Thu, Mar 11, 2010 at 7:50 AM, rihad <rihad@mail.ru> wrote: > Hi, our Intel 10 GigE cards are finally here, identified as > <Intel(R) PRO/10GbE PCI-Express Network Driver, Version - 1.3.3> > with the driver ixgbe-1.3.3 off the CD-ROM. > One card is used for input, the other for output, doing traffic limiting > (dummynet) and accounting in between. > At data rates of about 700-1000 mbps netstat -i shows many Input errors on > ix0 at a rate of 10-20K per second :( > > top -HS: > CPU: 1.3% user, 0.0% nice, 25.2% system, 14.1% interrupt, 59.3% idle > Mem: 1047M Active, 2058M Inact, 466M Wired, 126M Cache, 214M Buf, 239M Free > Swap: 2048M Total, 2048M Free > > PID USERNAME PRI NICE SIZE RES STATE C TIME WCPU COMMAND > 32 root -68 - 0K 16K CPU3 3 460:56 100.00% irq258: ix0 > 33 root -68 - 0K 16K CPU7 7 143:14 100.00% ix0 rxq > 13 root 171 ki31 0K 16K RUN 5 574:39 93.65% idle: cpu5 > 12 root 171 ki31 0K 16K RUN 6 507:08 88.33% idle: cpu6 > 14 root 171 ki31 0K 16K CPU4 4 424:04 80.37% idle: cpu4 > 18 root 171 ki31 0K 16K CPU0 0 395:34 75.00% idle: cpu0 > 16 root 171 ki31 0K 16K RUN 2 433:10 70.21% idle: cpu2 > 700 root -68 - 0K 16K - 2 292:19 56.64% dummynet > 17 root 171 ki31 0K 16K CPU1 1 399:02 50.39% idle: cpu1 > 37 root -68 - 0K 16K CPU1 1 196:19 39.50% ix1 rxq > 11 root 171 ki31 0K 16K RUN 7 510:39 14.79% idle: cpu7 > 36 root -68 - 0K 16K WAIT 5 36:36 8.64% irq260: ix1 > 19 root -32 - 0K 16K CPU6 6 36:52 5.08% swi4: clock > sio > > > Turning dummynet off (by short-circuiting the IPFW rule "allow ip from any > to any" before the "pipe tablearg") doesn't eliminate the input errors. > Turning ip.fastfowarding off (see below) doesn't help either (why would it), > only this time "swi" is chewing up the CPU time instead of "irq". Are we > hitting the CPU core limits here? It's a dual cpu quad-core Intel(R) Xeon(R) > CPU E5410 @ 2.33GHz (Dell PowerEdge 2950). > Shouldn't this $2.5K expensive card have decently-sized hardware buffers to > prevent any overruns? > > Some custom settings: > kern.hz=4000 > net.inet.ip.fastforwarding=1 > kern.ipc.nmbclusters=111111 > net.inet.ip.dummynet.io_fast=1 > net.isr.direct=0 > net.inet.ip.intr_queue_maxlen=5000 > hw.intr_storm_threshold=8000 #as suggested by the ixgbe-1.3.3 docs > > FreeBSD 7.1 kernel built with DEVICE_POLLING, even though it isn't used. > Should I nonetheless recompile without it? I heard the mere existence of > DEVICE_POLLING affects some cards' performance. > > Thanks for any tips. > _______________________________________________ > freebsd-net@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-net > To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org" >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2a41acea1003110942u717e2222hd984bd2859c3e477>