Date: Thu, 23 May 2013 11:00:16 -0700 From: Lino Sanfilippo <lsanfil@marvell.com> To: Hackers freeBSD <freebsd-hackers@freebsd.org> Cc: Axel Fischer <afischer@marvell.com>, Ralf Assmann <rassmann@marvell.com>, Markus Althoff <malthoff@marvell.com> Subject: Low Tx-Rx performance with 10Gb NICs Message-ID: <175CCF5F49938B4D99B2E3EF7F558EBE381FA6E5AA@SC-VEXCH4.marvell.com>
next in thread | raw e-mail | index | archive | help
Hi all, I am currently doing some performance tests with 10Gb NICs and encounter a = strange behavior in case when I do Rx and Tx at the same time: while the Rx rate stays more = or less stable (almost the same that I see with only Rx traffic) the Tx rate breaks down = drastically. The tests are done with netperf (4 TCP streams for Rx and Tx respectively),= the test machine is an Intel i7 (with HT 8 cores at 3,4 GHz) with 16GB RAM running 32 bit FreeB= SD 9.0 with default system settings. The results are like the following: TX Only: 2290.32 Mb/s Port=3D1001 TX 2357.73 Mb/s Port=3D1002 TX 2340.08 Mb/s Port=3D1003 TX 2382.87 Mb/s Port=3D1004 TX TX Total Result: Mb/s 9371 RX Only: 1257.43 Mb/s Port=3D1001 RX 1901.75 Mb/s Port=3D1002 RX 2605.19 Mb/s Port=3D1003 RX 1986.69 Mb/s Port=3D1004 RX RX Total Result: Mb/s 7751.06 Rx+TX: 251.11 Mb/s Port=3D1001 TX 3069.74 Mb/s Port=3D2001 RX 178.35 Mb/s Port=3D1002 TX 1118.30 Mb/s Port=3D2002 RX 138.05 Mb/s Port=3D1003 TX 1661.22 Mb/s Port=3D2003 RX 129.23 Mb/s Port=3D1004 TX 1851.75 Mb/s Port=3D2004 RX RX+TX Total Result: Mb/s 8397.75 While the results for Tx and Rx are similar to what I get on Linux, the results for Tx+Rx are not. On Linux I see total rates of about 15-18 Gb= /s. I expected to see the same on FreeBSD and I wonder why I do not. The result= s are quite the same for Intel, Broadcom and Marvell NICs, so I suspect the c= ause to be in the protocol stack rather than in the drivers. Is there a known issue concerning high traffic on Tx and Rx paths? Are the= re any system settings I could adjust to get the expected performance? Any hints are very= appreciated. Regards, Lino
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?175CCF5F49938B4D99B2E3EF7F558EBE381FA6E5AA>