Date: Sat, 23 Jan 2016 09:35:35 -0800 From: Luigi Rizzo <rizzo@iet.unipi.it> To: Marcus Cenzatti <cenzatti@hush.com> Cc: Navdeep Parhar <nparhar@gmail.com>, "freebsd-net@freebsd.org" <freebsd-net@freebsd.org> Subject: Re: Chelsio T520-SO-CR low performance (netmap tested) for RX Message-ID: <CA%2BhQ2%2Bg4kU4LA4PexRPBv7z49ZWh-mDqdpw18SeoYaBueHyjZg@mail.gmail.com> In-Reply-To: <20160123171300.0F448A0121@smtp.hushmail.com> References: <20160123053428.2091EA0121@smtp.hushmail.com> <20160123154052.GA4574@ox> <20160123171300.0F448A0121@smtp.hushmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Jan 23, 2016 at 9:12 AM, Marcus Cenzatti <cenzatti@hush.com> wrote: > > > On 1/23/2016 at 1:40 PM, "Navdeep Parhar" <nparhar@gmail.com> wrote: >> >>On Sat, Jan 23, 2016 at 03:34:27AM -0200, Marcus Cenzatti wrote: >>> hello, >>> >>> I am testing a chelsio t520-so-cr connected to a Intel card with >>ix(4) >>> driver, I can get the ncxl0 interface to transmit at 14Mpps to >>another >>> chelsio or to a Intel card. However I can only get 800Kpps-1Mpps >>for >>> RX tests from both chelsio or Intel. >>> >>> I have test with both FreeBSD 11 and FreeBSD 10.3-PRERELEASE. >>> >>> I tested it untuned first and later I have applied tuning >>> recommendations I found on BSDRP[1] website. Results still >>ranging >>> from 800Kpps to 1Mpps for RX. >>> >>> Tests are done w/ with pkt-gen in netmap mode on ncxl interface >>with >>> both IP address and MAC address source/dest. >> >>The ncxl interfaces have their own MAC addresses. Make sure the >>sender >>uses the MAC of the receiver's ncxl interface as the destination >>MAC. >>(netmap's pkt-gen -f tx transmits L2 broadcasts by default). >> >>Check for PAUSE frames coming out of the receiver (sysctl dev.cxl >>| grep >>tx_pause). If it's receiving frames on netmap interface the >>tx_pause >>counter should not move. >> >>Regards, >>Navdeep >> > > hello, > > yes, MAC addresses are correct, I did the tests again and tx_pause won't move, here is the full transcript for the tests: > > ===> BOX #1 CHELSIO > > chelsio# ifconfig -v ncxl0 > ncxl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > ether 00:07:43:33:8d:c1 > inet 10.1.1.2 netmask 0xffffff00 broadcast 10.1.1.255 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet 10Gbase-SR <full-duplex> > status: active > > chelsio# ifconfig -v cxl0 > cxl0: flags=8843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST> metric 0 mtu 1500 > options=ec00bb<RXCSUM,TXCSUM,VLAN_MTU,VLAN_HWTAGGING,JUMBO_MTU,VLAN_HWCSUM,VLAN_HWTSO,LINKSTATE,RXCSUM_IPV6,TXCSUM_IPV6> > ether 00:07:43:33:8d:c0 > nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL> > media: Ethernet 10Gbase-SR <full-duplex> > status: active > plugged: SFP/SFP+/SFP28 10G Base-SR (LC) > vendor: FINISAR CORP. PN: FTLX8571D3BCL-FC SN: AL1073K DATE: 2011-06-28 > module temperature: 42.79 C Voltage: 3.23 Volts > RX: 0.53 mW (-2.74 dBm) TX: 0.48 mW (-3.12 dBm) > > chelsio# ./pkt-gen -i ncxl0 -f rx -d 00:07:43:33:8d:c1 -s 00:07:e9:44:d2:ba > 311.132189 main [1715] interface is ncxl0 > 311.132447 extract_ip_range [291] range is 0.0.0.0:90 to 0.0.0.0:90 > 311.132472 extract_ip_range [291] range is 0.0.0.0:7 to 0.0.0.0:7 wait, the lower case -s and -d are for IP addresses, you need to use -S and -D for the MAC addresses. This way you are sending broadcasts, which likely means that the chelsio is replicating the packets to both the netmap and the regular port and the latter (which perhaps comes first) is likely dropping packets. cheers luigi
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CA%2BhQ2%2Bg4kU4LA4PexRPBv7z49ZWh-mDqdpw18SeoYaBueHyjZg>