Date: Fri, 28 Feb 2020 14:33:25 +0530 From: Rajesh Kumar <rajfbsd@gmail.com> To: freebsd-drivers@freebsd.org, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Network throughput not reaching line rate. Need clarification on iflib. Message-ID: <CAAO%2BANPvfRhLiCZX=0UWDToKWNACrZ66JgOd%2Bs_p9fAgc%2BrWTw@mail.gmail.com>
index | next in thread | raw e-mail
Hi FreeBSD team, I am writing a network driver using iflib framework and using "iperf3" tool for performance testing. 1) 10G link with 9000 MTU. Seeing line-rate numbers on both Tx and Rx path. 2) 10G link with 1500 MTU. Seeing line-rate numbers on Tx path, but low numbers on Rx path. I suspect the driver design here, but need some clarification on the iflib framework. My driver expects two memory buffers per receive descriptor. So I used "2 fls" per rxq (means isc_nrxq = 1, and isc_nfl = 2). For every descriptor, I use the memory buffers at same index from both the free-lists to refill my descriptors. Also, I return the fragments with appropriate index and flid from rxd_pkt_get. This approach works for me. But I am facing performance issue as stated above. So, Is this approach perfect? or any better way of doing? Also, I did a quick analysis of tcpdump on both iperf3 server (my test side) and client side(support side). iperf3 clients pumps traffic to iperf3 server. After a point, server sends an ACK with SACK option (SLE and SRE) to indicate missing frame to the client. And client is re-transmitting that segment as per protocol and the server receives it. But still I see the ACK sent again with SACK option. So client keeps on re-transmitting the packets and hence the performance drops. Any reason why server is not acknowledging the re-transmitted packet? If the driver Rx isn't proper, then I may not see the packet in tcpdump. But I see it. So any idea, what could be wrong here? Please let me know if any more details are needed. Thanks, Rajesh.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAO%2BANPvfRhLiCZX=0UWDToKWNACrZ66JgOd%2Bs_p9fAgc%2BrWTw>