Date: Fri, 28 Feb 2020 14:33:25 +0530 From: Rajesh Kumar <rajfbsd@gmail.com> To: freebsd-drivers@freebsd.org, FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Network throughput not reaching line rate. Need clarification on iflib. Message-ID: <CAAO%2BANPvfRhLiCZX=0UWDToKWNACrZ66JgOd%2Bs_p9fAgc%2BrWTw@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hi FreeBSD team, I am writing a network driver using iflib framework and using "iperf3" tool for performance testing. 1) 10G link with 9000 MTU. Seeing line-rate numbers on both Tx and Rx path. 2) 10G link with 1500 MTU. Seeing line-rate numbers on Tx path, but low numbers on Rx path. I suspect the driver design here, but need some clarification on the iflib framework. My driver expects two memory buffers per receive descriptor. So I used "2 fls" per rxq (means isc_nrxq = 1, and isc_nfl = 2). For every descriptor, I use the memory buffers at same index from both the free-lists to refill my descriptors. Also, I return the fragments with appropriate index and flid from rxd_pkt_get. This approach works for me. But I am facing performance issue as stated above. So, Is this approach perfect? or any better way of doing? Also, I did a quick analysis of tcpdump on both iperf3 server (my test side) and client side(support side). iperf3 clients pumps traffic to iperf3 server. After a point, server sends an ACK with SACK option (SLE and SRE) to indicate missing frame to the client. And client is re-transmitting that segment as per protocol and the server receives it. But still I see the ACK sent again with SACK option. So client keeps on re-transmitting the packets and hence the performance drops. Any reason why server is not acknowledging the re-transmitted packet? If the driver Rx isn't proper, then I may not see the packet in tcpdump. But I see it. So any idea, what could be wrong here? Please let me know if any more details are needed. Thanks, Rajesh.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAO%2BANPvfRhLiCZX=0UWDToKWNACrZ66JgOd%2Bs_p9fAgc%2BrWTw>