Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 28 Feb 2020 14:33:25 +0530
From:      Rajesh Kumar <rajfbsd@gmail.com>
To:        freebsd-drivers@freebsd.org, FreeBSD Hackers <freebsd-hackers@freebsd.org>
Subject:   Network throughput not reaching line rate. Need clarification on iflib.
Message-ID:  <CAAO%2BANPvfRhLiCZX=0UWDToKWNACrZ66JgOd%2Bs_p9fAgc%2BrWTw@mail.gmail.com>

next in thread | raw e-mail | index | archive | help
Hi FreeBSD team,

I am writing a network driver using iflib framework and using "iperf3" tool
for performance testing.

1) 10G link with 9000 MTU. Seeing line-rate numbers on both Tx and Rx path.
2) 10G link with 1500 MTU. Seeing line-rate numbers on Tx path, but low
numbers on Rx path.

I suspect the driver design here, but need some clarification on the iflib
framework.

My driver expects two memory buffers per receive descriptor. So I used "2
fls" per rxq (means isc_nrxq = 1, and isc_nfl = 2). For every descriptor, I
use the memory buffers at same index from both the free-lists to refill my
descriptors.  Also, I return the fragments with appropriate index and flid
from rxd_pkt_get. This approach works for me. But I am facing performance
issue as stated above. So, Is this approach perfect? or any better way of
doing?

Also, I did a quick analysis of tcpdump on both iperf3 server (my test
side) and client side(support side).  iperf3 clients pumps traffic to
iperf3 server. After a point, server sends an ACK with SACK option (SLE and
SRE) to indicate missing frame to the client. And client is re-transmitting
that segment as per protocol and the server receives it. But still I see
the ACK sent again with SACK option. So client keeps on re-transmitting the
packets and hence the performance drops.

Any reason why server is not acknowledging the re-transmitted packet? If
the driver Rx isn't proper, then I may not see the packet in tcpdump. But I
see it. So any idea, what could be wrong here?

Please let me know if any more details are needed.

Thanks,
Rajesh.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAO%2BANPvfRhLiCZX=0UWDToKWNACrZ66JgOd%2Bs_p9fAgc%2BrWTw>