Date: Mon, 7 Sep 2020 23:09:36 +0530 From: Rajesh Kumar <rajfbsd@gmail.com> To: freebsd-net@freebsd.org Subject: 10G Network interface not reaching wire speed on Receive path with single thread iperf run Message-ID: <CAAO%2BANOENM1Bt7bL4wfEtdLbAU4Wo1RAW9DT6DeZA=YtEvNx3A@mail.gmail.com>
index | next in thread | raw e-mail
Hi FreeBSD team, I am writing a network driver using the iflib framework and using the "iperf" tool for performance testing. 1) 10G link with 9000 MTU: Seeing line-rate numbers on both Tx and Rx path. 2) 10G link with 1500 MTU. Seeing line-rate numbers on Tx path, but low numbers on Rx path. Base issue seems to be that Rx buffers are not getting re-filled fast enough. With low thread count, I see higher CPU utilization and lower throughput. But with a higher number of threads, I could see better throughput and lesser cpu utilization. iperf with 1 Thread: CPU Utilization is 80% - 95% - Throughput is ~4Gbps. iperf with 2 Thread: CPU Utilization is 80% - 90% - Throughput is ~7.5Gbps. iperf with 4 Thread: CPU Utilization is 40% - 75% - Throughput is ~9Gbps. iperf with 8 Thread: CPU Utilization is 30% - 70% - Throughput is ~9.2Gbps. So, something is causing CPU utilization higher with lesser thread, which kind of leads to Rx buffers not getting filled up faster and hence the performance numbers drop. From a code perspective, I couldn't see any evident CPU consuming operations in the Rx path. It's plain hardware descriptors reading/processing. Anyway we can debug to find what is causing more CPU utilization (with lesser threads) in my case? Any clues on debugging this issue? Thanks, Rajesh.home | help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAAO%2BANOENM1Bt7bL4wfEtdLbAU4Wo1RAW9DT6DeZA=YtEvNx3A>