Date: Fri, 20 Apr 2018 00:03:37 -0400 From: Garrett Wollman <wollman@bimajority.org> To: freebsd-net@freebsd.org Subject: Diagnosing terrible ixl performance Message-ID: <23257.26265.720293.659892@hergotha.csail.mit.edu>
next in thread | raw e-mail | index | archive | help
I'm commissioning a new NFS server with an Intel dual-40G XL710 interface, running 11.1. I have a few other servers with this adapter, although not running 40G, and they work fine so long as you disable TSO. This one ... not so much. On the receive side, it gets about 600 Mbit/s with lots of retransmits. On the *sending* side, though, it's not even able to sustain 10 Mbit/s -- but there's no evidence of retransmissions, it's just sending really really slowly. (Other machines with XL710 adapters are able to sustain full 10G.) There is no evidence of any errors on either the adapter or the switch it's connected to. So far, I've tried: - Using the latest Intel driver (no change) - Using the latest Intel firmware (breaks the adapter) - Disabling performance tweaks in loader.conf and sysctl.conf - Changing congestion-control algorithms Anyone have suggestions while I still have time to test this? (My plan B is to fall back to an X520 card that I have in my spares kit, because I *know* those work great with no faffing about.) Any relevant MIBs to inspect? The test I'm doing here is simple iperf over TCP, with MTU 9120. It takes about 10 seconds for the sending side to complete, but buffers are severely constipated for 20 seconds after that (delaying all traffic, including ssh connections). I'm at the point of trying different switch ports just to eliminate that as a possibility. -GAWollman
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?23257.26265.720293.659892>