Date: Tue, 18 Aug 2015 15:22:45 +0200 From: Hans Petter Selasky <hps@selasky.org> To: Rick Macklem <rmacklem@uoguelph.ca>, Daniel Braniss <danny@cs.huji.ac.il> Cc: FreeBSD Net <freebsd-net@freebsd.org>, Slawa Olhovchenkov <slw@zxy.spb.ru>, FreeBSD stable <freebsd-stable@freebsd.org>, Christopher Forgeron <csforgeron@gmail.com> Subject: Re: ix(intel) vs mlxen(mellanox) 10Gb performance Message-ID: <55D331A5.9050601@selasky.org> In-Reply-To: <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca> References: <1D52028A-B39F-4F9B-BD38-CB1D73BF5D56@cs.huji.ac.il> <20150817094145.GB3158@zxy.spb.ru> <CAB2_NwBOLcL4EVjFN6=BvBC_YN=gmfZMweVbmb5ZPCsK4Hnx1g@mail.gmail.com> <17871443-E105-4434-80B1-6939306A865F@cs.huji.ac.il> <473274181.23263108.1439814072514.JavaMail.zimbra@uoguelph.ca> <7F892C70-9C04-4468-9514-EDBFE75CF2C6@cs.huji.ac.il> <805850043.24018217.1439848150695.JavaMail.zimbra@uoguelph.ca> <9D8B0503-E8FA-43CA-88F0-01F184F84D9B@cs.huji.ac.il> <1721122651.24481798.1439902381663.JavaMail.zimbra@uoguelph.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On 08/18/15 14:53, Rick Macklem wrote:
> If this is just a test machine, maybe you could test with these lines (at about #880)
> in sys/netinet/tcp_output.c commented out? (It looks to me like this will disable TSO
> for almost all the NFS writes.)
> - around line #880 in sys/netinet/tcp_output.c:
> /*
> * In case there are too many small fragments
> * don't use TSO:
> */
> if (len <= max_len) {
> len = max_len;
> sendalot = 1;
> tso = 0;
> }
>
> This was added along with the other stuff that did the if_hw_tsomaxsegcount, etc and I
> never noticed it until now (not my patch).
FYI:
These lines are needed by other hardware, like the mlxen driver. If you
remove them mlxen will start doing m_defrag(). I believe if you set the
correct parameters in the "struct ifnet" for the TSO size/count limits
this problem will go away. If you print the "len" and "max_len" and also
the cases where TSO limits are reached, you'll see what parameter is
triggering it and needs to be increased.
--HPS
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?55D331A5.9050601>