Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 01 Nov 2012 15:39:30 +0100
From:      Andre Oppermann <andre@freebsd.org>
To:        Jack F Vogel <jfv@FreeBSD.org>
Cc:        svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org
Subject:   Re: svn commit: r242421 - head/sys/dev/ixgbe
Message-ID:  <509289A2.7080304@freebsd.org>
In-Reply-To: <201210312350.q9VNoaAa065183@svn.freebsd.org>
References:  <201210312350.q9VNoaAa065183@svn.freebsd.org>

next in thread | previous in thread | raw e-mail | index | archive | help
On 01.11.2012 00:50, Jack F Vogel wrote:
> Author: jfv
> Date: Wed Oct 31 23:50:36 2012
> New Revision: 242421
> URL: http://svn.freebsd.org/changeset/base/242421
>
> Log:
>    A few important fixes:
>      - Testing TSO6 has led me to discover that HW RSC is
>        a problematic feature, it is ONLY designed to work
>        with IPv4 in the first place, and if IP forwarding
>        is done it can't be disabled as LRO in the stack,
>        also initial testing we've done at Intel shows an
>        equal performance using TSO[46] on the TX and LRO
>        on RX, if you ran older code on 82599 or later hardware
>        you actually could have detrimental performance for
>        this reason. So I am disabling the feature by default
>        and all our adapters will now use LRO instead.

Yes, it's very important that LRO is *not* used when forwarding
is enabled (= acting as a router).

>      - If you have flow control off and multiple queues it
>        was possible when the buffer of one queue becomes
>        full that all RX movement is stalled, to eliminate
>        this problem a feature bit is now set that will allow
>        packets to be dropped when full rather than stall.
>        Note, the default is to have flow control on, and this
>        keeps this from happening.
>
>      - Because of the recent fixes in the stack, LRO is now
>        auto-disabled when problematic, so I have decided to
>        enable it by default in the capabilities in the driver.

A very important cautionary note here: LRO is only good when combined
with very low RTTs (that is in LAN environments).  On everything over
5ms is breaks the TCP ACK clock badly and performance will suffer
greatly.  This is because every ACK increases the congestion window.
With a greatly reduced ACK rate the ramping up of CWND on startup and
after a loss event is severely limited.  Combined with ABC (appropriate
byte counting) where the CWND increases only once per ACK by at most
one MSS the effect is greatly pronounced as well.  The higher the RTT
goes the worse the effects become.  I haven't checked yet whether our
soft-LRO does ACK compression or not.  If it does, we need a workaround
and some tcp_input magic to reduce the negative impact.  I'm looking
into it.

>      - There are some 1G modules used by some customers, a couple
>        small tweaks to properly support those in the media code.
>
>      - A note: we have now done some testing of TSO6 and using
>        LRO with IPv6 and it all works great!! Seeing line rate
>        in both directions in best cases. Thanks bz for your
>        excellent work!!

Indeed!

> Modified:
>    head/sys/dev/ixgbe/ixgbe.c

-- 
Andre




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?509289A2.7080304>