Date: Sat, 13 Oct 2012 20:22:23 +0200 From: Luigi Rizzo <rizzo@iet.unipi.it> To: "Alexander V. Chernikov" <melifaro@freebsd.org> Cc: Jack Vogel <jfvogel@gmail.com>, net@freebsd.org Subject: Re: ixgbe & if_igb RX ring locking Message-ID: <20121013182223.GA73341@onelab2.iet.unipi.it> In-Reply-To: <5079A9A1.4070403@FreeBSD.org> References: <5079A9A1.4070403@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Oct 13, 2012 at 09:49:21PM +0400, Alexander V. Chernikov wrote:
> Hello list!
>
>
> Packets receiving code for both ixgbe and if_igb looks like the following:
>
>
> ixgbe_msix_que
>
> -- ixgbe_rxeof()
> {
> IXGBE_RX_LOCK(rxr);
> while
> {
> get_packet;
>
> -- ixgbe_rx_input()
> {
> ++ IXGBE_RX_UNLOCK(rxr);
> if_input(packet);
> ++ IXGBE_RX_LOCK(rxr);
> }
>
> }
> IXGBE_RX_UNLOCK(rxr);
> }
>
> Lines marked with ++ appeared in r209068(igb) and r217593(ixgbe).
>
> These lines probably do LORs masking (if any) well.
> However, such change introduce quite significant performance drop:
>
> On my routing setup (nearly the same from previous -Intel 10G thread in
> -net) adding lock/unlock causes 2.8MPPS decrease to 2.3MPPS which is
> nearly 20%.
one option could be (same as it is done in the timer
routine in dummynet) to build a list of all the packets
that need to be sent to if_input(), and then call
if_input with the entire list outside the lock.
It would be even easier if we modify the various *_input()
routines to handle a list of mbufs instead of just one.
cheers
luigi
> So my questions are:
>
> Can any real LORs happen in some complex setup? (I can't imagine any).
> If so: maybe we can somehow avoid/workaround such cases? (and consider
> removing those locks).
>
>
>
> --
> WBR, Alexander
>
> _______________________________________________
> freebsd-net@freebsd.org mailing list
> http://lists.freebsd.org/mailman/listinfo/freebsd-net
> To unsubscribe, send any mail to "freebsd-net-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121013182223.GA73341>