Date: Mon, 15 Oct 2012 20:29:26 +0400 From: Gleb Smirnoff <glebius@FreeBSD.org> To: "Alexander V. Chernikov" <melifaro@FreeBSD.org> Cc: Jack Vogel <jfvogel@gmail.com>, net@FreeBSD.org Subject: Re: ixgbe & if_igb RX ring locking Message-ID: <20121015162926.GV89655@FreeBSD.org> In-Reply-To: <5079A9A1.4070403@FreeBSD.org> References: <5079A9A1.4070403@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Oct 13, 2012 at 09:49:21PM +0400, Alexander V. Chernikov wrote:
A> Packets receiving code for both ixgbe and if_igb looks like the following:
A> ixgbe_msix_que
A>
A> -- ixgbe_rxeof()
A> {
A> IXGBE_RX_LOCK(rxr);
A> while
A> {
A> get_packet;
A>
A> -- ixgbe_rx_input()
A> {
A> ++ IXGBE_RX_UNLOCK(rxr);
A> if_input(packet);
A> ++ IXGBE_RX_LOCK(rxr);
A> }
A>
A> }
A> IXGBE_RX_UNLOCK(rxr);
A> }
A>
A> Lines marked with ++ appeared in r209068(igb) and r217593(ixgbe).
A>
A> These lines probably do LORs masking (if any) well.
A> However, such change introduce quite significant performance drop:
A>
A> On my routing setup (nearly the same from previous -Intel 10G thread in
A> -net) adding lock/unlock causes 2.8MPPS decrease to 2.3MPPS which is
A> nearly 20%.
A>
A> So my questions are:
A>
A> Can any real LORs happen in some complex setup? (I can't imagine any).
A> If so: maybe we can somehow avoid/workaround such cases? (and consider
A> removing those locks).
To me this unlock/lock looks like a legacy from times, when the driver
had a single mutex for both TX and RX parts.
And removing this re-locking in foo_rxeof() was one of the aims for separate
TX/RX locking.
Really, lurking through history shows that once driver had split its locking
to separate RX and TX part, these unlock/lock was removed. However, later
this unlock/lock was added back:
http://svnweb.freebsd.org/base/head/sys/dev/e1000/if_igb.c?revision=209068&view=markup
, without any comments for the reason it is added back.
--
Totus tuus, Glebius.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20121015162926.GV89655>