From owner-freebsd-net@FreeBSD.ORG Tue Oct 16 12:13:08 2012 Return-Path: Delivered-To: net@freebsd.org Received: from mx2.freebsd.org (mx2.freebsd.org [69.147.83.53]) by hub.freebsd.org (Postfix) with ESMTP id 6A9293A5; Tue, 16 Oct 2012 12:13:08 +0000 (UTC) (envelope-from melifaro@FreeBSD.org) Received: from dhcp170-36-red.yandex.net (freefall.freebsd.org [8.8.178.135]) by mx2.freebsd.org (Postfix) with ESMTP id C7F9A3B5C58; Tue, 16 Oct 2012 12:13:06 +0000 (UTC) Message-ID: <507D4F11.2030704@FreeBSD.org> Date: Tue, 16 Oct 2012 16:12:01 +0400 From: "Alexander V. Chernikov" User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:13.0) Gecko/20120627 Thunderbird/13.0.1 MIME-Version: 1.0 To: Ryan Stone Subject: Re: ixgbe & if_igb RX ring locking References: <5079A9A1.4070403@FreeBSD.org> <20121015162926.GV89655@FreeBSD.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Jack Vogel , net@freebsd.org X-BeenThere: freebsd-net@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Networking and TCP/IP with FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Oct 2012 12:13:08 -0000 On 16.10.2012 00:48, Ryan Stone wrote: > On Mon, Oct 15, 2012 at 12:29 PM, Gleb Smirnoff wrote: >> To me this unlock/lock looks like a legacy from times, when the driver >> had a single mutex for both TX and RX parts. >> >> And removing this re-locking in foo_rxeof() was one of the aims for separate >> TX/RX locking. >> >> Really, lurking through history shows that once driver had split its locking >> to separate RX and TX part, these unlock/lock was removed. However, later >> this unlock/lock was added back: >> >> http://svnweb.freebsd.org/base/head/sys/dev/e1000/if_igb.c?revision=209068&view=markup >> >> , without any comments for the reason it is added back. > > There's a convoluted LOR if you call into the stack with the RX lock > held which is described here: > > http://lists.freebsd.org/pipermail/freebsd-net/2012-September/033371.html Are you using stock ixgbe driver? lock order reversal:^M^M 1st 0xffffff800153c138 ix:rx (ix:rx) @ src/sys/dev/ixgbe/ixgbe.c:7113^M^M 2nd 0xffffffff80af9c48 udp (udp) @ src/sys/netinet/udp_usrreq.c:471^M^M It seems to me than ixgbe.c was always like ~5.5k lines of code, line 7113 seems a bit suspicious. 2nd 0xffffff8001539400 ixgbe0 (IXGBE Core Lock) @ src/sys/dev/ixgbe/ixgbe.c:1725 Nearest IXGBE_CORE_LOCK() in r217917 (8.2-R) resides at line 905. Maybe I'm missing something obvious? > -- WBR, Alexander