From owner-freebsd-net Tue Jan 7 7: 3:26 2003 Delivered-To: freebsd-net@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 62D0437B401; Tue, 7 Jan 2003 07:03:24 -0800 (PST) Received: from duke.cs.duke.edu (duke.cs.duke.edu [152.3.140.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id BB2DD43ED8; Tue, 7 Jan 2003 07:03:23 -0800 (PST) (envelope-from gallatin@cs.duke.edu) Received: from grasshopper.cs.duke.edu (grasshopper.cs.duke.edu [152.3.145.30]) by duke.cs.duke.edu (8.12.6/8.12.6) with ESMTP id h07F3Kro007789 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO); Tue, 7 Jan 2003 10:03:20 -0500 (EST) Received: (from gallatin@localhost) by grasshopper.cs.duke.edu (8.11.6/8.9.1) id h07F3Ff59535; Tue, 7 Jan 2003 10:03:15 -0500 (EST) (envelope-from gallatin@cs.duke.edu) From: Andrew Gallatin MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <15898.60467.681434.927797@grasshopper.cs.duke.edu> Date: Tue, 7 Jan 2003 10:03:15 -0500 (EST) To: "M. Warner Losh" Cc: tlambert2@mindspring.com, nate@root.org, current@FreeBSD.ORG, net@FreeBSD.ORG Subject: Re: Proper -current if_attach locking? In-Reply-To: <20030107.022617.23700606.imp@bsdimp.com> References: <20030107.001924.02080410.imp@bsdimp.com> <3E1A87A8.F9C079F@mindspring.com> <20030107.021428.126452776.imp@bsdimp.com> <20030107.022617.23700606.imp@bsdimp.com> X-Mailer: VM 6.75 under 21.1 (patch 12) "Channel Islands" XEmacs Lucid Sender: owner-freebsd-net@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org M. Warner Losh writes: <..> > However in if_slowtimo we have: > > if_slowtimo(arg) > { > ... IFNET_RLOCK(); > ... if (ifp->if_watchdog) > (*ifp->if_watchdog)(ifp); > ... IFNET_RUNLOCK(); > } > > and dc_watchdog does a DC_LOCK/UNLOCK pair). This is a Lock Order > Reversal, and not a LotR :-) > > What's worse is that dc_intr does: > > DC_LOCK > ...dc_start (which calls IF_PREPEND which does the IFNET_LOCK/UNLOCK thing) > DC_UNLOCK > > So even if we remove the one from attach, it looks like we have others > lurking in the code. > > Either that, or it is too late for me to be looking at code like this > :-( > I think its too late at night ;) The IFNET_RLOCK() called in if_slowtimo() is a global lock for the list of ifnet structs to ensure that no devices are removed or added while something may be using it. There is one ifnet list in the system. The lock in IF_PREPEND() (and more commonly used in drivers, IF_DEQUE()) is per-ifq, to protect against multiple accesses to a single ifq. There are many ifqs in the system. FWIW, I've been running my 3rd party Myrinet driver Giant-free and have had no problems, and no lock order reversals. I don't do bogus locking in my attach routine, though :) FWIW2: Running Giant-free brings -current TCP performance up to nearly 63% of -stable performance (from 39%), and udp xmit perf up to 87% of -stable. (testing w/o WITNESS). Drew To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-net" in the body of the message