From owner-freebsd-current@FreeBSD.ORG Wed Apr 2 18:54:11 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C387637B401 for ; Wed, 2 Apr 2003 18:54:11 -0800 (PST) Received: from angelica.unixdaemons.com (angelica.unixdaemons.com [209.148.64.135]) by mx1.FreeBSD.org (Postfix) with ESMTP id A3E4043F75 for ; Wed, 2 Apr 2003 18:54:10 -0800 (PST) (envelope-from bmilekic@unixdaemons.com) Received: from angelica.unixdaemons.com (localhost.unixdaemons.com [127.0.0.1])h332s5Zl092147; Wed, 2 Apr 2003 21:54:05 -0500 (EST) Received: (from bmilekic@localhost) by angelica.unixdaemons.com (8.12.9/8.12.1/Submit) id h332s47a092146; Wed, 2 Apr 2003 21:54:04 -0500 (EST) (envelope-from bmilekic@unixdaemons.com) X-Authentication-Warning: angelica.unixdaemons.com: bmilekic set sender to bmilekic@unixdaemons.com using -f Date: Wed, 2 Apr 2003 21:54:04 -0500 From: Bosko Milekic To: Nate Lawson Message-ID: <20030403025404.GA90770@unixdaemons.com> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.4.1i cc: current@freebsd.org Subject: Re: mbuf LOR X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2003 02:54:12 -0000 On Wed, Apr 02, 2003 at 12:39:30PM -0800, Nate Lawson wrote: > I was testing some changes to make fxp MPSAFE and got a LOR in allocating > the mbuf cluster and then finally a panic when trying to dereference the > cluster header. Is the mbuf system MPSAFE? Is it ok to call m_getcl > with a device lock held (but not Giant)? Not necessarily if you're doing a TRYWAIT mbuf allocation. Notably, the TRYWAIT allocation may end up in a call to the reclaim routine which calls the protocol drain routines which may end up in an LOR if you're holding some lock that can also be touched from the drain routines. So, in other words, if you do do that you'd have to be very careful. But there is another problem. kmem_malloc() still requires Giant for blockable allocations (TRYWAIT in this case) and so Giant must be held going in there if you're doing a TRYWAIT allocation. For what concerns DONTWAIT, you should theoretically be allowed to hold a driver lock. But again, there may be a problem. Specifically, I see that UMA code has some explicit Giant acquires/frees in certain places. When the UMA code gets called from the malloc() code while the bucket is being internally allocated in mb_alloc, you may be hitting one of those paths. In fact, unless you have a specific Giant acquire in the fxp_* routines you list in your trace below, that is undoubtedly what is happening because there are no explicit Giant acquires in the code path from m_getcl() to malloc(), so they must be happening higher up in the call stack. > The lock reversal was: 1. fxp softc lock, 2. Giant. > > Traceback: > zalloc... > malloc() > mb_pop_cont() > mb_alloc() > m_getcl() > fxp_add_rfabuf() > fxp_intr_body() > fxp_intr() -- locks fxp softc > > -Nate -- Bosko Milekic bmilekic@unixdaemons.com bmilekic@FreeBSD.org