From owner-svn-src-head@FreeBSD.ORG Sun May 9 17:04:17 2010 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 60BE8106566B; Sun, 9 May 2010 17:04:17 +0000 (UTC) (envelope-from marius@alchemy.franken.de) Received: from alchemy.franken.de (alchemy.franken.de [194.94.249.214]) by mx1.freebsd.org (Postfix) with ESMTP id C88D08FC08; Sun, 9 May 2010 17:04:16 +0000 (UTC) Received: from alchemy.franken.de (localhost [127.0.0.1]) by alchemy.franken.de (8.14.3/8.14.3/ALCHEMY.FRANKEN.DE) with ESMTP id o49H4FKe022289; Sun, 9 May 2010 19:04:15 +0200 (CEST) (envelope-from marius@alchemy.franken.de) Received: (from marius@localhost) by alchemy.franken.de (8.14.3/8.14.3/Submit) id o49H4FcP022288; Sun, 9 May 2010 19:04:15 +0200 (CEST) (envelope-from marius) Date: Sun, 9 May 2010 19:04:15 +0200 From: Marius Strobl To: Attilio Rao Message-ID: <20100509170414.GA35602@alchemy.franken.de> References: <201004192327.o3JNRsK4047856@svn.freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201004192327.o3JNRsK4047856@svn.freebsd.org> User-Agent: Mutt/1.4.2.3i Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r206878 - head/sys/kern X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 May 2010 17:04:17 -0000 On Mon, Apr 19, 2010 at 11:27:54PM +0000, Attilio Rao wrote: > Author: attilio > Date: Mon Apr 19 23:27:54 2010 > New Revision: 206878 > URL: http://svn.freebsd.org/changeset/base/206878 > > Log: > Fix a deadlock in the shutdown code: > When performing a smp_rendezvous() or more likely, on amd64 and i386, > a smp_tlb_shootdown() the caller will end up with the smp_ipi_mtx > spinlock held, busy-waiting for other CPUs to acknowledge the operation. > As long as CPUs are suspended (via cpu_reset()) between the active mask > read and IPI sending there can be a deadlock where the caller will wait > forever for a dead CPU to acknowledge the operation. > Please note that on CPU0 that is going to be someway heavier because of > the spinlocks being disabled earlier than quitting the machine. > > Fix this bug by calling cpu_reset() with the smp_ipi_mtx held. > Note that it is very likely that a saner offline/online CPUs mechanism > will help heavilly in fixing similar cases as it is likely more bugs > of this type may arise in the future. > This change causes a hang for me when running an SMP kernel on an UP machine or an MP machine with SMP disabled as in these cases smp_ipi_mtx isn't initialized (see mp_start()). The below patch fixes this for me. Marius Index: kern_shutdown.c =================================================================== --- kern_shutdown.c (revision 207463) +++ kern_shutdown.c (working copy) @@ -491,8 +491,8 @@ shutdown_reset(void *junk, int howto) /* * Acquiring smp_ipi_mtx here has a double effect: * - it disables interrupts avoiding CPU0 preemption - * by fast handlers (thus deadlocking against other CPUs) - * - it avoids deadlocks against smp_rendezvous() or, more + * by fast handlers (thus deadlocking against other CPUs) + * - it avoids deadlocks against smp_rendezvous() or, more * generally, threads busy-waiting, with this spinlock held, * and waiting for responses by threads on other CPUs * (ie. smp_tlb_shootdown()). @@ -500,9 +500,11 @@ shutdown_reset(void *junk, int howto) * For the !SMP case it just needs to handle the former problem. */ #ifdef SMP - mtx_lock_spin(&smp_ipi_mtx); + if (mtx_initialized(&smp_ipi_mtx) != 0) + mtx_lock_spin(&smp_ipi_mtx); + else #else - spinlock_enter(); + spinlock_enter(); #endif /* cpu_boot(howto); */ /* doesn't do anything at the moment */