From owner-svn-src-all@FreeBSD.ORG Mon Apr 19 23:27:54 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D6C951065675; Mon, 19 Apr 2010 23:27:54 +0000 (UTC) (envelope-from attilio@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id AB9478FC19; Mon, 19 Apr 2010 23:27:54 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o3JNRsrA047858; Mon, 19 Apr 2010 23:27:54 GMT (envelope-from attilio@svn.freebsd.org) Received: (from attilio@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o3JNRsK4047856; Mon, 19 Apr 2010 23:27:54 GMT (envelope-from attilio@svn.freebsd.org) Message-Id: <201004192327.o3JNRsK4047856@svn.freebsd.org> From: Attilio Rao Date: Mon, 19 Apr 2010 23:27:54 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r206878 - head/sys/kern X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 Apr 2010 23:27:54 -0000 Author: attilio Date: Mon Apr 19 23:27:54 2010 New Revision: 206878 URL: http://svn.freebsd.org/changeset/base/206878 Log: Fix a deadlock in the shutdown code: When performing a smp_rendezvous() or more likely, on amd64 and i386, a smp_tlb_shootdown() the caller will end up with the smp_ipi_mtx spinlock held, busy-waiting for other CPUs to acknowledge the operation. As long as CPUs are suspended (via cpu_reset()) between the active mask read and IPI sending there can be a deadlock where the caller will wait forever for a dead CPU to acknowledge the operation. Please note that on CPU0 that is going to be someway heavier because of the spinlocks being disabled earlier than quitting the machine. Fix this bug by calling cpu_reset() with the smp_ipi_mtx held. Note that it is very likely that a saner offline/online CPUs mechanism will help heavilly in fixing similar cases as it is likely more bugs of this type may arise in the future. Reported by: rwatson Discussed with: jhb Tested by: rnoland, Giovanni Trematerra MFC: 2 weeks Special deciation to: anyone who made possible to have 16-ways machines in Netperf Modified: head/sys/kern/kern_shutdown.c Modified: head/sys/kern/kern_shutdown.c ============================================================================== --- head/sys/kern/kern_shutdown.c Mon Apr 19 22:15:40 2010 (r206877) +++ head/sys/kern/kern_shutdown.c Mon Apr 19 23:27:54 2010 (r206878) @@ -62,7 +62,7 @@ __FBSDID("$FreeBSD$"); #include #include #include -#include /* smp_active */ +#include #include #include @@ -485,15 +485,20 @@ static void shutdown_reset(void *junk, int howto) { + printf("Rebooting...\n"); + DELAY(1000000); /* wait 1 sec for printf's to complete and be read */ + /* - * Disable interrupts on CPU0 in order to avoid fast handlers - * to preempt the stopping process and to deadlock against other - * CPUs. + * Acquiring smp_ipi_mtx here has a double effect: + * - it disables interrupts avoiding CPU0 preemption + * by fast handlers (thus deadlocking against other CPUs) + * - it avoids deadlocks against smp_rendezvous() or, more + * generally, threads busy-waiting, with this spinlock held, + * and waiting for responses by threads on other CPUs + * (ie. smp_tlb_shootdown()). */ - spinlock_enter(); + mtx_lock_spin(&smp_ipi_mtx); - printf("Rebooting...\n"); - DELAY(1000000); /* wait 1 sec for printf's to complete and be read */ /* cpu_boot(howto); */ /* doesn't do anything at the moment */ cpu_reset(); /* NOTREACHED */ /* assuming reset worked */