Date: Mon, 19 Apr 2010 23:27:54 +0000 (UTC) From: Attilio Rao <attilio@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r206878 - head/sys/kern Message-ID: <201004192327.o3JNRsK4047856@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: attilio Date: Mon Apr 19 23:27:54 2010 New Revision: 206878 URL: http://svn.freebsd.org/changeset/base/206878 Log: Fix a deadlock in the shutdown code: When performing a smp_rendezvous() or more likely, on amd64 and i386, a smp_tlb_shootdown() the caller will end up with the smp_ipi_mtx spinlock held, busy-waiting for other CPUs to acknowledge the operation. As long as CPUs are suspended (via cpu_reset()) between the active mask read and IPI sending there can be a deadlock where the caller will wait forever for a dead CPU to acknowledge the operation. Please note that on CPU0 that is going to be someway heavier because of the spinlocks being disabled earlier than quitting the machine. Fix this bug by calling cpu_reset() with the smp_ipi_mtx held. Note that it is very likely that a saner offline/online CPUs mechanism will help heavilly in fixing similar cases as it is likely more bugs of this type may arise in the future. Reported by: rwatson Discussed with: jhb Tested by: rnoland, Giovanni Trematerra <giovanni dot trematerra at gmail dot com> MFC: 2 weeks Special deciation to: anyone who made possible to have 16-ways machines in Netperf Modified: head/sys/kern/kern_shutdown.c Modified: head/sys/kern/kern_shutdown.c ============================================================================== --- head/sys/kern/kern_shutdown.c Mon Apr 19 22:15:40 2010 (r206877) +++ head/sys/kern/kern_shutdown.c Mon Apr 19 23:27:54 2010 (r206878) @@ -62,7 +62,7 @@ __FBSDID("$FreeBSD$"); #include <sys/reboot.h> #include <sys/resourcevar.h> #include <sys/sched.h> -#include <sys/smp.h> /* smp_active */ +#include <sys/smp.h> #include <sys/sysctl.h> #include <sys/sysproto.h> @@ -485,15 +485,20 @@ static void shutdown_reset(void *junk, int howto) { + printf("Rebooting...\n"); + DELAY(1000000); /* wait 1 sec for printf's to complete and be read */ + /* - * Disable interrupts on CPU0 in order to avoid fast handlers - * to preempt the stopping process and to deadlock against other - * CPUs. + * Acquiring smp_ipi_mtx here has a double effect: + * - it disables interrupts avoiding CPU0 preemption + * by fast handlers (thus deadlocking against other CPUs) + * - it avoids deadlocks against smp_rendezvous() or, more + * generally, threads busy-waiting, with this spinlock held, + * and waiting for responses by threads on other CPUs + * (ie. smp_tlb_shootdown()). */ - spinlock_enter(); + mtx_lock_spin(&smp_ipi_mtx); - printf("Rebooting...\n"); - DELAY(1000000); /* wait 1 sec for printf's to complete and be read */ /* cpu_boot(howto); */ /* doesn't do anything at the moment */ cpu_reset(); /* NOTREACHED */ /* assuming reset worked */
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201004192327.o3JNRsK4047856>