From owner-cvs-src-old@FreeBSD.ORG Tue May 24 13:36:53 2011 Return-Path: Delivered-To: cvs-src-old@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 40E821065673 for ; Tue, 24 May 2011 13:36:53 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repoman.freebsd.org (repoman.freebsd.org [IPv6:2001:4f8:fff6::29]) by mx1.freebsd.org (Postfix) with ESMTP id 1762C8FC12 for ; Tue, 24 May 2011 13:36:53 +0000 (UTC) Received: from repoman.freebsd.org (localhost [127.0.0.1]) by repoman.freebsd.org (8.14.4/8.14.4) with ESMTP id p4ODaqc5021577 for ; Tue, 24 May 2011 13:36:52 GMT (envelope-from jhb@repoman.freebsd.org) Received: (from svn2cvs@localhost) by repoman.freebsd.org (8.14.4/8.14.4/Submit) id p4ODaqsN021576 for cvs-src-old@freebsd.org; Tue, 24 May 2011 13:36:52 GMT (envelope-from jhb@repoman.freebsd.org) Message-Id: <201105241336.p4ODaqsN021576@repoman.freebsd.org> X-Authentication-Warning: repoman.freebsd.org: svn2cvs set sender to jhb@repoman.freebsd.org using -f From: John Baldwin Date: Tue, 24 May 2011 13:36:41 +0000 (UTC) To: cvs-src-old@freebsd.org X-FreeBSD-CVS-Branch: HEAD Subject: cvs commit: src/sys/kern subr_smp.c X-BeenThere: cvs-src-old@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: **OBSOLETE** CVS commit messages for the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 May 2011 13:36:53 -0000 jhb 2011-05-24 13:36:41 UTC FreeBSD src repository Modified files: sys/kern subr_smp.c Log: SVN rev 222254 on 2011-05-24 13:36:41Z by jhb Fix an issue with critical sections and SMP rendezvous handlers. Specifically, a critical_exit() call that drops the nesting level to zero has a brief window where the pending preemption flag is set and the nesting level is set to zero. This is done purposefully to avoid races where a preemption scheduled by an interrupt could be lost otherwise (see revision 144777). However, this does mean that if an interrupt fires during this window and enters and exits a critical section, it may preempt from the interrupt context. This is generally fine as the interrupt code is careful to arrange critical sections so that they are not exited until it is safe to preempt (e.g. interrupts EOI'd and masked if necessary). However, the SMP rendezvous IPI handler does not quite follow this rule, and in general a rendezvous can never be preempted. Rendezvous handlers are also not permitted to schedule threads to execute, so they will not typically trigger preemptions. SMP rendezvous handlers may use spinlocks (carefully) such as the rm_cleanIPI() handler used in rmlocks, but using a spinlock also enters and exits a critical section. If the interrupted top-half code is in the brief window of critical_exit() where the nesting level is zero but a preemption is pending, then releasing the spinlock can trigger a preemption. Because we know that SMP rendezvous handlers can never schedule a thread, we know that a critical_exit() in an SMP rendezvous handler will only preempt in this edge case. We also know that the top-half thread will happily handle the deferred preemption once the SMP rendezvous has completed, so the preemption will not be lost. This makes it safe to employ a workaround where we use a nested critical section in the SMP rendezvous code itself around rendezvous action routines to prevent any preemptions during an SMP rendezvous. The workaround intentionally avoids checking for a deferred preemption when leaving the critical section on the assumption that if there is a pending preemption it will be handled by the interrupted top-half code. Submitted by: mlaier (variation specific to rm_cleanIPI()) Obtained from: Isilon MFC after: 1 week Revision Changes Path 1.227 +43 -7 src/sys/kern/subr_smp.c