Date: Fri, 14 Jan 2005 16:24:34 -0500 From: John Baldwin <jhb@FreeBSD.org> To: current@FreeBSD.org Subject: [PATCH] Divorce critical sections from spin mutexes (round 2) Message-ID: <200501141624.34752.jhb@FreeBSD.org>
next in thread | raw e-mail | index | archive | help
Ok, in the process of updating my tree that held the earlier version of the critical section vs. spin mutexes patch I think I have found and fixed the bug that may have caused the lockups a few people reported. As such, I'd like folks to test the updated patch. Details and such of what the patch does: - spin locks and critical sections are divorced. Specifically, the sole purpose of a critical section is to keep the current thread from being preempted until it exits the section. Nothing requires that the critical section actually disable interrupts during the section as any interrupt threads scheduled would simply not preempt either because they would be picked up by another CPU or preempt the current thread when it exited the critical section. However, spin locks do need to prevent themselves from being interrupted by any code that can try to acquire a spin lock. Strictly speaking, only spin mutexes used in interrupt context (sched_lock, icu_lock, locks in INTR_FAST handlers, sleepq locks, etc.) need to block interrupts, but if you have a mutex that is only used in top half code, you should probably be using a normal mutex anyway, so the set of spin mutexes not used in interrupt context tends to be small to empty. So far in SMPng, almost all critical sections have been inside of spin mutexes (since spin mutexes also need to block preemptions in addition to interrupts). Thus, for the sake of simplicity, critical sections also included the interrupt blocking behavior. (Keep in mind that this was an evolutionary process. :) However, as SMPng progresses, it has now become useful to divorce the two concepts, especially as some folks are working on locking schemes which just use critical sections to protect per-CPU resources that are not accessed from interrupt context. What this change does is to move the interrupt blocking/deferment/whatever bits that spin mutexes need into a separate spinlock_enter()/spinlock_exit() API completely implemented in MD code. critical sections, on the other hand, are now reduced to a simple per-thread nesting count and are now completely MI. - The MI code that creates idle threads for each of the CPUs no longer tries to set curthread up for the APs and no longer messes with the critnest count for the idlethreads. Instead, the MD code now explicitly borrows the idlethread context for the APs when it needs it and is responsible for adjusting the critical section and spinlock nesting counts to account for the weirdness of borrowing the context for the first context switch. I've tested this on SMP i386, SMP sparc64, and UP alpha. Testing on other archs and on SMP would be greatly appreciated. Patch is at http://www.FreeBSD.org/~jhb/patches/spinlock.patch -- John Baldwin <jhb@FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200501141624.34752.jhb>