From owner-svn-src-all@FreeBSD.ORG Thu Jan 28 19:57:24 2010 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DE8C410656A3; Thu, 28 Jan 2010 19:57:24 +0000 (UTC) (envelope-from trasz@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id CCAA08FC21; Thu, 28 Jan 2010 19:57:24 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o0SJvOol037859; Thu, 28 Jan 2010 19:57:24 GMT (envelope-from trasz@svn.freebsd.org) Received: (from trasz@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o0SJvO23037857; Thu, 28 Jan 2010 19:57:24 GMT (envelope-from trasz@svn.freebsd.org) Message-Id: <201001281957.o0SJvO23037857@svn.freebsd.org> From: Edward Tomasz Napierala Date: Thu, 28 Jan 2010 19:57:24 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r203127 - head/share/man/man9 X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jan 2010 19:57:25 -0000 Author: trasz Date: Thu Jan 28 19:57:24 2010 New Revision: 203127 URL: http://svn.freebsd.org/changeset/base/203127 Log: Add description of bounded sleep vs unbounded sleep (aka blocking). Move rules into their own section. Modified: head/share/man/man9/locking.9 Modified: head/share/man/man9/locking.9 ============================================================================== --- head/share/man/man9/locking.9 Thu Jan 28 19:41:26 2010 (r203126) +++ head/share/man/man9/locking.9 Thu Jan 28 19:57:24 2010 (r203127) @@ -73,38 +73,15 @@ code. Mutexes are the most commonly used synchronization primitive in the kernel. Thread acquires (locks) a mutex before accessing data shared with other threads (including interrupt threads), and releases (unlocks) it afterwards. -If the mutex cannot be acquired, the thread requesting it will block. -.Pp -Sleeping while holding mutex is generally prohibited. -You may only call the -.Xr sleep 9 -call via -.Fn msleep -or the new -.Fn mtx_sleep -variant. -These will atomically drop the mutex and reacquire it -as part of waking up. -This is often however a -.Em BAD -idea because it generally relies on you having -such a good knowledge of all the call graph above you -and what assumptions it is making that there are a lot -of ways to make hard-to-find mistakes. -For example you MUST re-test all the assumptions you made before, -all the way up the call graph to where you got the lock. -You can not just assume that mtx_sleep can be inserted anywhere. -If any caller above you has any mutex or -rwlock, your sleep, will cause a panic. -If the sleep only happens rarely it may be years before the -bad code path is found. +If the mutex cannot be acquired, the thread requesting it will sleep. +Mutexes fully support priority propagation. .Pp See the .Xr mutex 9 page for more information. .Ss Spin mutexes Spin mutexes are variation of basic mutexes; the main difference between -the two is that spin mutexes never block - instead, they spin, waiting +the two is that spin mutexes never sleep - instead, they spin, waiting for the thread holding the lock, which runs on another CPU, to release it. Differently from ordinary mutex, spin mutexes disable interrupts when acquired. Since disabling interrupts is expensive, they are also generally slower. @@ -136,23 +113,12 @@ A thread with exclusive access is known .Em writer since it may modify protected data. .Pp -Although reader/writer locks look very similar to -.Xr sx 9 -(see below) locks, their usage pattern is different. Reader/writer locks can be treated as mutexes (see above and .Xr mutex 9 ) with shared/exclusive semantics. More specifically, regular mutexes can be considered to be equivalent to a write-lock on an .Em rw_lock. -In the future this may in fact -become literally the fact. -An -.Em rw_lock -can be locked while holding a regular mutex, but -can -.Em not -be held while sleeping. The .Em rw_lock locks have priority propagation like mutexes, but priority @@ -164,19 +130,15 @@ Another important property is that share can recurse, but exclusive locks are not allowed to recurse. This ability should not be used lightly and .Em may go away. -Users of recursion in any locks should be prepared to -defend their decision against vigorous criticism. .Pp See the .Xr rwlock 9 page for more information. .Ss Read-mostly locks Mostly reader locks are similar to -.Em Reader/write -locks but optimized for very infrequent -.Em writer -locking. -.Em rm_lock +.Em reader/writer +locks but optimized for very infrequent write locking. +.Em Read-mostly locks implement full priority propagation by tracking shared owners using a lock user supplied .Em tracker @@ -186,27 +148,12 @@ See the .Xr rmlock 9 page for more information. .Ss Shared/exclusive locks -Shared/exclusive locks are used to protect data that are read far more often -than they are written. -Mutexes are inherently more efficient than shared/exclusive locks, so -shared/exclusive locks should be used prudently. -The main reason for using an -.Em sx_lock -is that a thread may hold a shared or exclusive lock on an -.Em sx_lock -lock while sleeping. -As a consequence of this however, an -.Em sx_lock -lock may not be acquired while holding a mutex. -The reason for this is that, if one thread slept while holding an -.Em sx_lock -lock while another thread blocked on the same -.Em sx_lock -lock after acquiring a mutex, then the second thread would effectively -end up sleeping while holding a mutex, which is not allowed. -The -.Em sx_lock -should be considered to be closely related to +Shared/exclusive locks are similar to reader/writer locks; the main difference +between them is that shared/exclusive locks may be held during unbounded sleep +(and may thus perform an unbounded sleep). +They are inherently less efficient than mutexes, reader/writer locks +and read-mostly locks. They don't support priority propagation. +They should be considered to be closely related to .Xr sleep 9 . In fact it could in some cases be considered a conditional sleep. @@ -239,8 +186,7 @@ See the .Xr condvar 9 page for more information. .Ss Giant -Giant is a special instance of a sleep lock. -It has several special characteristics. +Giant is an instance of a mutex, with some special characteristics: .Bl -enum .It It is recursive. @@ -255,7 +201,7 @@ Giant must be locked first before other There are places in the kernel that drop Giant and pick it back up again. Sleep locks will do this before sleeping. -Parts of the Network or VM code may do this as well, depending on the +Parts of the network or VM code may do this as well, depending on the setting of a sysctl. This means that you cannot count on Giant keeping other code from running if your code sleeps, even if you want it to. @@ -321,7 +267,7 @@ See the page for more information. .Pp .Ss Lockmanager locks -Shared/exclusive sleep locks, used mostly in +Shared/exclusive locks, used mostly in .Xr VFS 9 , in particular as a .Xr vnode 9 @@ -334,19 +280,54 @@ See the .Xr lock 9 page for more information. .Sh INTERACTIONS -.Ss Interaction table. +.Ss Bounded vs. unbounded sleep +The following primitives perform bounded sleep: mutexes, pool mutexes, +reader/writer locks and read-mostly locks. +.Pp +The following primitives block (perform unbounded sleep): shared/exclusive locks, +counting semaphores, condition variables, sleep/wakeup and lockmanager locks. +.Pp +It is an error to do any operation that could result in any kind of sleep while +holding spin mutex. +.Pp +As a general rule, it is an error to do any operation that could result +in unbounded sleep while holding any primitive from the 'bounded sleep' group. +For example, it is an error to try to acquire shared/exclusive lock while +holding mutex, or to try to allocate memory with M_WAITOK while holding +read-write lock. +.Pp +As a special case, it is possible to call +.Fn sleep 9 +or +.Fn mtx_sleep 9 +while holding a mutex. +It will atomically drop the mutex and reacquire it +as part of waking up. +This is often however a bad +idea because it generally relies on you having +such a good knowledge of all the call graph above you +and what assumptions it is making that there are a lot +of ways to make hard-to-find mistakes. +For example you must re-test all the assumptions you made before, +all the way up the call graph to where you got the lock. +You can not just assume that mtx_sleep can be inserted anywhere. +If any caller above you has any mutex or +rwlock, your sleep, will cause a panic. +If the sleep only happens rarely it may be years before the +bad code path is found. +.Ss Interaction table The following table shows what you can and can not do if you hold one of the synchronization primitives discussed here: (someone who knows what they are talking about should write this table) -.Bl -column ".Ic xxxxxxxxxxxxxxxxxxxx" ".Xr XXXXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXX" -offset indent +.Bl -column ".Ic xxxxxxxxxxxxxxxxxxx" ".Xr XXXXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXX" -offset indent .It Xo -.Em "You have: You want:" Ta Spin_mtx Ta Slp_mtx Ta sx_lock Ta rw_lock Ta rm_lock Ta sleep +.Em "You have: You want:" Ta spin mtx Ta mutex Ta sx Ta rwlock Ta rmlock Ta sleep .Xc -.It Ic SPIN mutex Ta \&ok-1 Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no-3 -.It Ic Sleep mutex Ta \&ok Ta \&ok-1 Ta \&no Ta \&ok Ta \&ok Ta \&no-3 -.It Ic sx_lock Ta \&ok Ta \&ok Ta \&ok-2 Ta \&ok Ta \&ok Ta \&ok-4 -.It Ic rw_lock Ta \&ok Ta \&ok Ta \&no Ta \&ok-2 Ta \&ok Ta \&no-3 -.It Ic rm_lock Ta \&ok Ta \&ok Ta \&no Ta \&ok Ta \&ok-2 Ta \&no +.It Ic spin mtx Ta \&ok-1 Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no-3 +.It Ic mutex Ta \&ok Ta \&ok-1 Ta \&no Ta \&ok Ta \&ok Ta \&no-3 +.It Ic sxlock Ta \&ok Ta \&ok Ta \&ok-2 Ta \&ok Ta \&ok Ta \&ok-4 +.It Ic rwlock Ta \&ok Ta \&ok Ta \&no Ta \&ok-2 Ta \&ok Ta \&no-3 +.It Ic rmlock Ta \&ok Ta \&ok Ta \&no Ta \&ok Ta \&ok-2 Ta \&no .El .Pp .Em *1 @@ -371,12 +352,12 @@ Though one can sleep holding an sx lock, .Fn sx_sleep which atomically release this primitive when going to sleep and reacquire it on wakeup. -.Ss Context mode table. +.Ss Context mode table The next table shows what can be used in different contexts. At this time this is a rather easy to remember table. -.Bl -column ".Ic Xxxxxxxxxxxxxxxxxxxx" ".Xr XXXXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXX" -offset indent +.Bl -column ".Ic Xxxxxxxxxxxxxxxxxxx" ".Xr XXXXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXXX" ".Xr XXXXXX" -offset indent .It Xo -.Em "Context:" Ta Spin_mtx Ta Slp_mtx Ta sx_lock Ta rw_lock Ta rm_lock Ta sleep +.Em "Context:" Ta spin mtx Ta mutex Ta sx Ta rwlock Ta rmlock Ta sleep .Xc .It interrupt: Ta \&ok Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no .It idle: Ta \&ok Ta \&no Ta \&no Ta \&no Ta \&no Ta \&no