Date: Sun, 27 Jun 1999 00:33:35 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: "Daniel J. O'Connor" <doconnor@gsoft.com.au> Cc: Jesus Monroy <jesus.monroy@usa.net>, Ville-Pertti Keinonen <will@iki.fi>, hackers@FreeBSD.ORG, "Daniel J.OConnor" <darius@dons.net.au> Subject: Re: [Re: [Re: coarse vs fine-grained locking in SMP systems]] Message-ID: <199906270733.AAA10635@apollo.backplane.com> References: <XFMail.990626184454.doconnor@gsoft.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Here's the basic problem: The kernel is currently designed for
single-threaded operation plus interrupt handling. A piece of code
in the kernel can temporarily disable certain interrupts with the
spl*() codes to cover situations where a race on some system resource
might occur.
But with SMP, several cpu's may be running in supervisor mode
simultaniously. The spl*() model breaks down because while one
can block interrupts, one cannot easily block another cpu that
might be running conflicting code. Resource races can now occur between
mainline code running on several cpu's simultaniously as well as between
mainline code and interrupt code.
The traditional BSD kernel code cannot deal with this new type of
race. At the moment every entry into supervisor mode is being
governed by a "big giant lock" which only allows one cpu to run
mainline code in supervisor mode at any given moment. Both cpu's
can run usermode code simultaniously just fine, but only one can
run supervisor code.
In order to make SMP operation work better, pieces of the kernel are
slowly being moved outside the "big giant lock". Linux developers,
in fact, have already moved their core data copying code and their TCP
stack outside the lock. At the moment the FreeBSD-current kernel has
not moved anything outside the lock, but John Dyson has shown that it
is fairly easy to move certain specific pieces such as the uiomove()
code outside the lock, though inefficiencies from side-effects currently
make the improvement in performance less then steller.
The real question is how to manage concurrency as pieces get moved outside
the lock. There are lots of ways to do it. One can use spin locks to
protect resources or, as someone pointed out earlier, to protect sections
of code. I don't know which is better myself, it probably depends on the
situation so a hybrid will probably be the end result. One can also use
kernel threads to simplify resource management. The advantage of a
kernel thread verses a normal process is in the ability to switch between
kernel threads very quickly, allowing the time normally wasted spining in
certain types of locks to be used more efficiently.
The problem that generally needs to be solved is the problem of stalling
on a resource. For example, if you have several threads running
simultaniously and they all need access to the same resource, serialization
of the threads occurs due to the 'blockage' on access to the resource.
(serialization means that only one thread can run at a time within the
resource, which means your efficiency drops to the efficiency of only a
single cpu). There are lots of other issues (such as cache efficiency),
but that is the big one.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199906270733.AAA10635>