Date: Sun, 20 Dec 1998 23:13:26 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: Alfred Perlstein <bright@hotjobs.com> Cc: hackers@FreeBSD.ORG Subject: Re: question about re-entrancy. Message-ID: <199812210713.XAA35942@apollo.backplane.com> References: <Pine.BSF.4.05.9812201621560.6331-100000@bright.fx.genx.net>
next in thread | previous in thread | raw e-mail | index | archive | help
:I'm a bit confused. I've been watching the discussions going on about
:SMP. The issue of kernel re-entrancy has been brought up many times.
:
:So here's the question:
:
:I thought the kernel was re-entrant, when a process makes a syscall it's
:thread of execution enters the kernel where its arguments are validated
:and careful checks are done to ensure security then the syscall is done.
:
:If during the system call the process is put to sleep or rescheduled,
:another process may enter the kernel. As long as the splxxx() calls do
:not interfere with each other you have 2 processes excuting in the kernel
:at that point. (more can enter as well)
:
:Or.... what i'm now thinking is that once a process enters the kernel it
:must run until it exits the system call. I have a problem with this
:...
This is how it works: There is a big lock around the *execution* of
supervisor code. Once a process running in supervisor mode goes to
sleep, it is no longer executing and the lock is released. The lock
is regained when the supervisor-mode-context process is woken up and
gets cpu again.
So, for example, if you have a shell that is blocked in a read() system
call, it is not actually executing anything even though it is in a
supervisor context.
The spl*() calls work differently. For one thing, the process making
the calls already owns the overall SMP lock so, really, the spl*() stuff
is independant. SMP motherboards have an interrupt-forwarding
capability allowing interrupts to be assigned to specific cpu's. FreeBSD
just assigns all of them to one cpu or the other depending on which one
currently owns the SMP lock. Thus when an interrupt does occur, it
always occurs on the cpu that is *already* running in supervisor mode.
spl*() calls do not disable specific interrupts. Instead they add to
a 'bitmask' representing disabled interrupt sources. Various spl*() calls
will disable different (but often overlapping) masks. In fact, they
do not disable anything at all... they simply set a global variable.
When an interrupt occurs the global is checked and if the interrupt has
been marked as being masked, it sets a bit in another global saying that
it is pending and returns immediately without executing the interrupt
routine designated for that interrupt. When the spl is 'restored',
a check is made between the global disable mask and the global pending
mask and any pending interrupts that get unmasked are executed then.
The spl mask is a per-process element. When a process which has masked
interrupts goes to sleep, the mask is saved along with the process and
interrupts are effectively enabled again (when a process is switched in,
the mask is 'restored' based on whatever it was when the process went
to sleep). This is, in fact, a big problem with the current kernel
because sometimes a piece of code will disable interrupts and then
unexpectedly sleep synchronously. Even though the interrupts are still
disabled on resumption, the code might have assumed atomic operation
when in fact atomic operation wound up getting broken by the sleep.
-Matt
Matthew Dillon Engineering, HiWay Technologies, Inc. & BEST Internet
Communications & God knows what else.
<dillon@backplane.com> (Please include original email in any response)
:though, processes such as nfsiod/nfsd enter the kernel through a syscall
:and never exit (according to design and implementation) how then are other
:processes allowed into the kernel?
:
:Can anyone explain this a bit better?
:
:I understand deadlock can occur if something like this happens:
:
:process 1 process 2
:syscall()
:splbio()
:rescheduled ----->
: syscall()
: splimp()
: <----- rescheduled
:splimp()? -no,reschedule->
: <-no,reschedule- splbio()?
:splimp()? -no,reschedule->
: splbio()?
:
:some sorta flipflop deadlock happens at this point as nested locks are
:held and won't be released as two in kernel processes never relinquish
:resources.
:
:Or are the spl calls simply there to diable interupts? but that brings me
:back to the question about "well how do nfsd and nfsiod do it?"
:
:Perhaps once an spl is initiated the process can not block?
:
:the more I thought about this, the more my head ached :)
:
:Mike, Terry, Matt, John?
:
:thanks,
:Alfred Perlstein - Programmer, HotJobs Inc. - www.hotjobs.com
:-- There are operating systems, and then there's FreeBSD.
:-- http://www.freebsd.org/ 3.0-current
:
:
:To Unsubscribe: send mail to majordomo@FreeBSD.org
:with "unsubscribe freebsd-hackers" in the body of the message
:
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199812210713.XAA35942>