Date: Wed, 17 Jan 2001 11:07:26 -0800 (PST) From: Matt Dillon <dillon@earth.backplane.com> To: Alfred Perlstein <bright@wintelcom.net> Cc: Randell Jesup <rjesup@wgate.com>, Soren Schmidt <sos@freebsd.dk>, arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent Message-ID: <200101171907.f0HJ7Qe48680@earth.backplane.com> References: <200101171138.MAA11834@freebsd.dk> <ybug0iixiee.fsf@jesup.eng.tvol.net.jesup.eng.tvol.net> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
:* Alfred Perlstein <bright@wintelcom.net> [010117 09:24] wrote:
:>
:> I'm not going to axe it for a few days, this is a really amazing
:> API that Matt added, the problem is utility and useage over code
:> complexity.
:>
:> It's just a proposal.
:
:I found several places where it may be useful, but I'm not sure if the
:benefits outweigh the gains.
:...
:
:The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT.
:If wae used M_TRY'A'WAIT the code would probably look something like
:this:
The basic premis of using asleep()/await() is to allow you to
propogate a 'blocking condition' back up to a higher level rather
then blocking deep in side the kernel.
The original reasoning was to deal with memory allocation blockages.
For example, lets say you have three subsystem layers calling each
other. The top layer wishes to implement a non-blocking API but the
bottom layer might do an allocation that could block.
The bottom layer could do a non-blocking allocation and return NULL,
but how does the top layer (or an even higher layer) know when to try
again?
The original idea with asleep()/await() was for the bottom layer to
call asleep() on the resource that would block and then return NULL.
NULL would tell the higher layer(s) that someone down below couldn't
get some resource and that an asleep() has been setup.
The higher layers can then decide what to do with the situation.. they
can abort the operation entirely, they can do the blocking (await() call)
themselves, or they can propogate the condition to their own callers.
This way you can hold more then one lock (now mutex) through a number
of program layers without having to worry about them blocking on you.
--
For the c urrent SMP system, asleep()/await() could be used to deal with
complex situations where you (A) do not want to release a mutex through
a call to another subsystem (like the memory allocator), or (B) do not
know if the code calling you is already holding some mutex X and you
want to hold mutex Y while you make a call to another subsystem.
So, in that regard, you example:
: /* SOCKBUF_UNLOCK(&so->so_snd, 0); */
:again:
: if (top == 0) {
: MGETHDR(m, M_TRYWAIT, MT_DATA);
: if (m == NULL) {
: error = mawait(&so->so_snd.sb_mtx, -1, -1);
: if (error) {
: if (error == EWOULDBLOCK)
: error = ENOBUFS;
: goto release;
: }
: goto again;
: /* SOCKBUF_LOCK(&so->so_snd, 0); */
: }
: mlen = MHLEN;
:...
: /* SOCKBUF_LOCK(&so->so_snd, 0); */ /* XXX */
:
:Which means we don't have to drop the lock over the socket unless
:we'd block on allocation.
Works exactly as I originally intended.
:Matt, is this what you intended for it to do? So far I've only
:seen it used to avoid races, but not to optimize out mutex
:aquire/release.
:
:--
:-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org]
I've never liked the BSDI mutex rules, because subsystems have to have
major knowledge as to how other systems operate (in reagrds to whether
they can block or not), and the callee must have intimate knowledge of
the callers to know that it can hold a mutex and that be the only mutex
it is holding.
This makes for extremely fragile and complex coding.
So, this said... I'm still on the fence as to whether await()/asleep()
can be used effectively. As you said, there are not too many cases
at the moment and await()/asleep() does introduce significant code
complexity to the scheduler, and for it to really shine it needs to be
optimized to not require any sort of mutex at all in the calls to
asleep().
In order to get rid of the overhead, asleep() needs to simply initialize
the curproc fields and not try to actually queue the process to the
sleep queue. Right now asleep() queues the process to the sleep queue
(see kern/kern_synch.c) in order to support the ability for the system
to asynchronously 'wake the process up again' before it actually goes to
sleep (which causes the later await() to become a NOP). i.e. the
situation that caused the potential blockage might be resolved before
the process has a chance to sleep. Traditionally we have used SPL levels
(and now mutexes) to prevent the possibility of a condition being
satisfied between the test and the tsleep().
----
Proposal
Revamp asleep/await to be based on state variables rather then
tsleep's traditionaln 'fake' addresses. Rather then
have a traditional sleep/wakeup we instead have a state variable that
asleep/await operate on. For example lets say we have a memory allocator.
When the memory allocator finds it would block, it utilizes a global
state structure representing the blockage and clears the state
(blah.state = 0;). Then it calls asleep(&blah). asleep simply stores
the pointer to &blah in the process structure, it does not try to queue
it or do anything else. Thus no locking or mutexes or interrupt
disablement is required *at all*. The routine is entirely passive and
safe to call from anywhere.
Later, some event makes more memory available for allocation. That
even is asynchronous and simply sets the state variable to 1
and wakes up anybody on the sleep queue for that condition variable.
This event will NOT catch the guy in the previous paragraph who has
not yet called await(), however, since the call to asleep() does not
actually enqueue the process (which would require a mutex).
Later, the process that called asleep() finally decides to try
to go to sleep for real. await() checks p_state->state and if it
is zero await() places the process on the sleep queue for real and actually
goes to sleep. If p_state->state is non-zero, await() simply clears
the pointer (proc->p_state = NULL;) and returns (without sleeping).
I believe that this conforms to the state of the SMP system much much
better then my original asleep()/await() implementation, and has the
advantage of extremely *LOW* overhead (virtually none in many cases).
You could then use it to give the 'power of blocking' to the caller
rather then the callee. This in turn gives you much greater flexibility
in regards to who can hold mutexes when and who can hold mutexes through
procedure calls to other subsystems.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200101171907.f0HJ7Qe48680>