From owner-freebsd-current Wed Jan 17 11: 8: 3 2001 Delivered-To: freebsd-current@freebsd.org Received: from earth.backplane.com (placeholder-dcat-1076843399.broadbandoffice.net [64.47.83.135]) by hub.freebsd.org (Postfix) with ESMTP id 926F837B6CF; Wed, 17 Jan 2001 11:07:32 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.1/8.9.3) id f0HJ7Qe48680; Wed, 17 Jan 2001 11:07:26 -0800 (PST) (envelope-from dillon) Date: Wed, 17 Jan 2001 11:07:26 -0800 (PST) From: Matt Dillon Message-Id: <200101171907.f0HJ7Qe48680@earth.backplane.com> To: Alfred Perlstein Cc: Randell Jesup , Soren Schmidt , arch@FreeBSD.ORG, current@FreeBSD.ORG Subject: Re: HEADS-UP: await/asleep removal imminent References: <200101171138.MAA11834@freebsd.dk> <20010117092109.O7240@fw.wintelcom.net> <20010117100516.Q7240@fw.wintelcom.net> Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :* Alfred Perlstein [010117 09:24] wrote: :> :> I'm not going to axe it for a few days, this is a really amazing :> API that Matt added, the problem is utility and useage over code :> complexity. :> :> It's just a proposal. : :I found several places where it may be useful, but I'm not sure if the :benefits outweigh the gains. :... : :The lock must be unwound becasue we're calling MGETHDR with M_TRYWAIT. :If wae used M_TRY'A'WAIT the code would probably look something like :this: The basic premis of using asleep()/await() is to allow you to propogate a 'blocking condition' back up to a higher level rather then blocking deep in side the kernel. The original reasoning was to deal with memory allocation blockages. For example, lets say you have three subsystem layers calling each other. The top layer wishes to implement a non-blocking API but the bottom layer might do an allocation that could block. The bottom layer could do a non-blocking allocation and return NULL, but how does the top layer (or an even higher layer) know when to try again? The original idea with asleep()/await() was for the bottom layer to call asleep() on the resource that would block and then return NULL. NULL would tell the higher layer(s) that someone down below couldn't get some resource and that an asleep() has been setup. The higher layers can then decide what to do with the situation.. they can abort the operation entirely, they can do the blocking (await() call) themselves, or they can propogate the condition to their own callers. This way you can hold more then one lock (now mutex) through a number of program layers without having to worry about them blocking on you. -- For the c urrent SMP system, asleep()/await() could be used to deal with complex situations where you (A) do not want to release a mutex through a call to another subsystem (like the memory allocator), or (B) do not know if the code calling you is already holding some mutex X and you want to hold mutex Y while you make a call to another subsystem. So, in that regard, you example: : /* SOCKBUF_UNLOCK(&so->so_snd, 0); */ :again: : if (top == 0) { : MGETHDR(m, M_TRYWAIT, MT_DATA); : if (m == NULL) { : error = mawait(&so->so_snd.sb_mtx, -1, -1); : if (error) { : if (error == EWOULDBLOCK) : error = ENOBUFS; : goto release; : } : goto again; : /* SOCKBUF_LOCK(&so->so_snd, 0); */ : } : mlen = MHLEN; :... : /* SOCKBUF_LOCK(&so->so_snd, 0); */ /* XXX */ : :Which means we don't have to drop the lock over the socket unless :we'd block on allocation. Works exactly as I originally intended. :Matt, is this what you intended for it to do? So far I've only :seen it used to avoid races, but not to optimize out mutex :aquire/release. : :-- :-Alfred Perlstein - [bright@wintelcom.net|alfred@freebsd.org] I've never liked the BSDI mutex rules, because subsystems have to have major knowledge as to how other systems operate (in reagrds to whether they can block or not), and the callee must have intimate knowledge of the callers to know that it can hold a mutex and that be the only mutex it is holding. This makes for extremely fragile and complex coding. So, this said... I'm still on the fence as to whether await()/asleep() can be used effectively. As you said, there are not too many cases at the moment and await()/asleep() does introduce significant code complexity to the scheduler, and for it to really shine it needs to be optimized to not require any sort of mutex at all in the calls to asleep(). In order to get rid of the overhead, asleep() needs to simply initialize the curproc fields and not try to actually queue the process to the sleep queue. Right now asleep() queues the process to the sleep queue (see kern/kern_synch.c) in order to support the ability for the system to asynchronously 'wake the process up again' before it actually goes to sleep (which causes the later await() to become a NOP). i.e. the situation that caused the potential blockage might be resolved before the process has a chance to sleep. Traditionally we have used SPL levels (and now mutexes) to prevent the possibility of a condition being satisfied between the test and the tsleep(). ---- Proposal Revamp asleep/await to be based on state variables rather then tsleep's traditionaln 'fake' addresses. Rather then have a traditional sleep/wakeup we instead have a state variable that asleep/await operate on. For example lets say we have a memory allocator. When the memory allocator finds it would block, it utilizes a global state structure representing the blockage and clears the state (blah.state = 0;). Then it calls asleep(&blah). asleep simply stores the pointer to &blah in the process structure, it does not try to queue it or do anything else. Thus no locking or mutexes or interrupt disablement is required *at all*. The routine is entirely passive and safe to call from anywhere. Later, some event makes more memory available for allocation. That even is asynchronous and simply sets the state variable to 1 and wakes up anybody on the sleep queue for that condition variable. This event will NOT catch the guy in the previous paragraph who has not yet called await(), however, since the call to asleep() does not actually enqueue the process (which would require a mutex). Later, the process that called asleep() finally decides to try to go to sleep for real. await() checks p_state->state and if it is zero await() places the process on the sleep queue for real and actually goes to sleep. If p_state->state is non-zero, await() simply clears the pointer (proc->p_state = NULL;) and returns (without sleeping). I believe that this conforms to the state of the SMP system much much better then my original asleep()/await() implementation, and has the advantage of extremely *LOW* overhead (virtually none in many cases). You could then use it to give the 'power of blocking' to the caller rather then the callee. This in turn gives you much greater flexibility in regards to who can hold mutexes when and who can hold mutexes through procedure calls to other subsystems. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message