Date: Sun, 28 Nov 1999 16:22:46 -0800 (PST) From: Julian Elischer <julian@whistle.com> To: Matthew Dillon <dillon@apollo.backplane.com> Cc: arch@freebsd.org Subject: Re: Threads stuff Message-ID: <Pine.BSF.4.10.9911281602440.544-100000@current1.whistle.com> In-Reply-To: <199911281721.JAA45015@apollo.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Mat commnets on N KSEs for M threads: On Sun, 28 Nov 1999, Matthew Dillon wrote: > > :> > Advantages: > > * Kernel stack sharing by multiple threads in the non-blocking case, > which can lead to better memory and cache resource utilization. > > * An infinite number of threads can be created as long as they do not > all need a kernel stack simultaniously. > > * Kernel management issues are complex, but not terribly so. For > example, you have a thousand threads and they all suddenly decide > to block inside the kernel - the kernel needs a mechanism to > disallow kernel entry (setup the thread to force a restart of the > system call and then switch threads because no KSE resources are > available). The UTS can know its limit on KSEs and simply catch them on the way down. Count them at the door like a night club bouncer.. one out, one in.. :-) > > Problems: > > * You have to manage dynamic allocation and deallocation of KSE's > (verses simply preassigning one per thread). > I don't think this is too hard, and we may endup using the same code in things like interrupt handler threads. :-) > * You have non-deterministic resource utilization for an application. > The worst case KSE useage is going to be one per thread, but the > userland may well allow thousands of threads to be allocated first > and then realize only later that it does not have the kernel > resources to block in all of them at once, leading to > non-deterministic blockages. programmer error? In the current world, a programmer may decide to fork 1000000 times too. > > * You compound the management issues within the userland scheduler > itself because not only can the userland scheduler switch between > threads, now so can the kernel when it decides it must block. > (this is an argument to have a kernel entry point to schedule a > thread no matter what). No the blocked thread 'returns' to the UTS, notifying it of its blocking. There has to be a UTS no matter what, even if it's copntrolling the thread contexts by remote control in the kernel (yur suggestion). If the context is all in user memory, it's pageable. In the kernel it's wired kernel memory.. which is more valuable? > * The userland scheduler has no concept of certain types of blockages, > such as VM faults, and cannot schedule around such things itself. > This can lead to non-deterministic operation. The UTS can have upcalls for these events. (see the paper on Scheduler activations) > > * The userland scheduler must deal with scheduling the N cpu case > itself - this is something more suitable to the kernel because the > userland scheduler has no knowledge of other unrelated > processes/threads running in the system. This means that > if the userland scheduler is trying to switch or schedule threads > without making a system call, the whole mess becomes much more > complex when the kernel winds up having to manage the same > threads itself. > Our solution is that you assign threads to subprocesses, and subprocesses are assigned to a processor. A thread that is not running can be assigned to a new subprocess by sumply changing the list it is on. > This isn't to say that the userland scheduler cannot choose a > loosely (or hard) bound cpu for a thread, only that the kernel is > better able to actualy get the thread running deterministically > because the userland scheduler might end up 'stuck' in a VM > fault or something similar. that is unlikely to occur. I would assume that any thread library would get that memory wired down and it would be being touched all the time anyhow. > > I am beginning to warm to the dynamic KSE concept but I think the only > way to avoid unnecessary complexity AND to avoid potential resource > sharing problems is to impose two requirements: > > * First, that thread runnability be controlled through a system call. > Ther userland can determine when to halt and run a thread and can > even assign a cpu, but it must make a system call to actually > schedule or deschedule the thread. We are rforking to make subprocs. a subproc runs at one priority on one CPU. it has it's struct proc, (or most of one :-) and the threads are assigned to it. Any KSE that is created is automatically hung off that subproc. it will ony be run on that cpu unles there is intervention. (of which there may be may types, fro example a related subproc may be totally idle at teh time a KSE becomes unblocked.. it would be a waste to not use that other CPU) > > * Second, KSE management. The kernel has a limited number of KSE's > available and cannot allow any one process (containing multiple > threads) to hog them. > > KSE management must be dynamic on-entry. This means that when a > thread makes a system call and enters the kernel, the kernel must > make a determination in regards to the availability of KSE's. If > insufficient KSE's are available (or the process is using too many > already), the kernel must setup the thread to restart the system > call, deschedule it, and switch to another thread. The UTS could do the same thing.. and there has to be a UTS of some sort. > > i.e. there is always a 'current KSE' for a running thread, but if > it is the last one and the system call MIGHT block, the kernel > cannot afford to allow the thread to block in the one remaining > KSE and must deschedule the thread until KSEs are available to > handle the system call. > > > -Matt > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.10.9911281602440.544-100000>