Date: Sun, 28 Nov 1999 09:21:58 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: "Daniel M. Eischen" <eischen@vigrid.com> Cc: Julian Elischer <julian@whistle.com>, arch@freebsd.org Subject: Re: Threads stuff Message-ID: <199911281721.JAA45015@apollo.backplane.com> References: <Pine.BSF.4.10.9911271542410.544-100000@current1.whistle.com> <3840B1EC.4614AAF0@vigrid.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:>
:> I don't know what Danthiks but this has alot more involvement or the
:> kernel and boundary crossing than I was envisionning.
:
:I think it's basically right. If you saw the diagram at different
:stages, it would be easier to see.
:
: o A thread blocks in kernel, the KSE is saved, a new KSE is allocated,
: and an upcall is made to the scheduler with a unique KSE ID provided
: to identify the now blocked thread.
:
: o Scheduler receives notification of a thread blocking, tags the
: currently running thread with the KSE ID, chooses a new thread
: to run, switches to the new thread, and makes a system call to
: schedule a signal/upcall when the new threads quantum expires.
:
: o A KSE is woken up in the kernel.
:
: o Scheduler receives notification of a thread unblocking (finishing?)
: in the kernel.
:
: o At the request of the scheduler, the kernel schedules a timeout for
: the new quantum and resumes the now unblocked thread.
:
:> I will try make aset of diagrams that outline the state of various
:> objects related to a thread at various stages..
:
:How about an animated GIF or MOV? ;-)
:
:Dan Eischen
:eischen@vigrid.com
Advantages:
* Kernel stack sharing by multiple threads in the non-blocking case,
which can lead to better memory and cache resource utilization.
* An infinite number of threads can be created as long as they do not
all need a kernel stack simultaniously.
* Kernel management issues are complex, but not terribly so. For
example, you have a thousand threads and they all suddenly decide
to block inside the kernel - the kernel needs a mechanism to
disallow kernel entry (setup the thread to force a restart of the
system call and then switch threads because no KSE resources are
available).
Problems:
* You have to manage dynamic allocation and deallocation of KSE's
(verses simply preassigning one per thread).
* You have non-deterministic resource utilization for an application.
The worst case KSE useage is going to be one per thread, but the
userland may well allow thousands of threads to be allocated first
and then realize only later that it does not have the kernel
resources to block in all of them at once, leading to
non-deterministic blockages.
* You compound the management issues within the userland scheduler
itself because not only can the userland scheduler switch between
threads, now so can the kernel when it decides it must block.
(this is an argument to have a kernel entry point to schedule a
thread no matter what).
* The userland scheduler has no concept of certain types of blockages,
such as VM faults, and cannot schedule around such things itself.
This can lead to non-deterministic operation.
* The userland scheduler must deal with scheduling the N cpu case
itself - this is something more suitable to the kernel because the
userland scheduler has no knowledge of other unrelated
processes/threads running in the system. This means that
if the userland scheduler is trying to switch or schedule threads
without making a system call, the whole mess becomes much more
complex when the kernel winds up having to manage the same
threads itself.
This isn't to say that the userland scheduler cannot choose a
loosely (or hard) bound cpu for a thread, only that the kernel is
better able to actualy get the thread running deterministically
because the userland scheduler might end up 'stuck' in a VM
fault or something similar.
I am beginning to warm to the dynamic KSE concept but I think the only
way to avoid unnecessary complexity AND to avoid potential resource
sharing problems is to impose two requirements:
* First, that thread runnability be controlled through a system call.
Ther userland can determine when to halt and run a thread and can
even assign a cpu, but it must make a system call to actually
schedule or deschedule the thread.
* Second, KSE management. The kernel has a limited number of KSE's
available and cannot allow any one process (containing multiple
threads) to hog them.
KSE management must be dynamic on-entry. This means that when a
thread makes a system call and enters the kernel, the kernel must
make a determination in regards to the availability of KSE's. If
insufficient KSE's are available (or the process is using too many
already), the kernel must setup the thread to restart the system
call, deschedule it, and switch to another thread.
i.e. there is always a 'current KSE' for a running thread, but if
it is the last one and the system call MIGHT block, the kernel
cannot afford to allow the thread to block in the one remaining
KSE and must deschedule the thread until KSEs are available to
handle the system call.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199911281721.JAA45015>