From owner-freebsd-arch Sun Nov 28 9:22:20 1999 Delivered-To: freebsd-arch@freebsd.org Received: from ns1.yes.no (ns1.yes.no [195.204.136.10]) by hub.freebsd.org (Postfix) with ESMTP id DBD02150A3 for ; Sun, 28 Nov 1999 09:22:18 -0800 (PST) (envelope-from eivind@bitbox.follo.net) Received: from bitbox.follo.net (bitbox.follo.net [195.204.143.218]) by ns1.yes.no (8.9.3/8.9.3) with ESMTP id SAA24679 for ; Sun, 28 Nov 1999 18:22:17 +0100 (CET) Received: (from eivind@localhost) by bitbox.follo.net (8.8.8/8.8.6) id SAA58227 for freebsd-arch@freebsd.org; Sun, 28 Nov 1999 18:22:17 +0100 (MET) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by hub.freebsd.org (Postfix) with ESMTP id DA05815724 for ; Sun, 28 Nov 1999 09:22:00 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id JAA45015; Sun, 28 Nov 1999 09:21:58 -0800 (PST) (envelope-from dillon) Date: Sun, 28 Nov 1999 09:21:58 -0800 (PST) From: Matthew Dillon Message-Id: <199911281721.JAA45015@apollo.backplane.com> To: "Daniel M. Eischen" Cc: Julian Elischer , arch@freebsd.org Subject: Re: Threads stuff References: <3840B1EC.4614AAF0@vigrid.com> Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> :> I don't know what Danthiks but this has alot more involvement or the :> kernel and boundary crossing than I was envisionning. : :I think it's basically right. If you saw the diagram at different :stages, it would be easier to see. : : o A thread blocks in kernel, the KSE is saved, a new KSE is allocated, : and an upcall is made to the scheduler with a unique KSE ID provided : to identify the now blocked thread. : : o Scheduler receives notification of a thread blocking, tags the : currently running thread with the KSE ID, chooses a new thread : to run, switches to the new thread, and makes a system call to : schedule a signal/upcall when the new threads quantum expires. : : o A KSE is woken up in the kernel. : : o Scheduler receives notification of a thread unblocking (finishing?) : in the kernel. : : o At the request of the scheduler, the kernel schedules a timeout for : the new quantum and resumes the now unblocked thread. : :> I will try make aset of diagrams that outline the state of various :> objects related to a thread at various stages.. : :How about an animated GIF or MOV? ;-) : :Dan Eischen :eischen@vigrid.com Advantages: * Kernel stack sharing by multiple threads in the non-blocking case, which can lead to better memory and cache resource utilization. * An infinite number of threads can be created as long as they do not all need a kernel stack simultaniously. * Kernel management issues are complex, but not terribly so. For example, you have a thousand threads and they all suddenly decide to block inside the kernel - the kernel needs a mechanism to disallow kernel entry (setup the thread to force a restart of the system call and then switch threads because no KSE resources are available). Problems: * You have to manage dynamic allocation and deallocation of KSE's (verses simply preassigning one per thread). * You have non-deterministic resource utilization for an application. The worst case KSE useage is going to be one per thread, but the userland may well allow thousands of threads to be allocated first and then realize only later that it does not have the kernel resources to block in all of them at once, leading to non-deterministic blockages. * You compound the management issues within the userland scheduler itself because not only can the userland scheduler switch between threads, now so can the kernel when it decides it must block. (this is an argument to have a kernel entry point to schedule a thread no matter what). * The userland scheduler has no concept of certain types of blockages, such as VM faults, and cannot schedule around such things itself. This can lead to non-deterministic operation. * The userland scheduler must deal with scheduling the N cpu case itself - this is something more suitable to the kernel because the userland scheduler has no knowledge of other unrelated processes/threads running in the system. This means that if the userland scheduler is trying to switch or schedule threads without making a system call, the whole mess becomes much more complex when the kernel winds up having to manage the same threads itself. This isn't to say that the userland scheduler cannot choose a loosely (or hard) bound cpu for a thread, only that the kernel is better able to actualy get the thread running deterministically because the userland scheduler might end up 'stuck' in a VM fault or something similar. I am beginning to warm to the dynamic KSE concept but I think the only way to avoid unnecessary complexity AND to avoid potential resource sharing problems is to impose two requirements: * First, that thread runnability be controlled through a system call. Ther userland can determine when to halt and run a thread and can even assign a cpu, but it must make a system call to actually schedule or deschedule the thread. * Second, KSE management. The kernel has a limited number of KSE's available and cannot allow any one process (containing multiple threads) to hog them. KSE management must be dynamic on-entry. This means that when a thread makes a system call and enters the kernel, the kernel must make a determination in regards to the availability of KSE's. If insufficient KSE's are available (or the process is using too many already), the kernel must setup the thread to restart the system call, deschedule it, and switch to another thread. i.e. there is always a 'current KSE' for a running thread, but if it is the last one and the system call MIGHT block, the kernel cannot afford to allow the thread to block in the one remaining KSE and must deschedule the thread until KSEs are available to handle the system call. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message