Date: Fri, 27 Apr 2001 07:54:42 -0400 (EDT) From: Daniel Eischen <eischen@vigrid.com> To: Julian Elischer <julian@elischer.org> Cc: Arch@freebsd.org, alfred@freebsd.org, Robert Watson <rwatson@freebsd.org> Subject: Re: KSE threading support (first parts) Message-ID: <Pine.SUN.3.91.1010427072216.22918A-100000@pcnet1.pcnet.com> In-Reply-To: <3AE90567.CA50293E@elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 26 Apr 2001, Julian Elischer wrote: > Daniel Eischen wrote: > > Right, like I've said before, it's easier just to combine the KSEG and KSE > > into one entity and forget about fair scheduling. Limit the number of > > "combined KSE/KSEGs" to the number of processors (scope system threads > > still get their own "combined KSE/KSEG" to satisfy POSIX). The common > > case is a single processor system anyways, and in that case it doesn't > > make sense to have more than 1 KSE in a KSEG. > > By grouping KSEs ito an equivalence group, I give the system enough > information to allow it to schedule resuming KSECs on an 'equivalent' > but idle KSE. i.e. a syscall may be initiated on one kse and completed on > another as long as they are in the same KSEG. I think we need (in the UTS) to have curthread->curkse. Every time the thread changes KSEs, the UTS has to setup change this. We'll also have curkse that will have curkse->curthread. The curkse is the thing pointed at by the ldt (%fs). > The UTS doesn't really have to think to much about this except to > know that the paralelism in a KSEG is equal to the lesser of > the number of KSEs and the number of processors. (having more KSEs > than processors is completely pointless, and I happen to think it > should not be allowed.) Right, I thought we all agreed that more KSEs than CPUs is pointless. The problem is how does the UTS schedule threads to a set of KSEs within the same KSEG. There will be one scheduling queue for the main process (in the UTS) and this is where we'll potentially have multiple KSEs available. Scope system threads get their own KSEG/KSE, and the remaining threads run in the main process. If KSEs have processor affinity, then the UTS should try to keep threads running on the same KSE and still load balance to ensure each threads gets its fair share of processor time. > I tried to get away from having a KSEG but it ended up getting > more complicated again. > > Most threaded applications would have one KSEG and N KSEs > (N==num-processors) > or ONE KSEG and one KSE (which is still useful as you have N KSECs). > > Few apps would have more than one KSEG and hardly any would have > more than 2. Scope system threads get their own KSEG. I'm sure there are a lot of applications out there that create more than 1 or 2 scope system threads. > > In a multiprocessor system, if you have an extra quantum, who really > > cares? If someone really does care, then it can be a kernel tunable > > or resource limit. > > > > I just don't see any benefit for the added complexity. It certainly > > is easier for the UTS with a combined KSE/KSEG. > > but I think it will be simpler WITH it.. > Treat the KSEs in a KSEG as interchangable. threads that go into > the system on one may be reported to have completed their syscall > on another. But I don't think it _is_ easier. Whatever. I can't be too concerned with this right now. It's more important that we get _something_ that's better than the current threads library and the NxN Linuxthreads model. It can always be improved later. > > For the most part. But if the target thread is running in one > > of the other KSEs for that KSEG, then it will still require an > > assist from the kernel. > > Why? Surely it is to be considered to be processing another signal. > note teh signal, and it should pick it up when it's completed the one > its doing.. You've got 2 KSEs (A and B) running. Another process sends a SIGUSR1 to this process. The next KSE (A) to cross the kernel->user boundary gets an upcall to notify it that it got SIGUSR1. The UTS decides that it should be handled by the thread running on the other KSE (B). > > > This is tricky... when a KSE returns to userland it is running > > > NO threads. All threaded syscalls return to userland in the 'suspended' > > > state, so that the UTS can decide what to run. > > > > This is only when syscalls or when you need to notify the KSE of > > special events (signals, interruptions from other KSEs, etc). > > Normally, a syscall that doesn't block just returns without > > any upcall. > > well, I don't know that this is true.. the thread could starve other threads > by doing only non-blocking syscalls. Ihad thought that all syscalls would return > via upcalls to allow the UTS to decide whether to pre-empt them. We don't need fine-grained resolution like that. And I wouldn't want to run the UTS scheduler every time that a system call was made - yech! At a minimum, you'd have to have an extra context switch for each syscall. You only need to make an upcall when the KSE is preempted to run another process (or KSE). This limits the UTS thread quantum to a multiple of the kernels quantum. We could also continue to use a setitimer type of mechanism with a fixed interval. Scope system threads don't need to be notified of timing signals since there are no other threads to be run. It's only the KSEs within the main process that need some sort of timing signal. > > > All syscalls return via > > > an upcall to the UTS (actually the original newkse() call returns > > > infinitly many times.. that is how the upcall is achieved). The return > > > values, error returns and data movements have been made to the appropriate > > > memory locations.. It's as if the thread did a 'yield()' immediatly > > > after returning from a normal syscall.. > > > So we can be sure that THIS KSE isn't running the interrupt thread. > > > > Running threads can cause synchronous signals, so it's quite possible > > the running thread generated the signal. The KSE in which the thread > > was running would then get the notification. > > the KSE would, but when the upcall is made, the previous running thread > is made to lookas if it had yielded.. the KSE is running but the thread > is 'suspended'. OK, I just wanted to make sure we were on the same page. To me, 'suspended' is preempted. > > > If a KSEG is not running because it had no work, then > > > yes, you need to wake up one of its KSEs to handle the signal. > > > > Yeah, but I was thinking more along the lines of interrupting a > > currently running KSE. > > If you are returning from the kernel, all threads are effectively > suspeended and equivalent. The KSE is always 'idle' when returning > to the UTS. The UTS then decides what to run next.. (usually the > thread that was just active, but not always.) KSEs have upcalls, not KSEGs. If you return from the kernel in one KSE, that does not mean that threads are suspended and not running in one of the other KSEs within that KSEG (and you can certainly have scope system threads in their own KSEG/KSE pair that are running also). I'm thinking multiprocessor. -- Dan Eischen To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.SUN.3.91.1010427072216.22918A-100000>