From owner-freebsd-smp Tue Nov 21 15:55:43 2000 Delivered-To: freebsd-smp@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id EE2F737B4CF; Tue, 21 Nov 2000 15:55:33 -0800 (PST) Received: from mogadishu-42.budapest.interware.hu ([195.70.52.106] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 13yNFx-0002On-00; Wed, 22 Nov 2000 00:55:21 +0100 Message-ID: <3A1B0B64.6D694248@elischer.org> Date: Tue, 21 Nov 2000 15:55:16 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: arch@FreeBSD.ORG Cc: smp@FreeBSD.ORG Subject: Re: Threads (KSE etc) comments References: Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-smp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Note to all on FreeBSD-SMP.. this is to ARCH, and CC'd to SMP I'll trim SMP next time around so if you are reading this in SMP and are not on 'arch'...... Daniel Eischen wrote: > > On Tue, 21 Nov 2000, Terry Lambert wrote: > > > yes, but that gives the ability to use M times as much CPU as a > > > nonthreaded process. > > > > If you won't give it to me, I'll just take it, instead, either > > by using rfork() and a shared memory segment for my global data, > > which gets me the equivalenet of a threaded environment, for all > > practical purposes, or I'll just fork() and run multiple instances > > of myself. Either way, you don't get to tell me not to compete > > as multiple processes, and I can throw a KSEG based policy out the > > window, without needing your permission to do it. Worse, there > > is additional context switch overhead introduced by this (the > > same reason Linux kernel threads are a bad idea), and everyone > > gets to pay the penalty for my application. > > I have to agree with Terry. You can't dictate what application > writers will do. If they want more CPU and they can't get it > with the threading model we provide, then they will get it > another way. Limiting PTHREAD_SCOPE_PROCESS threads to 1 KSEG > with only 1 quantum doesn't stop someone from [r]fork()'ing > to get more CPU. At this stage I'm having the following thoughts.. 1/ I think we've almost talked ourselves out of the need for a KSEG. 2/ I don't want to actually say that until I've thought about it a bit.. there were some REALLY GOOD REASONS for it before, I just can't think what they were. I'll probably remember at 3AM this morning while I'm mostly asleep. My remaining use for KSEGs is for housekeeping: 3/ My KSEG thought was: The user can take as many quanta as he wants by having many KSEGs. If he uses default actions and simply fires off a lot of threads, he gets N quanta, per 'round-robin cycle, where N is the number of CPUs, because by default the threads package only fires up a max of N KSEs to run the threads on in each KSEG. The KSEG is an 'official' way for him to be more greedy. but he has to ask for it.. One KSEG gives a single timeslot per round-robin, on each CPU. but KSEs in the same KSEG don't compete with each other in the KSEG. 4/ I am not sure that #3 is completely useful these days either, I'm thinking about it. 5/ it's possible that if KSEs in the same KSEG can never be on the same CPU then they can never 'pre-empt' each other. This means that locking between the UTS instances in the same KSE can predict that locks on structures held private within the KSE can never 'block', but will always 'spin-out' in a short time (assuming the UTS is written correctly). This means that (for example) threads assigned to the same KSEG can be migrated around easily within the KSEG with very simple locking. If there are two KSE's in teh same KSEG on teh same CPU, there is always the danger that one may pre-empt the other while it holds a lock on a KSEG internal structure, and then we might deadlock. (this thought is also rather vague) Basically, KSE's are to provide concurrency across processors, where KSEGs are to provide competition in the scheduler. I see the two as being orthogonal. KSEGs may go away, or may be simply a 'virtual element' that never actually exists. but I still see the use of confining 'normal threads' to some bounded box. The fact that we give you the ability to make more boxes at you leasure shouldn't detract from the usefulness of the box. Scheduling within the box (and migration therin) needs to be fast and may happen very frequently. Moving between boxes is probably less frequent. For that reason I like having the boxes well defined in a way that ensures that they are well behaved. making each box contain only one KSE from each CPU allows as much concurrency as you can get, yet makes sure that we don't have to deal with KSEs tripping over each other. I think it will make the use of simple (tiny) spinlocks the most that will be needed for such things as process migration within the box. If you start having KSEs (there is one UTS 'incarnation' per KSE) tripping over each other's CPUs it starts to get more complicated. Having said that, I'm still thinking about whether KSEGs can go away. > Let's admit that this is what some applications will want to > do and allow them to do it within our threading model. No, > we won't do it by default, but we can provide simple hooks to > allow an application to request more "CPU reservations" > (a KSEG under the current definition). exactly We allow them to be greedy. but we make them do slightly more work for it. At least they KNOW they are being greedy. > > Note that I am only talking about scope process threads. > Scope system threads are bound to their own KSEG/KSE pair. KSEG/KSEs If A SCOPE_SYSTEM thread creates a new thread, that thread competes with it on however many KSEs it has, within it's KSEG. > > -- > Dan Eischen Was my explanation of how KSEs communicate with the UTS clear enough? It's hard to decide that for your own writing.... -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-smp" in the body of the message