From owner-freebsd-current@FreeBSD.ORG Wed Oct 15 17:16:29 2003 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4989916A4B3; Wed, 15 Oct 2003 17:16:29 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9310B43FE0; Wed, 15 Oct 2003 17:16:28 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by comcast.net (rwcrmhc13) with ESMTP id <2003101600162701500op6bie>; Thu, 16 Oct 2003 00:16:28 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA12161; Wed, 15 Oct 2003 17:16:26 -0700 (PDT) Date: Wed, 15 Oct 2003 17:16:25 -0700 (PDT) From: Julian Elischer To: deischen@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Jeff Roberson cc: FreeBSD current users Subject: Re: More ULE bugs fixed. X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Oct 2003 00:16:29 -0000 On Wed, 15 Oct 2003, Daniel Eischen wrote: > On Wed, 15 Oct 2003, Jeff Roberson wrote: > > > I fixed two bugs that were exposed due to more of the kernel running > > outside of Giant. ULE had some issues with priority propagation that > > stopped it from working very well. > > > > Things should be much improved. Feedback, as always, is welcome. I'd > > like to look into making this the default scheduler for 5.2 if things > > start looking up. I hope that scares you all into using it more. :-) > > Before you do that, can you look into changing the scheduler > interfaces to address David Xu's concern with it being > suboptimal for KSE processes? > There is also some work that I'd like to get done re: cleaning up the scheduler interface a bit.. I know that Jeff and I have doiscussed this before but it was a long time ago, and I've forgotten a lot and also learned a bit since then.. Here's my logic on the matter: Any process has a number (fixed or variable) of kernel entities tghat can be scheduled. In KSE (gotta get a better name) there are a variable number of them. In libthr they are 1:1. I would postulate that the action of scheduling these items in a fair way is up to the scheduler. I had a very crude fairness module added to the BSD4.4 scheduler but I think that fairness is a property of the scheduler and not of the threading package. If the scheduler doesn't care if threads are scheduled fairly than it can just schedule all threads equally. I would say that the ksegrp in question (which represents a rough unit of 'fairness'), should make a call to the scheduler on creation specifying the required concurrancy. At the moment KSE-M:N based ksegrps would specify N = NCPU, and THR based ksegrps would specify N = NTHREADS. KSE-1:1 runs with a KSEGRP with a concurrancy of 1 per thread. (I still think that THR should allocate a KSEGRP per thread not a KSE but it's not critical.) Basically What I'm saying is that each scheduler should taka a concurrency setting for each KSEGRP and how it implements it is hidden from higher layers. The current 4.4 scheduler would implement it using KSEs and the existitng code but other shcedulers may chose to implement it in different manners. I think the top layer API calls for the scheduler should be: setrunnable(thread) choosethread() sched_clocktick() sched_set_concurrancy() (plus all the other 'entrypoints') I think that the scheduler needs to be in control of scheduling threads because there is too much inside information needed for it to be done properly by an outside entity. For example if the scheduler is not a priority based scheduler then an outside entity can not know how to juggle which thread should be run next if there is a choice of which to do.. this would mean that each scheduler would neeed its own module to do this juggling instead of having a separate module to do it.. it makes the job of the scheduler more difficult, but in fact it has to be so, because true posix process-scope threads require that the scheduler do this work. a thread is made runnable (with a unix priority) the scheduler needs to look at this thread in the context of all the other threads from this process, the current concurrency rule for that ksegrp and the other runnable threads, and adjust things so that: 1/ the new thread is run some time 2/ the ksegrp doesn't get TOO MUCH cpu, possibly punishing other threads in the group to compensate.. This is all up for discussion, but it's my current thinking. Julian