From owner-cvs-all@FreeBSD.ORG Wed Oct 3 03:11:17 2007 Return-Path: Delivered-To: cvs-all@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A2E3816A419; Wed, 3 Oct 2007 03:11:17 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail34.syd.optusnet.com.au (mail34.syd.optusnet.com.au [211.29.133.218]) by mx1.freebsd.org (Postfix) with ESMTP id 3F95613C474; Wed, 3 Oct 2007 03:11:17 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from c220-239-235-248.carlnfd3.nsw.optusnet.com.au (c220-239-235-248.carlnfd3.nsw.optusnet.com.au [220.239.235.248]) by mail34.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id l933B1od015295 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 3 Oct 2007 13:11:01 +1000 Date: Wed, 3 Oct 2007 13:11:01 +1000 (EST) From: Bruce Evans X-X-Sender: bde@delplex.bde.org To: Jeff Roberson In-Reply-To: <20071002133334.W594@10.0.0.1> Message-ID: <20071003123512.W14348@delplex.bde.org> References: <20071001145257.EC9FC4500F@ptavv.es.net> <20071002133623.X40629@besplex.bde.org> <20071001232743.Q539@10.0.0.1> <20071002213829.F12287@delplex.bde.org> <20071002133334.W594@10.0.0.1> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: cvs-all@FreeBSD.org, src-committers@FreeBSD.org, cvs-src@FreeBSD.org, Jeff Roberson , Garance A Drosehn , Ben Kaduk , Bruce Evans Subject: Re: cvs commit: src/sys/kern sched_ule.c X-BeenThere: cvs-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: CVS commit messages for the entire tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 03 Oct 2007 03:11:17 -0000 On Tue, 2 Oct 2007, Jeff Roberson wrote: > On Tue, 2 Oct 2007, Bruce Evans wrote: > Sorry I don't have time for a point by point on this one. Thank you for your > interesting analysis. From this I'm taking away a couple of things: > > 1) I've noticed that ULE relied on PREEMPTION for a long time and lost the > NEEDRESCHED setting in cases where it doesn't set owepreempt. Restoring this > should improve some of the !PREEMPTION behavior and perhaps even > responsiveness in your nice tests. Not the problem here, since I always use PREEMPTION for UP. > 2) I need to try running with hz = 100 and see if there are some scaling > problems. I have heard reports that ULE scales better than 4BSD up to higher > hz values but I haven't investigated this much. It should work with lower as > well. Everything important to relative priorites and time slice allotment > runs off of stathz. Again, not the problem here, since I tested with both hz = 100 and hz = 1000. Also stathz = 100 and stathz = 128. I haven't tried huge hz lately. > 3) The code which adjusts priorities for fork may need some more fine > tuning. ULE agressively penalizes parents for forking expensive children. > This helps us learn that make should not create interactive children for > example. I checked what 5.2 and ~5.2 are doing: - fork: both just keep the parent's estcpu - exit: 5.2 adds the child's estcpu to the parent's, same as in 4.x (?), so that the parent's priority wants to be exponential in the number of children even of the children don't do anything (except when the initial estcpu is 0, doubling it makes no difference). So I don't understand why 5.2 seems to be able to repeatedly run acroread with no penalty. - exit: ~5.2 sets the parent's estcpu to the minimum of the parent and the child estcpu, so that parents are only penalized for expensive children. Parents aren't penalized enough if they have lots of not-so-expensive children, but with the estcpu clamp in 5.2 there is probably nothing better. However, ~5.2 doesn't clamp estcpu, so it could do better. ~5.2 has to do something better than add the estcpu's, since with true exponential growth the estcpu quicky reaches "infinity", and ~5.2 has dynamic scaling of estcpu to priorities which would give an infinite penalty for infinite estcpu, and with infinite estcpu taking infinitely long to decay, the infinitely penalized perant would never run. Note that 4BSD was broken by the infinities in most versions of FreeBSD-2 and FreeBSD-3. The bug was introduced early in FreeBSD-2 and later picked up by NetBSD. There were mysterious hangs, and only the infinities being not quite infinite allowed parents to run again. estcpu is an int[32_t] variable, so repeated doublings starting at value 1 eventually overflow to -0x80000000 and then to 0. Usually the priority becomes too high to run before 31 or 32 doublings cause overflow, so the overflow bugs don't help. Instead, estcpu reaches a huge value which takes only seconds or minutes to decay to a value which gives a small enough priority to run, or the runq becomes empty except for processes with infinite estcpu so those processes can run. > 4) I don't think you're losing interrupts when you ctrl+c. It's just taking > too long for the interrupted task to run. ctrl+z takes effect immediately > when the signal is delivered. This may be related to hz = 100 or running > without preemption. I am not able to reproduce this problem with a standard > GENERIC kernel + ULE. ^C is printed by the tty driver, so tty (keyboard) interrupts certainly work. IPIs probably work too. Oops, this is UP so there are no IPIs. So the problem is just the ^C signal never being delivered to the looping process, and it is surpising that ^Z is delivered. The interrupted task _is_ running (it is looping) and delivery would consist mainly of killing it (I forget if the shell sees the ^C before the child is killed). This is not related to PREEMPTION since I always use PREEMPTION for UP. It seems to be related to hz = 100. My kernel is similar to GENERIC in this area except for HZ and leaving out ADAPTIVE_GIANT. Bruce