From owner-freebsd-arch Sun Jan 26 18:12:47 2003 Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6B29D37B401; Sun, 26 Jan 2003 18:12:45 -0800 (PST) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 701F343E4A; Sun, 26 Jan 2003 18:12:44 -0800 (PST) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h0R2CX130388; Sun, 26 Jan 2003 21:12:33 -0500 (EST) (envelope-from jroberson@chesapeake.net) Date: Sun, 26 Jan 2003 21:12:33 -0500 (EST) From: Jeff Roberson To: Matthew Dillon Cc: Julian Elischer , Steve Kargl , Robert Watson , Gary Jennejohn , Subject: Re: New scheduler - ULE performance w/ cpu stealing In-Reply-To: <200301262045.h0QKjsVc067308@apollo.backplane.com> Message-ID: <20030126210706.V64928-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Sun, 26 Jan 2003, Matthew Dillon wrote: > I added cpu-stealing code to ULE and it seems to have made a > huge difference. Perhaps you missed something in your local patch > set. Here are the results: > > /usr/bin/time make -j 8 buildworld > > 4BSD - Original scheduler > 2414.84 real 2648.92 user 758.28 sys > 2399.05 real 2647.84 user 757.78 sys > > ULE - > 3435.42 real 2500.73 user 581.20 sys > 3343.95 real 2501.86 user 581.67 sys > > ULE - With cpu stealing code > 2489.76 real 2610.33 user 659.74 sys <<<<<<<<<<<<<<<<<< > > > Next I am going to try removing sched_pickcpu(), but I don't expect > it to improve things (nor do I expect the removal to make things > worse). So, yes, there is some marginal improvement. Not as much as I would expect to see with the addition of cpu affinity. I think perhaps this may be partially due to the amount of time we spend blocked on giant now. I'm going to try some runs with make -s. In the mean time, I have a very similar version of this that I will commit. I also reworked some of the load stuff so it is slightly more representative of real load. I still think the run queue depth is not the right metric. This mechanism is sufficient for compiles but it ignores problems in other situations. Apache presents one that is readily understandable and easily solvable. Since it uses a forked model there may be many apache processes that live for a long time that are bound to cpus. They need to be evenly spread across all cpus. Using only the pull model you may have 5 apaches on one cpu and 1 on the remaining one which will be sufficient for it not to trigger a pull. So to handle this scenario I'm writing a timeout that runs every second or so and balances cpus this way. So, in short, it looks like pull is good for many short lived processes. Push is good for balancing long term load. I believe the combination of the two is required. Cheers, Jeff To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message