From owner-freebsd-hackers Thu Jan 28 10:07:07 1999 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id KAA20142 for freebsd-hackers-outgoing; Thu, 28 Jan 1999 10:07:07 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from iquest3.iquest.net (iquest3.iquest.net [209.43.20.203]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id KAA20131 for ; Thu, 28 Jan 1999 10:07:05 -0800 (PST) (envelope-from toor@y.dyson.net) Received: (qmail 17262 invoked from network); 28 Jan 1999 18:07:01 -0000 Received: from dyson.iquest.net (HELO y.dyson.net) (198.70.144.127) by iquest3.iquest.net with SMTP; 28 Jan 1999 18:07:01 -0000 Received: (from root@localhost) by y.dyson.net (8.9.1/8.9.1) id NAA21641; Thu, 28 Jan 1999 13:07:01 -0500 (EST) Message-Id: <199901281807.NAA21641@y.dyson.net> Subject: Re: High Load cron patches - comments? In-Reply-To: <199901281748.LAA02377@home.dragondata.com> from Kevin Day at "Jan 28, 99 11:48:23 am" To: toasty@home.dragondata.com (Kevin Day) Date: Thu, 28 Jan 1999 13:07:01 -0500 (EST) Cc: dillon@apollo.backplane.com, dyson@iquest.net, wes@softweyr.com, hackers@FreeBSD.ORG From: "John S. Dyson" Reply-To: dyson@iquest.net X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Kevin Day said: > > To step back in here again... > > It's kinda interesting the discussion this generated. > > I think we're all discusssing slightly different problems, so we're coming > up with different ways to address them. > > > Here's my problem. > > Cron turned into a massive forkbomb every minute, and especially every 10 > minutes. Not only did the system nearly go dead at those points, but at > times, it took 5 minutes to catch up. > > Supposed you have to run 60 jobs per minute, and they all take around a > second to execute. If you run them one second at a time, you're likely to > get done with them every minute. If you try to run them all at once, you're > likely not to get finished after a minute, causing a backlog. > The same problem exists in other subsystems also. If you limit the rate by limiting peaks, you often (depending on app) end up with too small a limit, or still getting serious peaks. Sometimes an absolute limit is adequate, but a true rate limit would be / is a big plus. I added some stuff to the VM code and had gotten very interesting (positive) results for load/resource sharing. Brute force techniques can take too much CPU (IMO), but the scheme that I implemented has almost NO overhead. (I suspect that you also produced a low overhead implementation.) It seems that a general scheme would be nice for the system... I can see reasons for it in the VM code, fork code, and other places. I really don't know how it should be done (this is pretty much a food for thought discussion.) -- John | Never try to teach a pig to sing, dyson@iquest.net | it makes one look stupid jdyson@nc.com | and it irritates the pig. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message