From owner-freebsd-hackers Thu Jan 28 08:11:31 1999 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id IAA05475 for freebsd-hackers-outgoing; Thu, 28 Jan 1999 08:11:31 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from iquest3.iquest.net (iquest3.iquest.net [209.43.20.203]) by hub.freebsd.org (8.8.8/8.8.8) with SMTP id IAA05470 for ; Thu, 28 Jan 1999 08:11:29 -0800 (PST) (envelope-from toor@y.dyson.net) Received: (qmail 7294 invoked from network); 28 Jan 1999 16:11:21 -0000 Received: from dyson.iquest.net (HELO y.dyson.net) (198.70.144.127) by iquest3.iquest.net with SMTP; 28 Jan 1999 16:11:21 -0000 Received: (from root@localhost) by y.dyson.net (8.9.1/8.9.1) id LAA21412; Thu, 28 Jan 1999 11:11:18 -0500 (EST) Message-Id: <199901281611.LAA21412@y.dyson.net> Subject: Re: High Load cron patches - comments? In-Reply-To: <36B06FC5.1EA65667@softweyr.com> from Wes Peters at "Jan 28, 99 07:10:13 am" To: wes@softweyr.com (Wes Peters) Date: Thu, 28 Jan 1999 11:11:18 -0500 (EST) Cc: dyson@iquest.net, dillon@apollo.backplane.com, toasty@home.dragondata.com, hackers@FreeBSD.ORG From: "John S. Dyson" Reply-To: dyson@iquest.net X-Mailer: ELM [version 2.4ME+ PL38 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Wes Peters said: > > > > I can develop some rather complete and lossless calculations that > > provide accurate and useful load indicators. In fact, I think that > > a generic load mgmt scheme might be useful. > > Especially as we start diving more into SMP and threaded applications; > which will need some effective means of throttling themselves. The > problem with Matt's comment above is he doesn't offer any useful > alternative, and couting child processes just isn't an effective means > of throttling the overall load on a machine. > It is *sometimes* appropriate to criticize, even when alternatives aren't provided. The kind of technique that I have successfully experimented with is a scheme that has two phases: A costing mechanism and a stats mechanism. The costing mechanism is a direct call from when the resource is attempted to be allocated. It checks immediately if the cost (and recent incurred costs, since the last stats run) is greater than a threshold. If that threshold is attained, then the process blocks until the stats mechanism sees that the recent peak + less recently decayed peak is less than a certain threshold. The scheme is as follows (note the lack of arrays -- this is a single resource example). See that this looks like a nearly peak responding filter with a decay. I am also using floats to throw out scale factor issues: float rate_resource_history; float rate_resource_recent; float rate_resource_threshold; (In items / sec ) #define TC .1 seconds; #define TCATTACK .50 /* Attack time is approx TC/TCATTACK */ #define TCDECAY .02 /* Decay time is approx TC/TCDECAY */ rate_resource_charge() { rate_resource_recent += 1; while ((rate_resource_recent / TC) + rate_resource_history > rate_resource_threshold) { block_on_rate_resource_history; } } rate_resource_scan_TCorso() { float new_rate_resource_history, new_rate_resource_recent, new_rate_resource_add; new_rate_resource_add = rate_resource_recent * TCATTACK; rate_resource_recent -= new_rate_resource_add; if (new_rate_resource_add > new_rate_resource_history) new_rate_resource_history = new_rate_resource_add; else new_rate_resource_history = TCDECAY * new_rate_resource_add + (1 - TCDECAY) * rate_resource_history; rate_resource_history = new_rate_resource_history; if (wakeup_needed && ((rate_resource_history + rate_resource_recent) < rate_resource_threshold)) wakeup_rate_resource_history; } } There might be some errors in the above (because this was interpreted from my VM test code), but you can see the peak sensing behavior between TC sample rate samples. There is also an average decay behavior, and there is no way that the resource can be over allocated (in a rate sense), even in a peak rate sense (esp if the TCATTACK is set to 1.0.) The attack time allows for bursty behavior, and still limits the rate. This concept can also be extended in a hierarchical fashion. Problems with this include the issue of integer truncation (which can be dealt with by scaling.) No information is lost by using a slightly slower attack time also -- which does alot to allow for some burstiness without forgetting about it. This code also doesn't loose information, so hierarchical designs might make sense. -- John | Never try to teach a pig to sing, dyson@iquest.net | it makes one look stupid jdyson@nc.com | and it irritates the pig. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message