From owner-freebsd-hackers Fri Jan 22 03:46:25 1999 Return-Path: Received: (from majordom@localhost) by hub.freebsd.org (8.8.8/8.8.8) id DAA29028 for freebsd-hackers-outgoing; Fri, 22 Jan 1999 03:46:25 -0800 (PST) (envelope-from owner-freebsd-hackers@FreeBSD.ORG) Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (8.8.8/8.8.8) with ESMTP id DAA29023 for ; Fri, 22 Jan 1999 03:46:24 -0800 (PST) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.2/8.9.1) id DAA52510; Fri, 22 Jan 1999 03:46:13 -0800 (PST) (envelope-from dillon) Date: Fri, 22 Jan 1999 03:46:13 -0800 (PST) From: Matthew Dillon Message-Id: <199901221146.DAA52510@apollo.backplane.com> To: "John S. Dyson" Cc: hackers@FreeBSD.ORG Subject: Re: Error in vm_fault change References: <199901220831.DAA00487@y.dyson.net> Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :> When I removed this piece of code, the machine remained useable :> with four memory hogging processes running. :> :QED. You are not running a real load. This is a perfect example of optimizing :for the wrong thing. At this level, you want to maximize system throughput, even :if the evil processes are the ones that provide that. Limiting the RSS of the :evil processes should be done explicitly in a different way. (It could be called Sure I am... I am simulating what happens when a couple of users all decide to run the 'nn' newsreader at the same time. In BEST's early days, when the machines had only 128MB of ram, the above occured about half a dozen times every day. Each time the even would cause the machine to basically stop for 10 seconds while it furiously rearranged memory. The only reason it doesn't happen now is that we throw 256MB (minimum) on the platforms and the cache is big enough to absorb nn's 10-15 MB footprint without going into a tizzy. Basically, what happens is that a few programs create a thrashing situation The proper action is to *not* start penalizing the rest of the machine! My personal opinion is that memory allocation needs some sort of scheduler - something that works somewhat like the cpu scheduler but schedules memory rather then cpu. This way the static processes are still allowed to run while the heavy-memory processes ( in a low memory situation ) get blocked more. With this in place, the pager gets a view of the overall load on the machine that is not skewed by a small number of memory hogging processes. A scheduler is not an artificial block/sleep mechanism such as the piece of code I ripped out -- it is scheduling available free memory to processes needing memory based on their memory-hogging priority. So, for example, a process that has been freeing memory recently is given a very high allocation priority whereas a process that has been allocating a lot of memory is given a lower allocation priority. The priority of the requester is run through the scheduler -- say a fractional/fair scheduler, which decides how to dole out the available free memory. Processes only block when memory is exhausted, and then only based on their scheduling priority. The root ssh login and shell still works, as do the processes that tend to be memory-neutral. A fractional/fair scheduler usually has a very smooth degredation curve, too. -Matt :-- :John | Never try to teach a pig to sing, :dyson@iquest.net | it makes one look stupid :jdyson@nc.com | and it irritates the pig. Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message