Date: Fri, 22 Jan 1999 03:46:13 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: "John S. Dyson" <dyson@iquest.net> Cc: hackers@FreeBSD.ORG Subject: Re: Error in vm_fault change Message-ID: <199901221146.DAA52510@apollo.backplane.com> References: <199901220831.DAA00487@y.dyson.net>
index | next in thread | previous in thread | raw e-mail
:> When I removed this piece of code, the machine remained useable
:> with four memory hogging processes running.
:>
:QED. You are not running a real load. This is a perfect example of optimizing
:for the wrong thing. At this level, you want to maximize system throughput, even
:if the evil processes are the ones that provide that. Limiting the RSS of the
:evil processes should be done explicitly in a different way. (It could be called
Sure I am... I am simulating what happens when a couple of users all decide
to run the 'nn' newsreader at the same time.
In BEST's early days, when the machines had only 128MB of ram,
the above occured about half a dozen times every day. Each time the
even would cause the machine to basically stop for 10 seconds
while it furiously rearranged memory. The only reason it doesn't happen
now is that we throw 256MB (minimum) on the platforms and the cache
is big enough to absorb nn's 10-15 MB footprint without going into a
tizzy.
Basically, what happens is that a few programs create a thrashing situation
The proper action is to *not* start penalizing the rest of the machine!
My personal opinion is that memory allocation needs some sort of
scheduler - something that works somewhat like the cpu scheduler but
schedules memory rather then cpu. This way the static processes are
still allowed to run while the heavy-memory processes ( in a low memory
situation ) get blocked more. With this in place, the pager gets a
view of the overall load on the machine that is not skewed by a small
number of memory hogging processes.
A scheduler is not an artificial block/sleep mechanism such as the
piece of code I ripped out -- it is scheduling available free memory
to processes needing memory based on their memory-hogging priority.
So, for example, a process that has been freeing memory recently is
given a very high allocation priority whereas a process that has been
allocating a lot of memory is given a lower allocation priority. The
priority of the requester is run through the scheduler -- say a
fractional/fair scheduler, which decides how to dole out the available
free memory. Processes only block when memory is exhausted, and then
only based on their scheduling priority. The root ssh login and shell
still works, as do the processes that tend to be memory-neutral. A
fractional/fair scheduler usually has a very smooth degredation curve,
too.
-Matt
:--
:John | Never try to teach a pig to sing,
:dyson@iquest.net | it makes one look stupid
:jdyson@nc.com | and it irritates the pig.
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199901221146.DAA52510>