Date: Thu, 15 Apr 1999 12:38:06 -0700 (PDT) From: Matthew Dillon <dillon@apollo.backplane.com> To: Mikhail Teterin <mi@misha.cisco.com> Cc: current@FreeBSD.ORG Subject: Re: swap on Irix (overcommiting, etc.) Message-ID: <199904151938.MAA48113@apollo.backplane.com> References: <199904151838.OAA98565@misha.cisco.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:Mmm, that's how they use(d) it then... But can we have a way for an
:admin to control the amount of overcommited memory? Ranging from 0 -- no
:overcommitment to the entire addressable space less the phisical storage
:(the current situation)... And this is different from the per-user class
:limit.
I think the people who keep on rooting about this 'overcommit' thing
don't really understand the complexity of VM management or the total
absurdity of the 'overcommit' argument.
The OS *CANNOT* tell how much actual memory a process is going to use.
This concept of 'overcommit' does not work in reality ... if you calculate
the absolute worst-case swap required for a set of processes, the resulting
number is INSANE. I will repeat that: It is an INSANE number. It is so
insane that it is completely unuseable for any sort of measurement, much
less any sort of requirement.
Here are some simple examples: A process allocates and touches 100MB of
memory. Ok, so there you have a process taking up 100MB of swappable
memory. Now the process forks. Now it forks 10 times. How much real
memory do these 10 processes need? The aggregate could be anything
from 100MB ( all shared ) to 1GB ( all the children write to all their
memory, forcing a copy-on-write for all theiir pages ). The OS can't tell
in advance. This does not mean that the program might touch all of that
memory... programs tend to have relatively static memory use
characteristics. To assume that it might to touch all of the memory in
each child is absurd.
Example #2: A process mmap()'s a 100MB file MAP_PRIVATE. If the process
does not write to any of the associates pages, this 100MB mapping requires
*NO* swap and *NO* real memory - it is backed by the original file.
If the process touches all of the pages, this mapping eats 100MB of
swap+real. Now lets say you have 10 of these processes. To guarentee
no overcommit you would need 1GB of swap. The actual amount of swap
likely to be allocated by the system is almost certainly going to be
less then 10% that figure. In many cases, less then 1% of that figure.
Example #3: You run a program. It allocates nothing. How much
real+swap does this program require? It could be as little as 16K and
as much as the entire Data+BSS area of the program, depending on what
it touches.
Now combine it all together.
To be blunt, the 'total VM' used by a system can run into the gigabytes
while the actual real memory + swap allocation is 1/10 of that... or
better. It all depends on what you are runing.
The real kicker, however, is performance. What use is guarenteeing
no overcommit ( and allocating gigabytes of swap space on your disks )
when the machine's performance dies before it uses up even 1/5 of that
space? Just because you are guarenteeing sufficient swap doesn't mean
that you are guarenteeing that the machine will remain useable.
The general rule of thumb, "allocate twice as much swap as real memory",
is as much related to the average machine's ability to perform as it is
to the amount of swap the machine might actually need. Generally speaking,
only a complete idiot focuses on guarenteeing memory without looking at
the performance of the machine at the performance degradation that would
occur if that much swap were actually used. ( read: machine comes to
a nearly complete stop trying to page ).
If some of you are wondering why some of us are saying this guarenteeing
of memory is a crap argument, it's because it *IS* a crap argument.
-Matt
Matthew Dillon
<dillon@backplane.com>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199904151938.MAA48113>