From owner-freebsd-current Thu Apr 15 12:41:46 1999 Delivered-To: freebsd-current@freebsd.org Received: from apollo.backplane.com (apollo.backplane.com [209.157.86.2]) by hub.freebsd.org (Postfix) with ESMTP id C061C14E6E for ; Thu, 15 Apr 1999 12:41:42 -0700 (PDT) (envelope-from dillon@apollo.backplane.com) Received: (from dillon@localhost) by apollo.backplane.com (8.9.3/8.9.1) id MAA48113; Thu, 15 Apr 1999 12:38:06 -0700 (PDT) (envelope-from dillon) Date: Thu, 15 Apr 1999 12:38:06 -0700 (PDT) From: Matthew Dillon Message-Id: <199904151938.MAA48113@apollo.backplane.com> To: Mikhail Teterin Cc: current@FreeBSD.ORG Subject: Re: swap on Irix (overcommiting, etc.) References: <199904151838.OAA98565@misha.cisco.com> Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :Mmm, that's how they use(d) it then... But can we have a way for an :admin to control the amount of overcommited memory? Ranging from 0 -- no :overcommitment to the entire addressable space less the phisical storage :(the current situation)... And this is different from the per-user class :limit. I think the people who keep on rooting about this 'overcommit' thing don't really understand the complexity of VM management or the total absurdity of the 'overcommit' argument. The OS *CANNOT* tell how much actual memory a process is going to use. This concept of 'overcommit' does not work in reality ... if you calculate the absolute worst-case swap required for a set of processes, the resulting number is INSANE. I will repeat that: It is an INSANE number. It is so insane that it is completely unuseable for any sort of measurement, much less any sort of requirement. Here are some simple examples: A process allocates and touches 100MB of memory. Ok, so there you have a process taking up 100MB of swappable memory. Now the process forks. Now it forks 10 times. How much real memory do these 10 processes need? The aggregate could be anything from 100MB ( all shared ) to 1GB ( all the children write to all their memory, forcing a copy-on-write for all theiir pages ). The OS can't tell in advance. This does not mean that the program might touch all of that memory... programs tend to have relatively static memory use characteristics. To assume that it might to touch all of the memory in each child is absurd. Example #2: A process mmap()'s a 100MB file MAP_PRIVATE. If the process does not write to any of the associates pages, this 100MB mapping requires *NO* swap and *NO* real memory - it is backed by the original file. If the process touches all of the pages, this mapping eats 100MB of swap+real. Now lets say you have 10 of these processes. To guarentee no overcommit you would need 1GB of swap. The actual amount of swap likely to be allocated by the system is almost certainly going to be less then 10% that figure. In many cases, less then 1% of that figure. Example #3: You run a program. It allocates nothing. How much real+swap does this program require? It could be as little as 16K and as much as the entire Data+BSS area of the program, depending on what it touches. Now combine it all together. To be blunt, the 'total VM' used by a system can run into the gigabytes while the actual real memory + swap allocation is 1/10 of that... or better. It all depends on what you are runing. The real kicker, however, is performance. What use is guarenteeing no overcommit ( and allocating gigabytes of swap space on your disks ) when the machine's performance dies before it uses up even 1/5 of that space? Just because you are guarenteeing sufficient swap doesn't mean that you are guarenteeing that the machine will remain useable. The general rule of thumb, "allocate twice as much swap as real memory", is as much related to the average machine's ability to perform as it is to the amount of swap the machine might actually need. Generally speaking, only a complete idiot focuses on guarenteeing memory without looking at the performance of the machine at the performance degradation that would occur if that much swap were actually used. ( read: machine comes to a nearly complete stop trying to page ). If some of you are wondering why some of us are saying this guarenteeing of memory is a crap argument, it's because it *IS* a crap argument. -Matt Matthew Dillon To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message