From owner-freebsd-questions Sat Aug 31 02:41:28 1996 Return-Path: owner-questions Received: (from root@localhost) by freefall.freebsd.org (8.7.5/8.7.3) id CAA01975 for questions-outgoing; Sat, 31 Aug 1996 02:41:28 -0700 (PDT) Received: from mail.crl.com (mail.crl.com [165.113.1.22]) by freefall.freebsd.org (8.7.5/8.7.3) with SMTP id CAA01846 for ; Sat, 31 Aug 1996 02:38:39 -0700 (PDT) Received: from root.com (implode.root.com) by mail.crl.com with SMTP id AA29342 (5.65c/IDA-1.5 for ); Sat, 31 Aug 1996 02:37:14 -0700 Received: from localhost (localhost [127.0.0.1]) by root.com (8.7.5/8.6.5) with SMTP id CAA26208; Sat, 31 Aug 1996 02:32:48 -0700 (PDT) Message-Id: <199608310932.CAA26208@root.com> X-Authentication-Warning: implode.root.com: Host localhost [127.0.0.1] didn't use HELO protocol To: Nadav Eiron Cc: Scott Blachowicz , Eric Berenguier , questions@FreeBSD.org Subject: Re: FreeBSD 2.1.0 CRASH! In-Reply-To: Your message of "Sat, 31 Aug 1996 11:31:44 +0200." From: David Greenman Reply-To: dg@root.com Date: Sat, 31 Aug 1996 02:32:48 -0700 Sender: owner-questions@FreeBSD.org X-Loop: FreeBSD.org Precedence: bulk >that context), and its younger clone, WinNT. The point is that when you >use local replacement you can control how much swap space a process is >going to ever get (it's called VIRTPAGCNT in VMS), and thus calculate how Actually, the global parameter is "VIRTUALPAGECNT" which sets the system maximum, but there is also a per-process "paging file quota" that can limit this on an per-account basis. >much swap space you'll need before it's needed (however, you need to >manage those parameters, and many other, manually, and they are usually Actually, this is not correct. VMS does not pre-allocate page file space based on the process page file quota (or the global limit). >not set correctly as most managers don't fully understand the ver complex >paging and swapping algorithms in VMS). This way, VMS can tell you when >you have the chance of running out of paging space (I think NT does this >too, giving the message "System is running low on virtual memory") long >before it causes any ill effects (as most processes don't eat up all the >memory at once). As "The Design and Implementation of 4.4BSD" says, the >decision not to use those algorithms in BSD may cause processes to be >killed and even the system to crash when swap space is exhausted. This is more or less how FreeBSD works. The difference is 1) FreeBSD doesn't warn you like it does in VMS [this would be trivial to implement], and 2) FreeBSD looks for a process to kill when it runs out - VMS (at least v5.x) just hangs. >Bottom line is that if you used VMS you'd probably be noticing that your >pagefiles have *negative* reserved space left long before anything bad would >happen. However, if you used VMS your bank account would also show a >negative balance before you get anything done :-). "Reservable" space works like this: whenever VMS allocates space out of the pagefile, it allocates it in large chunks - typically 2048 pages so as to minimize fragmentation. The "reservable space" is the space that is left if you add up the completely unused chunks, and "free" space is the space in the completely unused chunks plus the space in the partially used chunks. Given this, it's arguable how useful the "reservable" statistic is. We actually have something sort of similar to this inside FreeBSD, it's just that we never assigned a statistic to count it - i.e. space is allocated in chunks but the unused portions can be reclaimed if necessary. VMS does indeed do localized page replacement. Back when memory was extremely expensive and in short supply on a system, it was thought that the best way to maintain good system-wide interactive performance while large batch jobs were running was to limit each process's working set via a working set quota. In order to effectively manage this quota, pages have to be sorted within a process...a localized page replacement algorithm. This isn't the most efficient way to manage resources, however. To maximize system throughput, you want to look at the usage of the pages on a global basis. If you have the memory and are willing to sacrifice some interactive performance, global page replacement is definately the way to go. We experimented with implementing a soft RSS limiting mechanism in FreeBSD (and in fact we still support it), but the challenge has always been how to determine what to trim from a process - especially when parts of it are shared among many processes. My conclusion is that I don't think it can ever be made to work very well. -DG David Greenman Core-team/Principal Architect, The FreeBSD Project