From owner-freebsd-questions Mon Sep 23 2: 1:49 2002 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.FreeBSD.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6FFF337B401 for ; Mon, 23 Sep 2002 02:01:46 -0700 (PDT) Received: from smtp.infracaninophile.co.uk (happy-idiot-talk.infracaninophile.co.uk [81.2.69.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6444943E6E for ; Mon, 23 Sep 2002 02:01:45 -0700 (PDT) (envelope-from m.seaman@infracaninophile.co.uk) Received: from happy-idiot-talk.infracaninophile.co.uk ([IPv6:::1]) by smtp.infracaninophile.co.uk (8.12.6/8.12.6) with ESMTP id g8N91WC0005264; Mon, 23 Sep 2002 10:01:32 +0100 (BST) (envelope-from matthew@happy-idiot-talk.infracaninophile.co.uk) Received: (from matthew@localhost) by happy-idiot-talk.infracaninophile.co.uk (8.12.6/8.12.6/Submit) id g8N91Qej005263; Mon, 23 Sep 2002 10:01:26 +0100 (BST) Date: Mon, 23 Sep 2002 10:01:26 +0100 From: Matthew Seaman To: Duncan Anker Cc: freebsd-questions@FreeBSD.ORG Subject: Re: Virtual memory allocations and signal 11 core dumps Message-ID: <20020923090126.GA4587@happy-idiot-talk.infracaninophi> References: <1032743700.20802.44.camel@duncan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1032743700.20802.44.camel@duncan> User-Agent: Mutt/1.5.1i X-Spam-Status: No, hits=-13.9 required=5.0 tests=IN_REP_TO,QUOTED_EMAIL_TEXT,REFERENCES,SPAM_PHRASE_03_05, USER_AGENT,USER_AGENT_MUTT version=2.41 Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Mon, Sep 23, 2002 at 11:14:59AM +1000, Duncan Anker wrote: > Irrespective of how much memory you really have, you have a virtual > memory size of 4Gb (maximum imposed by the 32-bit architecture). Of > this, the kernel will use 1Gb leaving 3Gb for user processes. Each > process by default is granted 128Mb data size up to a 512Mb limit. This > limit can be raised up to just under 2Gb by setting MAXDSIZ (due to the > use of a signed int, I suspect). > Given this information, and I would appreciate corrections if I have any > of it wrong, my question then becomes this - if I have, for example, > 512Mb of real memory and 2Gb of swap space then that would give me a > maximum 2.5Gb (depending on how the swapper handles this). What happens > once this maximum is used up, since the system has a theoretical maximum > of another 1.5Gb? If the system actually tries to use the space > allocated through the over-committing strategy is the net result going > to be that dreaded signal 11? When you run short of memory, there are two mechanisms that come into play. The first is pretty obvious: functions that attempt to allocate more memory to a process will fail with ENOMEM errors. Much of the time this will result in the process that attempted to allocate the memory quitting. No SIGSEGV (11) involved. Some badly written processes may fail to check the return value from the memory allocation and attempt to use a zero or a garbage pointer, which will result in SIGSEGV or SIGBUS, but that would be extremely unlikely in a well written piece of software like apache. The second is due to the kernel. It may kill off processes in an attempt to free sufficient memory to keep going --- that would be a SIGTERM (15) or a SIGKILL (9). It will choose the process to kill by some criterion to do with the amount of memory and the CPU time used. Because the system has no way of choosing the processes that aren't vital to the intended function of the system, this situation effectively means the machine is out of action. If memory still remains tight, eventually some vital process will be unable to run, and the system will hang up or crash. > For that matter, even if one does have 4Gb available, whether real or > swap, it seems entirely possible that one could run several large > processes and go over this limit. Would this cause a crash? 4Gb memory is the maximum amount of memory addressable using 32bit pointers: it's simply not possible to generate a bit pattern in the pointer variable that corresponds to an address outside that range. On FreeBSD, having several large processes corresponds entirely to the normal "running out of memory" scenario above, even if there is more than 4Gb physical ram+swap in the machine. (It's actually quite difficult to run FreeBSD in this situation --- you have to set the MAXMEM option in the kernel config or the hw.physmem sysctl in the loader --- see /usr/src/sys/boot/common/help.common --- and tune some other variables). You might think that if you have much more than 4Gb physical memory it should be possible to have a number of processes each seeing their own separate 4Gb addressable space, but mapping onto different sections of real memory. That is possible on some OSes, and it has been discussed on freebsd-hackers, but as far as I know, no one has made available any sort of implementation of it. Note: out of all the processors FreeBSD supports, the above applies only to the IA32 processor family --- the other processors supported by or soon to be supported by FreeBSD (Alpha, Ultrasparc, IA64) all have 64 bit memory addressing, which shouldn't run into this sort of address space problem for a few years yet. > A more practical question might be "Should I ensure that the number of > precesses running won't use more than my total available memory less the > 1Gb for the kernel?" Practically, you want to ensure that you have sufficient RAM in the machine that it doesn't start to swap during normal usage. If that's impossible, then the best approach is to acquire another machine and spread the load that way. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks Savill Way Marlow Tel: +44 1628 476614 Bucks., SL7 1TH UK To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message