Date: Mon, 23 Sep 2002 10:01:26 +0100 From: Matthew Seaman <m.seaman@infracaninophile.co.uk> To: Duncan Anker <d.anker@au.darkbluesea.com> Cc: freebsd-questions@FreeBSD.ORG Subject: Re: Virtual memory allocations and signal 11 core dumps Message-ID: <20020923090126.GA4587@happy-idiot-talk.infracaninophi> In-Reply-To: <1032743700.20802.44.camel@duncan> References: <1032743700.20802.44.camel@duncan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Sep 23, 2002 at 11:14:59AM +1000, Duncan Anker wrote:
> Irrespective of how much memory you really have, you have a virtual
> memory size of 4Gb (maximum imposed by the 32-bit architecture). Of
> this, the kernel will use 1Gb leaving 3Gb for user processes. Each
> process by default is granted 128Mb data size up to a 512Mb limit. This
> limit can be raised up to just under 2Gb by setting MAXDSIZ (due to the
> use of a signed int, I suspect).
> Given this information, and I would appreciate corrections if I have any
> of it wrong, my question then becomes this - if I have, for example,
> 512Mb of real memory and 2Gb of swap space then that would give me a
> maximum 2.5Gb (depending on how the swapper handles this). What happens
> once this maximum is used up, since the system has a theoretical maximum
> of another 1.5Gb? If the system actually tries to use the space
> allocated through the over-committing strategy is the net result going
> to be that dreaded signal 11?
When you run short of memory, there are two mechanisms that come into
play.
The first is pretty obvious: functions that attempt to allocate more
memory to a process will fail with ENOMEM errors. Much of the time
this will result in the process that attempted to allocate the memory
quitting. No SIGSEGV (11) involved. Some badly written processes may
fail to check the return value from the memory allocation and attempt
to use a zero or a garbage pointer, which will result in SIGSEGV or
SIGBUS, but that would be extremely unlikely in a well written piece
of software like apache.
The second is due to the kernel. It may kill off processes in an
attempt to free sufficient memory to keep going --- that would be a
SIGTERM (15) or a SIGKILL (9). It will choose the process to kill by
some criterion to do with the amount of memory and the CPU time used.
Because the system has no way of choosing the processes that aren't
vital to the intended function of the system, this situation
effectively means the machine is out of action.
If memory still remains tight, eventually some vital process will be
unable to run, and the system will hang up or crash.
> For that matter, even if one does have 4Gb available, whether real or
> swap, it seems entirely possible that one could run several large
> processes and go over this limit. Would this cause a crash?
4Gb memory is the maximum amount of memory addressable using 32bit
pointers: it's simply not possible to generate a bit pattern in the
pointer variable that corresponds to an address outside that range.
On FreeBSD, having several large processes corresponds entirely to the
normal "running out of memory" scenario above, even if there is more
than 4Gb physical ram+swap in the machine. (It's actually quite
difficult to run FreeBSD in this situation --- you have to set the
MAXMEM option in the kernel config or the hw.physmem sysctl in the
loader --- see /usr/src/sys/boot/common/help.common --- and tune some
other variables).
You might think that if you have much more than 4Gb physical memory it
should be possible to have a number of processes each seeing their own
separate 4Gb addressable space, but mapping onto different sections of
real memory. That is possible on some OSes, and it has been discussed
on freebsd-hackers, but as far as I know, no one has made available
any sort of implementation of it.
Note: out of all the processors FreeBSD supports, the above applies
only to the IA32 processor family --- the other processors supported
by or soon to be supported by FreeBSD (Alpha, Ultrasparc, IA64) all
have 64 bit memory addressing, which shouldn't run into this sort of
address space problem for a few years yet.
> A more practical question might be "Should I ensure that the number of
> precesses running won't use more than my total available memory less the
> 1Gb for the kernel?"
Practically, you want to ensure that you have sufficient RAM in the
machine that it doesn't start to swap during normal usage. If that's
impossible, then the best approach is to acquire another machine and
spread the load that way.
Cheers,
Matthew
--
Dr Matthew J Seaman MA, D.Phil. 26 The Paddocks
Savill Way
Marlow
Tel: +44 1628 476614 Bucks., SL7 1TH UK
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020923090126.GA4587>