Date: Thu, 05 Apr 2001 14:06:33 +0900
From: Seigo Tanimura <tanimura@r.dl.itc.u-tokyo.ac.jp>
To: arch@FreeBSD.org
Cc: bde@zeta.org.au, Seigo Tanimura <tanimura@r.dl.itc.u-tokyo.ac.jp>
Subject: Mmap(2) should start just below stack (was: Re: Bumping up {MAX,DFL}*SIZ in i386)
Message-ID: <200104050506.f3556Xw28400@rina.r.dl.itc.u-tokyo.ac.jp>
In-Reply-To: <200103230517.f2N5HXx08605@rina.r.dl.itc.u-tokyo.ac.jp>
References: <200103191056.f2JAuox00630@rina.r.dl.itc.u-tokyo.ac.jp> <Pine.BSF.4.21.0103201350200.41190-100000@besplex.bde.org> <200103230517.f2N5HXx08605@rina.r.dl.itc.u-tokyo.ac.jp>
next in thread | previous in thread | raw e-mail | index | archive | help
bde and I were discussing of what is likely to be if mmap(2) is switched to start at the max of RLIMIT_DATA. As he seems to be busy now, I post this matter to -arch. Below depicts how we use a process vm space for an i386 elf executable binary: +--------------------+ 4GB | KVM | +--------------------+ 3GB | Process Stack | +--------------------+ down to 3GB - max of RLIMIT_STACK |Reserved for Process| | Stack | +--------------------+ 3GB - max of RLIMIT_STACK | Mmap(2) Heap | +--------------------+ up to 3GB - max of RLIMIT_STACK | Mmap(2)ed space | +--------------------+ MAXDSIZ | Unused | +--------------------+ end of bss + max of RLIMIT_DATA |Reserved for Malloc | +--------------------+ up to end of bss + max of RLIMIT_DATA (break) | Malloc(3) Heap | +--------------------+ end of bss | Executable binary | +--------------------+ _init? | | +--------------------+ 0 The first problem we encountered is that increasing MAXDSIZ limits the size of the address space reserved for mmap(2). Hence it is difficult to run an malloc(3)-bound application and an mmap(2)-bound one on the same machine. bde has proposed to solve this problem by starting mmap(2) at the end of bss + max of RLIMIT_DATA so that a user/application can reserve an arbitrary size of the mmap(2) heap without recompiling a kernel. The usage of a process vm space now looks something like this: | Process Stack | +--------------------+ down to 3GB - max of RLIMIT_STACK |Reserved for Process| | Stack | +--------------------+ 3GB - max of RLIMIT_STACK | Mmap(2) Heap | +--------------------+ up to 3GB - max of RLIMIT_STACK | Mmap(2)ed space | This may be fragmented. +--------------------+ end of bss + max of RLIMIT_DATA |Reserved for Malloc | +--------------------+ up to end of bss + max of RLIMIT_DATA (break) | Malloc(3) Heap | While the max of RLIMIT_DATA now seems to be able to go up, it cannot cross over any mmap(2)ed regions. Since an elf dynamic linker and shared objects are loaded by mmap(2) to reside just above the end of bss + max of RLIMIT_DATA, the max of RLIMIT_DATA cannot be increased in most cases. I propose a solution against that problem by allocating the space just below the stack of a process for mmap(2), as depicted below: | Process Stack | +--------------------+ down to 3GB - max of RLIMIT_STACK |Reserved for Process| | Stack | +--------------------+ 3GB - max of RLIMIT_STACK | Mmap(2)ed space | | (mmap(2), dynamic | This may be fragmented. | linker, shared | | objects, etc) | +--------------------+ down to end of bss + max of RLIMIT_DATA | Mmap(2) Heap | +--------------------+ end of bss + max of RLIMIT_DATA |Reserved for Malloc | +--------------------+ up to end of bss + max of RLIMIT_DATA (break) | Malloc(3) Heap | As the end of bss + max of RLIMIT_DATA gets less likely to cross over mapped regions, we can provide much better flexibility of memory usage than as of now to both malloc(3)-bound and mmap(2)-bound applications. Since we are likely to consume a much smaller size of memory for the stack than for malloc(3) and mmap(2), collision of the stack and the mmap(2)ed space should not be troublesome. Solaris seems to have already taken this method. -- Seigo Tanimura <tanimura@r.dl.itc.u-tokyo.ac.jp> <tanimura@FreeBSD.org> To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200104050506.f3556Xw28400>