Date: Tue, 22 Nov 2011 19:43:57 +0400 From: Maxim Dounin <mdounin@mdounin.ru> To: Kostik Belousov <kostikbel@gmail.com> Cc: is@nginx.org, arch@freebsd.org, current@freebsd.org Subject: Re: RLIMIT_DATA and malloc(3) use of mmap(2) Message-ID: <20111122154357.GI95664@mdounin.ru> In-Reply-To: <20111122124410.GP50300@deviant.kiev.zoral.com.ua> References: <20111122124410.GP50300@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello! On Tue, Nov 22, 2011 at 02:44:10PM +0200, Kostik Belousov wrote: > I was reminded about the patch I wrote for Igor Sysoev some time ago. > The issue the patch tries to handle is that jemalloc uses mmap() instead > of sbrk() for pages allocation, and thus RLIMIT_DATA limit is no longer > effective to put the bounds on the process heap. Since reverting to sbrk > for such purpose is worse then the issue itself, I proposed a solution > of 'self-restricting malloc'. Just a little clarification for others: currently, there is no way to *safely* limit memory usage of a process while using jemalloc with mmap(). The only thing available is RLIMIT_VMEM, but it's not safe as it may be reached on stack grow (leaving no possibility for an application to handle this). > The patch adds a flag MAP_DATALIMIT to the flags argument of mmap(2), > which instructs the system to account the mapping in the RLIMIT_DATA > resource count. The malloc(3) also gets new option 'L' to enable > passing MAP_DATALIMIT to mmap() when allocating pages. By default, > the 'L' option is not activated. > > Now, if user wants to ensure that process heap is restricted by the > ulimit -d and still use mmap() for jemalloc, he supplies the option > using any mechanism. The behaviour is voluntaristic, to prevent the > trashing use RLIMIT_SWAP. > > Do people consider the facility useful ? Yes, at least some way to safely limit process memory usage is certainly needed. It's a bit sad this isn't enabled by default, but it's probably too late for this. RLIMIT_DATA was (almost) a noop for too long and making it work again to limit all memory allocations will break POLA. > Any comments for the patch itself ? > > http://people.freebsd.org/~kib/misc/map_datalimit.1.patch Patch looks ok for me (though I'm not a VM expert). Another possible aproach would be to introduce separate anonymous (private?) mmap limit, this will allow to do essentially the same thing in a bit more consistent (IMHO) manner. Maxim Dounin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111122154357.GI95664>