Date: Fri, 27 Nov 2009 18:22:38 -0800 From: Maxim Sobolev <sobomax@sippysoft.com> To: Jason Evans <jasone@FreeBSD.org> Cc: FreeBSD Hackers <freebsd-hackers@FreeBSD.org> Subject: Re: heap limits: mmap(2) vs. break(2) on i386 Message-ID: <4B10896E.3080201@sippysoft.com> In-Reply-To: <4B107D29.5030307@FreeBSD.org> References: <4B1041EB.9020109@sippysoft.com> <4B1059CA.6040605@FreeBSD.org> <4B10687D.3050209@sippysoft.com> <4B107D29.5030307@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Jason Evans wrote: > Maxim Sobolev wrote: >> Jason Evans wrote: >>> Maxim Sobolev wrote: >>>> I am trying to figure out why java fails to start with 1024MB of >>>> heap on i386 with 4GB of RAM and 4GB of swap. Both MAXDSIZ and >>>> DFLDSIZ are set to 2GB. >>> >>> Some memory (1GiB?) is reserved for kernel address space, and you >>> reserved 2GiB for DSS. That leaves less than 1GiB available after >>> shared libraries and whatnot are mapped. If there is more than 1GiB >>> available, mmap can still fail due to the memory being non-contiguous. >> >> So, are you saying that by allocating 2GB to MAXDSIZ, I limit myself >> less than 1GB left to be allocated via mmap()? > > Yes, my recollection is that MAXDSIZ controls the amount of virtual > address space dedicated to DSS, and this address space will not be > mapped via anonymous mmap. I wanted to move completely away from using > sbrk in malloc, but we can't completely remove DSS for backward > compatibility reasons, which means less heap address space than would be > ideal. > >> What is the best strategy if I want to maximize amount of memory >> available to applications? Most of modern applications use mmap(), >> isn't it? Then where MAXDSIZ can bite me if I set it to say 512MB? > > I would set MAXDSIZ to 0, so that the maximum amount of memory is > available for mapping shared libraries and files, and allocating via > malloc. This may cause problems with a couple of ports that implement > their own memory allocators based on sbrk, but otherwise it should be > all good. You might also set /etc/malloc.conf to 'd' in order to > disable the sbrk calls. I see, thank you for the explanation. One of the problem that we are having is that we use a lot of interpreted languages in our environment (python, php etc), and most of those implement their own memory allocators, some of which rely on sbrk(2) unfortunately. I believe that's where that 2GB limit of ours comes from - one of our Python applications is very memory hungry and we had to bump that limit to allow it sufficient room. Crazy idea, perhaps, but has anyone considered wrapping up sbrk(2) into mmap(2), so that there is only one memory pool to draw from? Switch to 64-bit certainly helps, however there are lot of 32-bit machines hanging around and we will see them for a while in the embedded space. Certainly current situation with two separate sources of heap memory is not normal. -Maxim
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4B10896E.3080201>