Date: Fri, 12 Sep 2014 14:31:18 -0400 From: John Baldwin <jhb@freebsd.org> To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: src-committers@freebsd.org, Peter Wemm <peter@wemm.org>, Alan Cox <alc@rice.edu>, alc@freebsd.org, svn-src-all@freebsd.org, Dmitry Morozovsky <marck@rinet.ru>, Andriy Gapon <avg@freebsd.org>, Steven Hartland <killing@multiplay.co.uk>, "Matthew D. Fuller" <fullermd@over-yonder.net>, svn-src-head@freebsd.org, Nikolai Lifanov <lifanov@mail.lifanov.com> Subject: Re: svn commit: r270759 - in head/sys: cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/fs/zfs vm Message-ID: <2555840.19ciXdqXnI@ralph.baldwin.cx> In-Reply-To: <20140910192921.GB7129@zxy.spb.ru> References: <201408281950.s7SJo90I047213@svn.freebsd.org> <1758851.BtHUmxkW3Q@ralph.baldwin.cx> <20140910192921.GB7129@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday, September 10, 2014 11:29:21 PM Slawa Olhovchenkov wrote: > On Wed, Sep 10, 2014 at 02:29:30PM -0400, John Baldwin wrote: > > > Application must change behaviour when reach limit (run GC, switch > > > algorithm and etc.). > > > But mmap of big data file for scaning and processing don't touch this > > > limit. > > > Must be mmap of some temoprary file touch this limit? I don't know. > > > Must be MAP_ANON touch this limit? I think yes. > > > How to make distinction of mmap data file for processing and mmap > > > temporary file for bypass this limit? I don't know. > > > > Consider also mmap() of shm_open(SHM_ANON). > > No, shm limited separatly. mmap of shm_open(SHM_ANON) don't need to adjust > this limit. It would be another trivial way of escaping RLIMIT_DATA however. > > > PS: question about jemalloc. How I can stop jemalloc to give back > > > memory to OS for some application? > > > > You would have to hack pages_purge() in chunk_mmap.c to always return true > > and not call madvise(). > > And got memory leak? No, that doesn't leak memory. jemalloc() considers that address range free regardless and will always reuse it for a future allocation. This is just a hint to let the VM system take back the backing pages because they are currently not in use by the application, and if the address is reused, the current contents of the pages are not important (can be thrown away). > I am talk about change behaviour from periodic mmap and madvise memory > to only mmap when need and leave for future use when freed by > aplication. I think (may be wrong -- please correct) in some cases > mmap/madvise will be too often. The madvise() does not cause a future mmap(). You would have to munmap() to require a future mmap(). Instead, the madvise() can result in the page being released back to the system. Later when that address is reused by jemalloc, the process would fault on the address requiring the VM to locate a new page. Disabling the madvise() only prevents that fault from occurring (However, leaving all the pages around and marked as in-use may increase memory pressure and cause you to swap and still have to fault the page back in in the future, but at greater expense.) -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2555840.19ciXdqXnI>