Date: Wed, 10 Sep 2014 14:29:30 -0400 From: John Baldwin <jhb@freebsd.org> To: Slawa Olhovchenkov <slw@zxy.spb.ru> Cc: src-committers@freebsd.org, Peter Wemm <peter@wemm.org>, Alan Cox <alc@rice.edu>, alc@freebsd.org, svn-src-all@freebsd.org, Dmitry Morozovsky <marck@rinet.ru>, Andriy Gapon <avg@freebsd.org>, Steven Hartland <killing@multiplay.co.uk>, "Matthew D. Fuller" <fullermd@over-yonder.net>, svn-src-head@freebsd.org, Nikolai Lifanov <lifanov@mail.lifanov.com> Subject: Re: svn commit: r270759 - in head/sys: cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/fs/zfs vm Message-ID: <1758851.BtHUmxkW3Q@ralph.baldwin.cx> In-Reply-To: <20140910163058.GA7129@zxy.spb.ru> References: <201408281950.s7SJo90I047213@svn.freebsd.org> <1540004.aItdQtGhVh@ralph.baldwin.cx> <20140910163058.GA7129@zxy.spb.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wednesday, September 10, 2014 08:30:58 PM Slawa Olhovchenkov wrote: > On Wed, Sep 10, 2014 at 10:44:46AM -0400, John Baldwin wrote: > > On Tuesday, September 09, 2014 02:25:18 PM Slawa Olhovchenkov wrote: > > > On Mon, Sep 08, 2014 at 11:17:51AM -0400, John Baldwin wrote: > > > > On Sunday, September 07, 2014 04:56:49 PM Slawa Olhovchenkov wrote: > > > > > PS: very bad that 'data limit' don't anymore reflect application > > > > > memory consumer. and very small application can adapt to 'no memory' > > > > > from system. > > > > > > > > You can use RLIMIT_AS instead of RLIMIT_DATA. jemalloc can also be > > > > configured to use sbrk(), though I think there's no way to prevent it > > > > from falling back to mmap(MAP_ANON) if sbrk() fails. > > > > > > Formally you are right. Realy this is scorn. And I don't know how > > > rightly. May be account in RLIMIT_DATA MAP_ANON? Anyway firefox don't > > > run GC if malloc fail and this is bad. > > > > It was not intended as scorn, simply stating what options you have > > Hmm, this is may bad english and try to translate some russian sentence. > I am try to do some tuning for satisfaction me. > You proposal do some limiting, but don't delight. Ok. > > available as alternatives to what you are asking for. The Firefox > > behavior certainly seems unfortunate. :( However, using RLIMIT_AS > > for that should work as the mmap() call malloc() makes will fail > > causing malloc() to fail. However, you need RLIMIT_AS to be a bit > > larger than what you would set RLIMIT_DATA to. > > > > As to having MAP_ANON honor RLIMIT_DATA, I'm not sure how well that > > would work. I assume that was discussed as an option in the previous > > threads on the topic of RLIMIT_DATA being useless with jemalloc. > > I don't recall it though. Perhaps Alan can speak to whether or not > > that is a good idea? > > What is sense of RLIMIT_DATA (as I understand)? > This is signaling to application of memory consumption. Yes, traditionally it only affected malloc() and not other portions of the address space like the stack (RLIMIT_STACK) or anything in shared libraries (including text/data/bss). > Limiting RSS don't visible to application. Yes, RLIMIT_RSS is not often useful because it is not synchronous with the RSS changing. > Application must change behaviour when reach limit (run GC, switch > algorithm and etc.). > But mmap of big data file for scaning and processing don't touch this > limit. > Must be mmap of some temoprary file touch this limit? I don't know. > Must be MAP_ANON touch this limit? I think yes. > How to make distinction of mmap data file for processing and mmap > temporary file for bypass this limit? I don't know. Consider also mmap() of shm_open(SHM_ANON). > And again, most application don't handle correctly NULL of malloc(). > I think this is depreciate of all. This is true, but it seems Firefox might, and you could set RLIMIT_AS for Firefox in particular so it gets adequate feedback. I do think if you can determine the "correct" value for RLIMIT_AS for Firefox that mmap(MAP_ANON) would fail and result in NULL from malloc(). > PS: question about jemalloc. How I can stop jemalloc to give back > memory to OS for some application? You would have to hack pages_purge() in chunk_mmap.c to always return true and not call madvise(). -- John Baldwin
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1758851.BtHUmxkW3Q>