From owner-svn-src-all@FreeBSD.ORG Wed Sep 10 18:34:20 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C677BE09; Wed, 10 Sep 2014 18:34:20 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 978DEB14; Wed, 10 Sep 2014 18:34:20 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 8067BB94F; Wed, 10 Sep 2014 14:34:19 -0400 (EDT) From: John Baldwin To: Slawa Olhovchenkov Subject: Re: svn commit: r270759 - in head/sys: cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/fs/zfs vm Date: Wed, 10 Sep 2014 14:29:30 -0400 Message-ID: <1758851.BtHUmxkW3Q@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <20140910163058.GA7129@zxy.spb.ru> References: <201408281950.s7SJo90I047213@svn.freebsd.org> <1540004.aItdQtGhVh@ralph.baldwin.cx> <20140910163058.GA7129@zxy.spb.ru> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 10 Sep 2014 14:34:19 -0400 (EDT) X-Mailman-Approved-At: Wed, 10 Sep 2014 19:28:24 +0000 Cc: src-committers@freebsd.org, Peter Wemm , Alan Cox , alc@freebsd.org, svn-src-all@freebsd.org, Dmitry Morozovsky , Andriy Gapon , Steven Hartland , "Matthew D. Fuller" , svn-src-head@freebsd.org, Nikolai Lifanov X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Sep 2014 18:34:21 -0000 On Wednesday, September 10, 2014 08:30:58 PM Slawa Olhovchenkov wrote: > On Wed, Sep 10, 2014 at 10:44:46AM -0400, John Baldwin wrote: > > On Tuesday, September 09, 2014 02:25:18 PM Slawa Olhovchenkov wrote: > > > On Mon, Sep 08, 2014 at 11:17:51AM -0400, John Baldwin wrote: > > > > On Sunday, September 07, 2014 04:56:49 PM Slawa Olhovchenkov wrote: > > > > > PS: very bad that 'data limit' don't anymore reflect application > > > > > memory consumer. and very small application can adapt to 'no memory' > > > > > from system. > > > > > > > > You can use RLIMIT_AS instead of RLIMIT_DATA. jemalloc can also be > > > > configured to use sbrk(), though I think there's no way to prevent it > > > > from falling back to mmap(MAP_ANON) if sbrk() fails. > > > > > > Formally you are right. Realy this is scorn. And I don't know how > > > rightly. May be account in RLIMIT_DATA MAP_ANON? Anyway firefox don't > > > run GC if malloc fail and this is bad. > > > > It was not intended as scorn, simply stating what options you have > > Hmm, this is may bad english and try to translate some russian sentence. > I am try to do some tuning for satisfaction me. > You proposal do some limiting, but don't delight. Ok. > > available as alternatives to what you are asking for. The Firefox > > behavior certainly seems unfortunate. :( However, using RLIMIT_AS > > for that should work as the mmap() call malloc() makes will fail > > causing malloc() to fail. However, you need RLIMIT_AS to be a bit > > larger than what you would set RLIMIT_DATA to. > > > > As to having MAP_ANON honor RLIMIT_DATA, I'm not sure how well that > > would work. I assume that was discussed as an option in the previous > > threads on the topic of RLIMIT_DATA being useless with jemalloc. > > I don't recall it though. Perhaps Alan can speak to whether or not > > that is a good idea? > > What is sense of RLIMIT_DATA (as I understand)? > This is signaling to application of memory consumption. Yes, traditionally it only affected malloc() and not other portions of the address space like the stack (RLIMIT_STACK) or anything in shared libraries (including text/data/bss). > Limiting RSS don't visible to application. Yes, RLIMIT_RSS is not often useful because it is not synchronous with the RSS changing. > Application must change behaviour when reach limit (run GC, switch > algorithm and etc.). > But mmap of big data file for scaning and processing don't touch this > limit. > Must be mmap of some temoprary file touch this limit? I don't know. > Must be MAP_ANON touch this limit? I think yes. > How to make distinction of mmap data file for processing and mmap > temporary file for bypass this limit? I don't know. Consider also mmap() of shm_open(SHM_ANON). > And again, most application don't handle correctly NULL of malloc(). > I think this is depreciate of all. This is true, but it seems Firefox might, and you could set RLIMIT_AS for Firefox in particular so it gets adequate feedback. I do think if you can determine the "correct" value for RLIMIT_AS for Firefox that mmap(MAP_ANON) would fail and result in NULL from malloc(). > PS: question about jemalloc. How I can stop jemalloc to give back > memory to OS for some application? You would have to hack pages_purge() in chunk_mmap.c to always return true and not call madvise(). -- John Baldwin