From owner-svn-src-head@FreeBSD.ORG Fri Sep 12 20:35:15 2014 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id C87837CD; Fri, 12 Sep 2014 20:35:15 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9A3A6639; Fri, 12 Sep 2014 20:35:15 +0000 (UTC) Received: from ralph.baldwin.cx (pool-173-70-85-31.nwrknj.fios.verizon.net [173.70.85.31]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 96D7DB9C9; Fri, 12 Sep 2014 16:35:14 -0400 (EDT) From: John Baldwin To: Slawa Olhovchenkov Subject: Re: svn commit: r270759 - in head/sys: cddl/compat/opensolaris/kern cddl/compat/opensolaris/sys cddl/contrib/opensolaris/uts/common/fs/zfs vm Date: Fri, 12 Sep 2014 14:31:18 -0400 Message-ID: <2555840.19ciXdqXnI@ralph.baldwin.cx> User-Agent: KMail/4.10.5 (FreeBSD/10.0-STABLE; KDE/4.10.5; amd64; ; ) In-Reply-To: <20140910192921.GB7129@zxy.spb.ru> References: <201408281950.s7SJo90I047213@svn.freebsd.org> <1758851.BtHUmxkW3Q@ralph.baldwin.cx> <20140910192921.GB7129@zxy.spb.ru> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Fri, 12 Sep 2014 16:35:14 -0400 (EDT) X-Mailman-Approved-At: Fri, 12 Sep 2014 21:23:39 +0000 Cc: src-committers@freebsd.org, Peter Wemm , Alan Cox , alc@freebsd.org, svn-src-all@freebsd.org, Dmitry Morozovsky , Andriy Gapon , Steven Hartland , "Matthew D. Fuller" , svn-src-head@freebsd.org, Nikolai Lifanov X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Sep 2014 20:35:15 -0000 On Wednesday, September 10, 2014 11:29:21 PM Slawa Olhovchenkov wrote: > On Wed, Sep 10, 2014 at 02:29:30PM -0400, John Baldwin wrote: > > > Application must change behaviour when reach limit (run GC, switch > > > algorithm and etc.). > > > But mmap of big data file for scaning and processing don't touch this > > > limit. > > > Must be mmap of some temoprary file touch this limit? I don't know. > > > Must be MAP_ANON touch this limit? I think yes. > > > How to make distinction of mmap data file for processing and mmap > > > temporary file for bypass this limit? I don't know. > > > > Consider also mmap() of shm_open(SHM_ANON). > > No, shm limited separatly. mmap of shm_open(SHM_ANON) don't need to adjust > this limit. It would be another trivial way of escaping RLIMIT_DATA however. > > > PS: question about jemalloc. How I can stop jemalloc to give back > > > memory to OS for some application? > > > > You would have to hack pages_purge() in chunk_mmap.c to always return true > > and not call madvise(). > > And got memory leak? No, that doesn't leak memory. jemalloc() considers that address range free regardless and will always reuse it for a future allocation. This is just a hint to let the VM system take back the backing pages because they are currently not in use by the application, and if the address is reused, the current contents of the pages are not important (can be thrown away). > I am talk about change behaviour from periodic mmap and madvise memory > to only mmap when need and leave for future use when freed by > aplication. I think (may be wrong -- please correct) in some cases > mmap/madvise will be too often. The madvise() does not cause a future mmap(). You would have to munmap() to require a future mmap(). Instead, the madvise() can result in the page being released back to the system. Later when that address is reused by jemalloc, the process would fault on the address requiring the VM to locate a new page. Disabling the madvise() only prevents that fault from occurring (However, leaving all the pages around and marked as in-use may increase memory pressure and cause you to swap and still have to fault the page back in in the future, but at greater expense.) -- John Baldwin