From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 3 15:03:59 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 8CF7E9AA for ; Thu, 3 Apr 2014 15:03:59 +0000 (UTC) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 643148D4 for ; Thu, 3 Apr 2014 15:03:59 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 654F4B968; Thu, 3 Apr 2014 11:03:54 -0400 (EDT) From: John Baldwin To: Dmitry Sivachenko Subject: Re: madvise() vs posix_fadvise() Date: Thu, 3 Apr 2014 11:02:38 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.4-CBSD-20130906; KDE/4.5.5; amd64; ; ) References: <201403271141.41487.jhb@freebsd.org> <0AF273E6-CD43-417C-A00C-5B7445090D5B@gmail.com> In-Reply-To: <0AF273E6-CD43-417C-A00C-5B7445090D5B@gmail.com> MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Message-Id: <201404031102.38598.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 03 Apr 2014 11:03:55 -0400 (EDT) Cc: freebsd-hackers@freebsd.org, Trond =?utf-8?q?Endrest=C3=B8l?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 15:03:59 -0000 On Thursday, April 03, 2014 7:29:03 am Dmitry Sivachenko wrote: >=20 > On 27 =D0=BC=D0=B0=D1=80=D1=82=D0=B0 2014 =D0=B3., at 19:41, John Baldwin= wrote: > >>=20 > >> I know about mlock(2), it is a bit overkill. > >> Can someone please explain the difference between madvise(MADV_WILLNEE= D) and=20 > > posix_fadvise(POSIX_FADV_WILLNEED)? > >=20 > > Right now FADV_WILLNEED is a nop. (I have some patches to implement it= for > > UFS.) I can't recall off the top of my head if MADV_WILLNEED is also a= nop. > > However, if both are fully implemented they should be similar in terms = of > > requesting async read-ahead. MADV_WILLNEED might also conceivably > > pre-create PTEs while FADV_WILLNEED can be used on a file that isn't > > mapped but is accessed via read(2). > >=20 >=20 >=20 > Hello and thanks for your reply. >=20 > Right now I am facing the following problem (stable/10): > There is a (home-grown) webserver which mmap's a large amount of data fil= es (total size is a bit below of RAM, say ~90GB of files with 128GB of RAM). > Server writes access.log (several gigabytes per day). >=20 > Some of mmaped data files are used frequently, some are used rarely. On s= tartup, server walks through all of these data files so it's content is rea= d=20 from disk. >=20 > After some time of running, I see that rarely used data files are purged = from RAM (access to them leads to long-running disk reads) in favour of dis= k=20 cache > (at 0:00, when I rotate and gzip log file I see Inactive memory goes down= to the value of log file size). >=20 > Is there any way to tell VM system not to push mmap'ed regions out of RAM= in favour of disk caches? Use POSIX_FADV_NOREUSE with fadvise() for the log files. They are a perfect use case for this flag. This will tell the VM system to throw the log data (move it to cache) after it writes the file. =2D-=20 John Baldwin