From owner-freebsd-hackers@FreeBSD.ORG Thu Apr 3 11:29:09 2014 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 63F39C13; Thu, 3 Apr 2014 11:29:09 +0000 (UTC) Received: from mail-lb0-x230.google.com (mail-lb0-x230.google.com [IPv6:2a00:1450:4010:c04::230]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id AF913F6E; Thu, 3 Apr 2014 11:29:08 +0000 (UTC) Received: by mail-lb0-f176.google.com with SMTP id 10so1188832lbg.21 for ; Thu, 03 Apr 2014 04:29:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=Jd0ADkuz6D4KNxXNmxaFa259uswGDdTzddl0lOo1nlE=; b=KEZFC25dx5SlsI6P9/sbgL8Vo5FFeTgyU9irkksycJqb4qdpw5AYSVPlu4JVbgCN1x w3fatVVFFKfI+H3eecjSCk8HwKSnpttWB/o1ok318aXDRXsfFmTZez9ykcgsFykLB+4T FziRUzeJVXf4Qw0vUDQwMcJtUSiX1t4O69l3831UIX6xeA/woNkJ9VNVHtpn7/vcUGGW G90mocIXYZ5vm53J9xridxm3C6ZIeluAHrsGh0kz6pIfbCy/qUGi13WhO88cyDKoCh+V NRyZtpfYHzB+uEB9zuPP0hV7MhsfZG5JsMlml7iJlDQNFaDk3ir4EZHCxVYsVY/BlwVY tXxA== X-Received: by 10.152.19.7 with SMTP id a7mr4255097lae.16.1396524546610; Thu, 03 Apr 2014 04:29:06 -0700 (PDT) Received: from ?IPv6:2a02:6b8::408:d99c:501a:169e:2455? ([2a02:6b8:0:408:d99c:501a:169e:2455]) by mx.google.com with ESMTPSA id kz7sm4622628lab.16.2014.04.03.04.29.04 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 03 Apr 2014 04:29:05 -0700 (PDT) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: madvise() vs posix_fadvise() From: Dmitry Sivachenko In-Reply-To: <201403271141.41487.jhb@freebsd.org> Date: Thu, 3 Apr 2014 15:29:03 +0400 Content-Transfer-Encoding: quoted-printable Message-Id: <0AF273E6-CD43-417C-A00C-5B7445090D5B@gmail.com> References: <201403271141.41487.jhb@freebsd.org> To: John Baldwin X-Mailer: Apple Mail (2.1874) Cc: freebsd-hackers@freebsd.org, =?utf-8?Q?Trond_Endrest=C3=B8l?= X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Apr 2014 11:29:09 -0000 On 27 =D0=BC=D0=B0=D1=80=D1=82=D0=B0 2014 =D0=B3., at 19:41, John = Baldwin wrote: >>=20 >> I know about mlock(2), it is a bit overkill. >> Can someone please explain the difference between = madvise(MADV_WILLNEED) and=20 > posix_fadvise(POSIX_FADV_WILLNEED)? >=20 > Right now FADV_WILLNEED is a nop. (I have some patches to implement = it for > UFS.) I can't recall off the top of my head if MADV_WILLNEED is also = a nop. > However, if both are fully implemented they should be similar in terms = of > requesting async read-ahead. MADV_WILLNEED might also conceivably > pre-create PTEs while FADV_WILLNEED can be used on a file that isn't > mapped but is accessed via read(2). >=20 Hello and thanks for your reply. Right now I am facing the following problem (stable/10): There is a (home-grown) webserver which mmap's a large amount of data = files (total size is a bit below of RAM, say ~90GB of files with 128GB = of RAM). Server writes access.log (several gigabytes per day). Some of mmaped data files are used frequently, some are used rarely. On = startup, server walks through all of these data files so it's content is = read from disk. After some time of running, I see that rarely used data files are purged = from RAM (access to them leads to long-running disk reads) in favour of = disk cache (at 0:00, when I rotate and gzip log file I see Inactive memory goes = down to the value of log file size). Is there any way to tell VM system not to push mmap'ed regions out of = RAM in favour of disk caches?