From owner-freebsd-hackers@FreeBSD.ORG Fri Oct 11 05:57:29 2013 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 26613DE3 for ; Fri, 11 Oct 2013 05:57:29 +0000 (UTC) (envelope-from trtrmitya@gmail.com) Received: from mail-lb0-x22c.google.com (mail-lb0-x22c.google.com [IPv6:2a00:1450:4010:c04::22c]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id A296528A5 for ; Fri, 11 Oct 2013 05:57:28 +0000 (UTC) Received: by mail-lb0-f172.google.com with SMTP id x18so2975004lbi.31 for ; Thu, 10 Oct 2013 22:57:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=content-type:mime-version:subject:from:in-reply-to:date:cc :content-transfer-encoding:message-id:references:to; bh=9U9XUUcvh8qK2WVIt3qJOCjItz6xW42OwOaHWrujU+g=; b=bsogWxjTcudiXWiI61ATl4r8X63Cw1WocypcnxX8L28bxfABS+2ggHo/VYgvY07vmC oWEOAYID5Wdcms5U0Xz0yk33djvHR6eSeedxJjBM5gb3yzz3GYjzEglHMTNAjfNwjNoa Grz/pR7fkjpEIXcM1KMK+7BGXf4iJ7fOjvk3M8fQUc8rGbPcUcoKguU5/IdfoUjIMpVk HvyOliG4fCvi3/ty0IXexWfWUGVAqS0wMqBcCE91Kqqgb1QrHQ1nU49d8s1Xzkx2Wtus yvX2ZflA/SN+sjkhqJw+Ma91EIUe7x+hcYTY4HmztL31JhzVzPbHXAGQMn/xB5V2Paex oedg== X-Received: by 10.112.42.68 with SMTP id m4mr15407678lbl.4.1381471046645; Thu, 10 Oct 2013 22:57:26 -0700 (PDT) Received: from [10.0.1.20] (ip-95-220-138-189.bb.netbynet.ru. [95.220.138.189]) by mx.google.com with ESMTPSA id js17sm43503143lab.5.1969.12.31.16.00.00 (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 10 Oct 2013 22:57:25 -0700 (PDT) Content-Type: text/plain; charset=us-ascii Mime-Version: 1.0 (Mac OS X Mail 6.6 \(1510\)) Subject: Re: mmap() question From: Dmitry Sivachenko In-Reply-To: <20131011051702.GE41229@kib.kiev.ua> Date: Fri, 11 Oct 2013 09:57:24 +0400 Content-Transfer-Encoding: quoted-printable Message-Id: References: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> <20131011051702.GE41229@kib.kiev.ua> To: Konstantin Belousov X-Mailer: Apple Mail (2.1510) X-Mailman-Approved-At: Fri, 11 Oct 2013 11:55:30 +0000 Cc: "hackers@freebsd.org" X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Oct 2013 05:57:29 -0000 On 11.10.2013, at 9:17, Konstantin Belousov wrote: > On Wed, Oct 09, 2013 at 03:42:27PM +0400, Dmitry Sivachenko wrote: >> Hello! >>=20 >> I have a program which mmap()s a lot of large files (total size more = that RAM and I have no swap), but it needs only small parts of that = files at a time. >>=20 >> My understanding is that when using mmap when I access some memory = region OS reads the relevant portion of that file from disk and caches = the result in memory. If there is no free memory, OS will purge = previously read part of mmap'ed file to free memory for the new chunk. >>=20 >> But this is not the case. I use the following simple program which = gets list of files as command line arguments, mmap()s them all and then = selects random file and random 1K parts of that file and computes a XOR = of bytes from that region. >> After some time the program dies: >> pid 63251 (a.out), uid 1232, was killed: out of swap space >>=20 >> It seems I incorrectly understand how mmap() works, can you please = clarify what's going wrong? >>=20 >> I expect that program to run indefinitely, purging some regions out = of RAM and reading the relevant parts of files. >>=20 >=20 > You did not specified several very important parameters for your test: > 1. total amount of RAM installed 24GB > 2. count of the test files and size of the files To be precise: I used 57 files with size varied form 74MB to 19GB. The total size of these files is 270GB. > 3. which filesystem files are located at UFS @ SSD drive > 4. version of the system. FreeBSD 9.2-PRERELEASE #0 r254880M: Wed Aug 28 11:07:54 MSK 2013=