From owner-freebsd-hackers@FreeBSD.ORG Wed Oct 9 13:45:42 2013 Return-Path: Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id A4B9CB4C for ; Wed, 9 Oct 2013 13:45:42 +0000 (UTC) (envelope-from rwmaillists@googlemail.com) Received: from mail-bk0-x22d.google.com (mail-bk0-x22d.google.com [IPv6:2a00:1450:4008:c01::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 3C83027B7 for ; Wed, 9 Oct 2013 13:45:42 +0000 (UTC) Received: by mail-bk0-f45.google.com with SMTP id mx11so332374bkb.4 for ; Wed, 09 Oct 2013 06:45:40 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20120113; h=date:from:to:subject:message-id:in-reply-to:references:mime-version :content-type:content-transfer-encoding; bh=8+6PSDn3P3Wza3oC4+xRcq51v/ZwzfstpmeH5N+zjN0=; b=XtL4AvDHk8FxXAk32aJKqniT0HTHEFziTLF6n4FCIR2tx2YyIjDuqHO8BSedXyqY5H BNyVizaInyy8RgXrdwOzjeGqZD4cOjofwKTv5NQYNQjU1IJLzfo29aanYmDi8OnpL7Rt 4/G2A974pn66f5tGu7vfmYG7N3VMEles9NF1zdaE92XRt8T6miNM8G4XP9H0uW8XpKhR zumMpdIFzZ0GKjpTaj4cftJSN3N4p51uGzoqt7O11QanvxBSRhmiexSLhNlsRqoDFhFx SExMxN30juOK5P7YqD3qHm1Lbfiyyryclhdk8vouukkMn7vZ1NzqPlzVla87bte8kJJs cuQQ== X-Received: by 10.205.14.197 with SMTP id pr5mr7083012bkb.6.1381326340444; Wed, 09 Oct 2013 06:45:40 -0700 (PDT) Received: from gumby.homeunix.com (87-194-105-247.bethere.co.uk. [87.194.105.247]) by mx.google.com with ESMTPSA id a4sm23896162bko.11.1969.12.31.16.00.00 (version=SSLv3 cipher=RC4-SHA bits=128/128); Wed, 09 Oct 2013 06:45:40 -0700 (PDT) Date: Wed, 9 Oct 2013 14:45:37 +0100 From: RW To: freebsd-hackers@freebsd.org Subject: Re: mmap() question Message-ID: <20131009144537.4b1a5eb7@gumby.homeunix.com> In-Reply-To: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> References: <95E0B821-BF9B-4EBF-A1E5-1DDCBB1C3D1B@gmail.com> X-Mailer: Claws Mail 3.9.2 (GTK+ 2.24.19; amd64-portbld-freebsd10.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Oct 2013 13:45:42 -0000 On Wed, 9 Oct 2013 15:42:27 +0400 Dmitry Sivachenko wrote: > Hello! > > I have a program which mmap()s a lot of large files (total size more > that RAM and I have no swap), but it needs only small parts of that > files at a time. > > My understanding is that when using mmap when I access some memory > region OS reads the relevant portion of that file from disk and > caches the result in memory. If there is no free memory, OS will > purge previously read part of mmap'ed file to free memory for the new > chunk. > >... > > It seems I incorrectly understand how mmap() works, can you please > clarify what's going wrong? > > I expect that program to run indefinitely, purging some regions out > of RAM and reading the relevant parts of files. I think your problem is that you are accessing the memory so rapidly that the pages can't even get out of the active queue. The VM system isn't optimized for this kind of abnormal access.