From owner-freebsd-arch@FreeBSD.ORG Wed Nov 9 04:35:13 2011 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0476C106564A; Wed, 9 Nov 2011 04:35:13 +0000 (UTC) (envelope-from bright@elvis.mu.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id E31BB8FC0C; Wed, 9 Nov 2011 04:35:12 +0000 (UTC) Received: by elvis.mu.org (Postfix, from userid 1192) id 85D101A3C68; Tue, 8 Nov 2011 20:35:12 -0800 (PST) Date: Tue, 8 Nov 2011 20:35:12 -0800 From: Alfred Perlstein To: Tim Kientzle Message-ID: <20111109043512.GT6110@elvis.mu.org> References: <201110281426.00013.jhb@freebsd.org> <4EB2C9DD.9090606@FreeBSD.org> <20111104160319.GD6110@elvis.mu.org> <201111080800.32717.jhb@freebsd.org> <20111109033504.GS6110@elvis.mu.org> <840E509B-0D63-41C2-B26A-31655F1D42C2@kientzle.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <840E509B-0D63-41C2-B26A-31655F1D42C2@kientzle.com> User-Agent: Mutt/1.4.2.3i Cc: Bruce Cran , Ed Schouten , Jilles Tjoelker , arch@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] fadvise(2) system call X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Nov 2011 04:35:13 -0000 * Tim Kientzle [111108 20:11] wrote: > On Nov 8, 2011, at 7:35 PM, Alfred Perlstein wrote: > > > > One other optimization would be to set the bit when tar or other such apps > > notice that the file exceeds the memory of the system, effectively noting > > that it would be blowing out the entire cache. > > Tar does not know the amount of RAM in the system > nor the size of the cache. It definitely does not know > how the kernel caching policies interact with any particular > access pattern. It would be smart to have a library function to do so, basically state "I'm about to access a file of X bytes, will this fit into cache?" A query to sysctl, perhaps 'vfs.bufspace' (I haven't bothered to look at the actual correct sysctl node to query) would give a real indication of the amount of buffer size there is. If the file to extract is larger than that size, it would be pretty obvious to use the "will not need" flag for file access. > It should be able to provide hints to the kernel > about the likely access pattern, though. There is a difference between "will access sequentially" and "will access sequentially and never again". The kernel can't decide which of those are going to happen, the only thing it might be able to do is put a process in a "penalty box" (where all of its accesses are put near the end of LRU) if it happens to be experiencing little or no cache hits. Unfortunately there is a problem with this were you can wind up putting a process in the penalty box and throttling a process and limiting its speed based on a bad guess by this hueristic. It's quite a bit smarter for an app to ask the kernel "how much cache do I have?", then have the kernel respond and based on that the app can decide to ask for a cache behavior (unless the user overrides it). -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer