Date: Tue, 8 Nov 2011 20:35:12 -0800 From: Alfred Perlstein <alfred@freebsd.org> To: Tim Kientzle <tim@kientzle.com> Cc: Bruce Cran <bruce@cran.org.uk>, Ed Schouten <ed@80386.nl>, Jilles Tjoelker <jilles@stack.nl>, arch@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] fadvise(2) system call Message-ID: <20111109043512.GT6110@elvis.mu.org> In-Reply-To: <840E509B-0D63-41C2-B26A-31655F1D42C2@kientzle.com> References: <201110281426.00013.jhb@freebsd.org> <4EB2C9DD.9090606@FreeBSD.org> <20111104160319.GD6110@elvis.mu.org> <201111080800.32717.jhb@freebsd.org> <20111109033504.GS6110@elvis.mu.org> <840E509B-0D63-41C2-B26A-31655F1D42C2@kientzle.com>
next in thread | previous in thread | raw e-mail | index | archive | help
* Tim Kientzle <tim@kientzle.com> [111108 20:11] wrote: > On Nov 8, 2011, at 7:35 PM, Alfred Perlstein wrote: > > > > One other optimization would be to set the bit when tar or other such apps > > notice that the file exceeds the memory of the system, effectively noting > > that it would be blowing out the entire cache. > > Tar does not know the amount of RAM in the system > nor the size of the cache. It definitely does not know > how the kernel caching policies interact with any particular > access pattern. It would be smart to have a library function to do so, basically state "I'm about to access a file of X bytes, will this fit into cache?" A query to sysctl, perhaps 'vfs.bufspace' (I haven't bothered to look at the actual correct sysctl node to query) would give a real indication of the amount of buffer size there is. If the file to extract is larger than that size, it would be pretty obvious to use the "will not need" flag for file access. > It should be able to provide hints to the kernel > about the likely access pattern, though. There is a difference between "will access sequentially" and "will access sequentially and never again". The kernel can't decide which of those are going to happen, the only thing it might be able to do is put a process in a "penalty box" (where all of its accesses are put near the end of LRU) if it happens to be experiencing little or no cache hits. Unfortunately there is a problem with this were you can wind up putting a process in the penalty box and throttling a process and limiting its speed based on a bad guess by this hueristic. It's quite a bit smarter for an app to ask the kernel "how much cache do I have?", then have the kernel respond and based on that the app can decide to ask for a cache behavior (unless the user overrides it). -- - Alfred Perlstein .- VMOA #5191, 03 vmax, 92 gs500, 85 ch250, 07 zx10 .- FreeBSD committer
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20111109043512.GT6110>