From owner-freebsd-fs Wed Dec 26 20:31:43 2001 Delivered-To: freebsd-fs@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id 37C8537B419; Wed, 26 Dec 2001 20:31:40 -0800 (PST) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.11.6/8.11.5) with SMTP id fBR4VVD53292; Wed, 26 Dec 2001 23:31:32 -0500 (EST) (envelope-from robert@fledge.watson.org) Date: Wed, 26 Dec 2001 23:31:31 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org To: Alfred Perlstein Cc: Terry Lambert , Ian Dowse , mckusick@freebsd.org, fs@freebsd.org Subject: Re: fsck and predictive readahead? In-Reply-To: <20011226132034.N91594@elvis.mu.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-fs@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org On Wed, 26 Dec 2001, Alfred Perlstein wrote: > > The easiest way to handle this would be with a windowed mmap(), > > with the other process being given either atonomy to read one > > byte per page, predictively, or the calculation ahead. Then > > the pages would be in core by the time the first process needed > > them, and then all we are talking about is mapping. > > > > This really looks like a job for rfork(), rather than another > > process, or true threads, so that the address space is shared, > > and there is not a remapping requirement (PTE thrashing would > > potentially slow you down with any other approach). > > Agreed, first I'd like to see how it performs if I can get the IO done. > Perhaps aio, but that's kinda icky. :) There were a number of pretty neat papers done on the topic of read-ahead by the folks at CMU's PDL (Parallel Data Labatory), including a paper on balancing resource allocation (in particular memory) between various sorts of tasks: straight read-ahead, caching, and possibly reliable prefetch data. They define API's to allow a process to report prospective up-coming read and write operations, which allows the operating system to manage resources better, as well as evaluate the correctness of the prefetch requests. If we're going to try at a more general solution here, investigating some of that work would probably make sense: since the application can't measure available cache/etc resources well, especially when running on a live system via background fsck, it will have to rely on the kernel to DTRT when it comes to not letting it get ahead of itself -- you could easily see fsck slowing itself down if it assumes the wrong things about how large that read-ahead window can get. If you want specific papers, I can dig up the references. Generally, it's work by Gibson, et al. Robert N M Watson FreeBSD Core Team, TrustedBSD Project robert@fledge.watson.org NAI Labs, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message