From owner-freebsd-hackers Mon Apr 19 14: 1:37 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from sumatra.americantv.com (sumatra.americantv.com [207.170.17.37]) by hub.freebsd.org (Postfix) with ESMTP id 8162915637 for ; Mon, 19 Apr 1999 14:01:25 -0700 (PDT) (envelope-from jlemon@americantv.com) Received: from right.PCS (right.PCS [148.105.10.31]) by sumatra.americantv.com (8.8.5/8.8.5) with ESMTP id PAA26703; Mon, 19 Apr 1999 15:58:58 -0500 (CDT) Received: from free.pcs (free.PCS [148.105.10.51]) by right.PCS (8.6.13/8.6.4) with ESMTP id PAA02158; Mon, 19 Apr 1999 15:58:27 -0500 Received: (from jlemon@localhost) by free.pcs (8.8.6/8.8.5) id PAA16517; Mon, 19 Apr 1999 15:58:27 -0500 (CDT) Date: Mon, 19 Apr 1999 15:58:27 -0500 (CDT) From: Jonathan Lemon Message-Id: <199904192058.PAA16517@free.pcs> To: Arjan.deVet@adv.iae.nl, hackers@freebsd.org Subject: Re: Directories not VMIO cached at all! X-Newsgroups: local.mail.freebsd-hackers In-Reply-To: References: Organization: Architecture and Operating System Fanatics Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG In article you write: >In article <199904191650.JAA24137@vashon.polstra.com> John Polstra writes: > >>If I understand your description correctly, this fix could really >>benefit master CVSup servers such as freefall. Those servers >>typically have 8-12 running cvsupd processes, all doing tree walks >>over the same CVS repository and making a stat() call on every file. >> >>Do you think it would help? > >It helps to some extend I think. The Squid server I've been testing can >keep 32MB worth of directories cached after some tuning but because of >the enormous amount of reads and writes being done half of the >directories get removed from the cache after 5-10 minutes. We're >speaking about 750,000-1,000,000 files in 4096 directories in a >two-level hierarchy where 80% of the files is 9 KB or less in size. >B.t.w., the Squid people are working on a SquidFS which will not use >individual files anymore. Which is why Peregrine prefers to use raw disk partitions (along with a userland variant of LFS) to store the pages, since the filesystem currently imposes too much overhead for good performance. It's interesting, LFS seems to be a great web-cache filesystem, you don't really need to preserve every file, you just throw some away. No fsck; if the system crashes, you can just start all over again; after all, it _is_ a cache, right? (In Peregrine, this behavior is tunable; some environments don't want to lose the entire cache). -- Jonathan To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message