From owner-freebsd-current Mon Mar 17 12:22:40 2003 Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D38637B404; Mon, 17 Mar 2003 12:22:38 -0800 (PST) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4320843FAF; Mon, 17 Mar 2003 12:22:37 -0800 (PST) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc02.attbi.com (sccrmhc02) with ESMTP id <2003031720223600200rqence>; Mon, 17 Mar 2003 20:22:36 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id MAA70592; Mon, 17 Mar 2003 12:22:34 -0800 (PST) Date: Mon, 17 Mar 2003 12:22:33 -0800 (PST) From: Julian Elischer To: FreeBSD current users Cc: fs@freebsd.org Subject: Anyone working on fsck? Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-current@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Is there anyone working on fsck? Recent timings with a fast machine with 1TB filesystems show that it takes abuot 6 hours to fsck such a filesystem (on a fast array with a lot of RAM) This is with a version of fsck that already has some locally developed speedups and changes. I have not dared time the standard one yet. Pass 1 takes 2 hours on its own. but the cpu is only about 5% busy. Looking at the access patterns on the drive suggests that there MAY be some chance to speed this up using such things reading in teh active cylinder group in advance, and performing the indirect block checks in another thread/process. Does anyone have any plans to work on this sort of thing? On another issue: The memory requirements for fsck are about 700MB+ per TB of filesystem on FreeBSD4.x With the advent of UFS2 this increases, and the filesyste,s can also get bigger. This means that a 2+TB UFS2 filesystem will be IMPOSSIBLE to check on an i386 machine. One solution is to implement an "offline, non-in-place" filesystem checker using all those techniques we all learned in CS101 relating to mag-tape merge sorts etc. basically you'd have to have a small disk partition set asside to hold working files, to which teh checker would write files of records detailing block numbers etc. then you would do merge or block sorts on those files to get them in various orders (depending on fields in the records) And recombine them to find such things as: multiple referenced blocks, and other file inconsitancies. It wouldn;t be super fast but at least it COULD be used to check a 30TB array, where the in-memory version would beed a process VM space of 24MB which is clearly impossible on a x86. Julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message