From owner-freebsd-arch@FreeBSD.ORG  Wed Jan  5 06:57:34 2005
Return-Path: <owner-freebsd-arch@FreeBSD.ORG>
Delivered-To: freebsd-arch@freebsd.org
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP
	id 58EA216A4CE; Wed,  5 Jan 2005 06:57:34 +0000 (GMT)
Received: from pimout2-ext.prodigy.net (pimout2-ext.prodigy.net
	[207.115.63.101])	by mx1.FreeBSD.org (Postfix) with ESMTP
	id E1B4B43D49; Wed,  5 Jan 2005 06:57:33 +0000 (GMT)
	(envelope-from julian@elischer.org)
Received: from [192.168.1.102] (adsl-216-100-134-143.dsl.snfc21.pacbell.net
	[216.100.134.143])j056vOXO011736;	Wed, 5 Jan 2005 01:57:29 -0500
Message-ID: <41DB8FD3.3030101@elischer.org>
Date: Tue, 04 Jan 2005 22:57:23 -0800
From: Julian Elischer <julian@elischer.org>
User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.8a3) Gecko/20041017
X-Accept-Language: en, hu
MIME-Version: 1.0
To: "M. Warner Losh" <imp@bsdimp.com>
References: <20050104224043.GM784@darkness.comp.waw.pl>
	<41DB2B24.6050005@elischer.org> <20050104.203502.85411551.imp@bsdimp.com>
In-Reply-To: <20050104.203502.85411551.imp@bsdimp.com>
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
cc: arch@freebsd.org
cc: scottl@freebsd.org
cc: pjd@freebsd.org
Subject: Re: BigDisk project: du(1) 64bit clean.
X-BeenThere: freebsd-arch@freebsd.org
X-Mailman-Version: 2.1.1
Precedence: list
List-Id: Discussion related to FreeBSD architecture
	<freebsd-arch.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arch>
List-Post: <mailto:freebsd-arch@freebsd.org>
List-Help: <mailto:freebsd-arch-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arch>,
	<mailto:freebsd-arch-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 05 Jan 2005 06:57:34 -0000

M. Warner Losh wrote:
> In message: <41DB2B24.6050005@elischer.org>
>             Julian Elischer <julian@elischer.org> writes:
> : 
> : 
> : Pawel Jakub Dawidek wrote:
> : 
> : >Hi.
> : >
> : >I want you to look at two patches which makes du(1) 64bit clean.
> : >This work is part of the BigDisk project:
> : >
> : >	http://www.freebsd.org/projects/bigdisk/
> : >  
> : >
> : One thing that needs to be done is an 2ndary storage fsck.
> :  that doesn't try put everything in RAM.
> : Basically this will mean extracting all the metadata from filesystems into
> : files and running sort operations of various kinds on them
> : to order the data in ways that allows consistencies to be checked.
> : It will take a bit longer than a RAM fsck but maybe not as much as
> : one might fear.
> : We all remember those "sort a mag-tape larger than RAM"
> : lessons from CS101 don't we?
> : At least it doesn't have to be "in place" so merge sorts are OK. :-)
> : 
> : why?
> : 
> : A bitmap of 1TB of 512 byte records is 244MB so with a 4BG machine
> : with 3GB available to the process you can't even fit the bitmaps into
> : memory for a 12TB Filesystem let alone other metadata.
> : 
> : Going to 2048 byte frags helps but you still run into a limit.
> : last I tried it, you need about 600MB per TB of fileysstem to check.
> : 
> : So I think a special fsck that uses files is a must for really big
> : filesystems, unless they (the filesystems) can be broken up in
> : a logical way (IBM did that many years ago I believe).
> : I think you should add that to your list.
> 
> I think that a big amount of this could be reduced by using simple
> arrays rather than lists which are more memory efficient...
> 
> Warner

but that just puts the problem a bit further away.