From owner-freebsd-hackers Thu May 27 3:41:54 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by hub.freebsd.org (Postfix) with ESMTP id 7808314C0B for ; Thu, 27 May 1999 03:41:52 -0700 (PDT) (envelope-from julian@whistle.com) Received: from current1.whistle.com (current1.whistle.com [207.76.205.22]) by alpo.whistle.com (8.9.1a/8.9.1) with SMTP id DAA75646; Thu, 27 May 1999 03:41:40 -0700 (PDT) Date: Thu, 27 May 1999 03:41:40 -0700 (PDT) From: Julian Elischer To: Graeme Tait Cc: David Scheidt , freebsd-hackers@FreeBSD.ORG, info@boatbooks.com Subject: Re: File system gets too fragmented ??? In-Reply-To: <374D356F.80E@echidna.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Thu, 27 May 1999, Graeme Tait wrote: > > I've received several answers along this direction, but I want to emphasize one > point that I think is being overlooked. When the filesystem is fresh and a new > archive is expanded to create ~900,000 small files each of 2-5 512 byte frags > in size, the filesystem appears quite well-behaved, and space seems to be > efficiently utilized. > > The problem seems to be that with successive updates that slightly change the > size of files, or add or delete files, that a large number of unallocated > fragments are created. imagine: [A][A][B][B][B][C][C][C] <-- 8 frags in 3 files delete a bunch of small files.. (position A or B) Get a bunch of fractionally larger files They aint going to fit.. need a new block... of course it Is posible that where could be a bug where we don't reallocate frags that are early in a block, but I find it a bit hard to believe.. julian > > I don't understand how the FFS stores files of sub-block size. Do the > fragments used need to be contiguous, or entirely within a single block? The excess over an integral number of blocks (at the tail of the file) is sized, and a single block with that number of contiguous frag blocks free is sought and allocated. up to 8 files may use frags in the same block, but each can only have 1 contiguous range of frags at it's end. julian > > > The choice of 512 byte frags is based on average wastage per file of > half a frag, or about 230MB with 900,000 files. It's quite possible that > a 2k frag/16k blocksize would improve utilization of fragments, as the > vast majority of files would then fit in a single fragment, but in this > case there would be of order 800MB wastage, and the files would not fit > the existing disk. sounds a fair tradeoff to me.. a 4GB disk is what? $200? > > > BTW, I realize there is probably a better way to do this (database, > etc.), and we are thinking in that direction, but for the moment we have > a legacy setup and not much time to rework it. Also, I would have > thought that having many tiny files was not that unusual (e.g., a news > spool). > > > -- > Graeme Tait - Echidna > > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message