From owner-freebsd-hackers Wed May 26 21:23:30 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from alpo.whistle.com (alpo.whistle.com [207.76.204.38]) by hub.freebsd.org (Postfix) with ESMTP id 69A7014E17 for ; Wed, 26 May 1999 21:23:27 -0700 (PDT) (envelope-from julian@whistle.com) Received: from current1.whistle.com (current1.whistle.com [207.76.205.22]) by alpo.whistle.com (8.9.1a/8.9.1) with SMTP id VAA68670; Wed, 26 May 1999 21:23:25 -0700 (PDT) Message-ID: <374CC8BB.2F1CF0FB@whistle.com> Date: Wed, 26 May 1999 21:23:23 -0700 From: Julian Elischer Organization: Whistle Communications X-Mailer: Mozilla 3.0Gold (X11; I; FreeBSD 2.2.8-STABLE i386) MIME-Version: 1.0 To: David Scheidt Cc: Graeme Tait , freebsd-hackers@FreeBSD.ORG, info@boatbooks.com Subject: Re: File system gets too fragmented ??? References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG I missed the original post, so may be mislead.... (The extract below doesn't give all the symptoms) David Scheidt wrote: > > On Wed, 26 May 1999, Graeme Tait wrote: > > > It contains about 900,000 files, most of which are small, occupying > > around 2-5 fragments. The small files are updated monthly from a tar > > > > > However, I don't understand how the FFS works, so I'm just probing and > > guessing as to what's going on here. > > > > Could someone please shed a little light on this? Is FreeBSD not able to > > self-manage a filesystem operated in this way? Is there some way of > > preventing the problem, or of periodically cleaning things up (rebuilding > > the whole filesystem from backup means being down for over an hour on a > > 24x7 server)? > > The problem you have is that a file on FFS file system can have at most one > fragmented block. With the number of small files that you have, it isn't > terribly suprising that you are running out of full blocks when there is > still space free on the disk. I don't think there is a whole lot you can do > about the problem. A 1024B frag size might mask the problem of having space > free, but with no aligned blocks, not allocatable. You should also think > hard about why you need a million 1536 byte files on the same filesystem. I > don't know what you are doing, but you might consider a real database. > > David Scheidt > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message if each file uses 5 frags and there are 8 frags to a block then there will be only 3-frag long freespace clusters. thus each file will essentially use 8 frags because the other 3 will not be usable by any files you have. I don't see if you have a 4K or 8K blocksize and what your fragsize is. let me see.. if you are talking about 512 byte frags, and 4k blocks, with average files taking 2.5K, then you might consider a 8K blocksize, as you would on average get 2 files per block, each of 3 frags (and an ocasional one taking the last 2K) you might also experiment with 16K/2K Counter intuitive I know.... As David said.. if we knew more about what you are doing...... Oh yeah... 1 million 3KB files is only 3GB.... an 18GB disk would also solve your problem :-) julian To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message