From owner-freebsd-questions@FreeBSD.ORG Thu Jul 28 19:34:07 2005 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E21F716A41F for ; Thu, 28 Jul 2005 19:34:07 +0000 (GMT) (envelope-from lowell@be-well.ilk.org) Received: from mail26.sea5.speakeasy.net (mail26.sea5.speakeasy.net [69.17.117.28]) by mx1.FreeBSD.org (Postfix) with ESMTP id 980AB43D48 for ; Thu, 28 Jul 2005 19:34:07 +0000 (GMT) (envelope-from lowell@be-well.ilk.org) Received: (qmail 5103 invoked from network); 28 Jul 2005 19:34:07 -0000 Received: from dsl092-078-145.bos1.dsl.speakeasy.net (HELO be-well.ilk.org) ([66.92.78.145]) (envelope-sender ) by mail26.sea5.speakeasy.net (qmail-ldap-1.03) with SMTP for ; 28 Jul 2005 19:34:07 -0000 Received: by be-well.ilk.org (Postfix, from userid 1147) id 7BDBC2F; Thu, 28 Jul 2005 15:34:06 -0400 (EDT) Sender: lowell@be-well.ilk.org To: freebsd-questions@freebsd.org References: <060c01c59397$2876b460$c901a8c0@workdog> From: Lowell Gilbert Date: 28 Jul 2005 15:34:06 -0400 Message-ID: <44ek9iu3m9.fsf@be-well.ilk.org> Lines: 54 User-Agent: Gnus/5.09 (Gnus v5.9.0) Emacs/21.3 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Subject: Re: defragmentation in FreeBSD 4.11 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 28 Jul 2005 19:34:08 -0000 Freminlins writes: > Gayn Winters wrote: > > > What I get from reading this article is that if the use of the file > > system is to store lots of small files, then use a small block size. Am > > I missing something? > > No and yes! There is a minimum block and fragment size. In this case > there were not enough contiguous fragments to enable an 8K file to be > created. Without checking I believe Solaris uses 8K blocks. That's correct, and also note that Solaris does not offer the flexibility in block sizes that is available from FreeBSD. And it is correct that a small block size will vastly reduce the ability to hit this fully-fragmented condition. At a cost of higher overhead, of course. > > Also, in most situations, buying a big enough disk is far better than > > worrying about what happens when a not-big-enough disk starts to get > > full. > > Indeed. But... in the case I linked to there was apparently plenty of > free space, just not enough free contiguous space. The author also > implies that a bigger disk would not solve the problem: > > "it creates and deletes tons of small files and thus the fragmentation > over a period of time." Where "plenty of free space" is a large absolute number, but still a small fraction of the disk. Full fragmentation of the empty space becomes exponentially less likely as the amount of free space increases. Note that when deciding where to allocate a fragment, there are two possible policies the filesystem can follow: either a space-efficient method that is more likely to require moving data when a file expands, or a time-efficient method that will create new fragments more often. FreeBSD can automatically shift between the two as the filesystem moves over or under a threshold "percent full" value. Solaris, I believe, needs to be explicitly changed between the two. > As mentioned though I have never seen this myself despite running very > busy mail and web servers with what must be billions of files being > created/deleted in that time. > > It certainly grabbed my attention so I thought it may be of interest to others. It's an academic curiosity, but not an issue to take into account in planning your data center. Planning your filesystem layout, however, may well be. Be well.