From owner-freebsd-fs@FreeBSD.ORG Fri Jan 9 16:56:46 2004 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6737616A4CE for ; Fri, 9 Jan 2004 16:56:46 -0800 (PST) Received: from carver.gumbysoft.com (carver.gumbysoft.com [66.220.23.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id BEA1C43D2D for ; Fri, 9 Jan 2004 16:56:39 -0800 (PST) (envelope-from dwhite@gumbysoft.com) Received: by carver.gumbysoft.com (Postfix, from userid 1000) id B199572DBF; Fri, 9 Jan 2004 16:56:39 -0800 (PST) Received: from localhost (localhost [127.0.0.1]) by carver.gumbysoft.com (Postfix) with ESMTP id AF2C472DB5; Fri, 9 Jan 2004 16:56:39 -0800 (PST) Date: Fri, 9 Jan 2004 16:56:39 -0800 (PST) From: Doug White To: Tom Arnold In-Reply-To: <20040108002627.GH22041@moo.sysabend.org> Message-ID: <20040109165514.X24587@carver.gumbysoft.com> References: <20040108002627.GH22041@moo.sysabend.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-fs@freebsd.org Subject: Re: Large filesystems with massive inodes on BSD5.2 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 10 Jan 2004 00:56:46 -0000 On Wed, 7 Jan 2004, Tom Arnold wrote: > I'm building a fileserver thats going to house a large number of small > files ( no, not an innd server but same idea sorta ). > I've got a 3.3TB RAID hanging off a FreeBSD 5.2-RC box ( cvsup'd to RELENG_5_2 > monday night ). > I've broken it up into 2 1TB filesystems newfs'd "stock" and the remainder > a 1TB partition and a 300gig partition. wow. :) > Now the fun part. > newfs'd the final two partitions -b 4096 -f 512 -i 512 as they are going to > have many many small files on them. Needless to say if the server reboots > unexpectedly fsck takes a long long to run. So this is why for a production > server I'm playing with 5.2 Unless you're going to fill the filesystem with 512 byte files, you probably have the sizes set too low. You're probably OK with the default of 4096 bytes/inode. > Newfs'd UFS2 softupdates enabled. Crash the box to force a dirty reboot. > Boots finally after about 20 minutes of playing with snapshots. About > 4 hours later I get this lovely error : > > Jan 7 02:58:12 tarnold2 fsck: /dev/da1s1e: CANNOT CREATE SNAPSHOT /export/database/.snap/fsck_snapshot: File > too large > Jan 7 02:58:12 tarnold2 fsck: > Jan 7 02:58:12 tarnold2 fsck: /dev/da1s1e: UNEXPECTED INCONSISTENCY; RUN fsck MANUALLY. > Jan 7 20:21:24 tarnold2 su: tarnold to root on /dev/ttyp0 > > da1s1e is the largest or the two filesystems with huge numbers of inodes. Heh. Thats funny. I'd like to see Kirk's reaction to that. > Am I running into some kind of filesystem limits by having 2billion inodes > on it? Am I even asking on the correct mailing list or should I ask this > on -Current? This is the right spot. Interesting that fsck fell over creating the snapshot due to E2BIG. -- Doug White | FreeBSD: The Power to Serve dwhite@gumbysoft.com | www.FreeBSD.org