From owner-freebsd-questions@FreeBSD.ORG Sun Nov 9 02:40:49 2008 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2B7DD106567B for ; Sun, 9 Nov 2008 02:40:49 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from QMTA07.westchester.pa.mail.comcast.net (qmta07.westchester.pa.mail.comcast.net [76.96.62.64]) by mx1.freebsd.org (Postfix) with ESMTP id CD15B8FC08 for ; Sun, 9 Nov 2008 02:40:48 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from OMTA08.westchester.pa.mail.comcast.net ([76.96.62.12]) by QMTA07.westchester.pa.mail.comcast.net with comcast id cfvm1a0020Fqzac57qgDs7; Sun, 09 Nov 2008 02:40:13 +0000 Received: from koitsu.dyndns.org ([69.181.141.110]) by OMTA08.westchester.pa.mail.comcast.net with comcast id cqgn1a0032P6wsM3UqgnuB; Sun, 09 Nov 2008 02:40:48 +0000 X-Authority-Analysis: v=1.0 c=1 a=QycZ5dHgAAAA:8 a=rTD7BgE3feIFTYgHgW8A:9 a=wUgJOANk20RqicLGvcAA:7 a=ljUMr3YkEcnFBTKOQ9u3jbXj-x4A:4 a=EoioJ0NPDVgA:10 a=LY0hPdMaydYA:10 Received: by icarus.home.lan (Postfix, from userid 1000) id EE7345C19; Sat, 8 Nov 2008 18:40:46 -0800 (PST) Date: Sat, 8 Nov 2008 18:40:46 -0800 From: Jeremy Chadwick To: no-spam@people.net.au Message-ID: <20081109024046.GB27423@icarus.home.lan> References: <50261.1226194851@people.net.au> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <50261.1226194851@people.net.au> User-Agent: Mutt/1.5.18 (2008-05-17) Cc: freebsd-questions@freebsd.org Subject: Re: UFS2 limits X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 09 Nov 2008 02:40:49 -0000 On Sun, Nov 09, 2008 at 01:40:51AM +0000, no-spam@people.net.au wrote: > Hi, > I have a FreeBSD server that has about 10,500 subdirectories within a single > directory. > This number will keep rising and I assume UFS2 has a limit to the number of > sub-directories in a single directory - can anyone tell me what it is? As far as I know, there is no such limit on the number of files/dirs inside of a directory. I don't want to change the topic of discussion, but I *highly* recommend you ***stop*** whatever it is you're doing that is creating such a directory structure. Software which has to iterate through that directory using opendir() and readdir() will get slower and slower as time goes on. If this is something you've written or have control over or can work with engineers in regards to, I recommend you change your directory naming scheme to have separate subdirectories with the first 2 or 3 letters of the directory you wish to create. E.g.: /some/place/00/00ilikezeros/* /some/place/01/01binaryheaven/* /some/place/aa/aardvarks/* /some/place/ab/abuse/* /some/place/ac/actuary/* ... /some/place/xy/xylophones/* You get the point. Traversing this structure is much more efficient, and requires very little code change on your part. Those who run nameservers that host many zones, for example, use this structure to ensure the daemon doesn't take 32498231 years to start up. > What about ZFS? > > At some point I'll have to re-arrange things so that I have a deeper directory > structure, just wondering when I'll hit the limit so I can plan in advance :-) What baffles me is why you're looking at this problem from a " "how can the filesystem solve this engineering mistake I made for me" standpoint, rather than "how can I solve this engineering mistake I made so that it doesn't impact the filesystem". Very strange. Sometimes looking at things in a different light makes all the difference. Hope this helps. P.S. -- I hope this mail makes it to you, because your From line is no-spam@people.net.au (I'll be surprised if your account name really is that!). -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |