From owner-freebsd-questions Tue Mar 11 7: 9:41 2003 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7C1EA37B401 for ; Tue, 11 Mar 2003 07:09:39 -0800 (PST) Received: from pa-plum1b-166.pit.adelphia.net (pa-plum1b-13.pit.adelphia.net [24.53.161.13]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7CA5943F85 for ; Tue, 11 Mar 2003 07:09:38 -0800 (PST) (envelope-from wmoran@potentialtech.com) Received: from potentialtech.com (working [172.16.0.95]) by pa-plum1b-166.pit.adelphia.net (8.12.7/8.12.7) with ESMTP id h2BF9TTb006754; Tue, 11 Mar 2003 10:09:30 -0500 (EST) (envelope-from wmoran@potentialtech.com) Message-ID: <3E6DFC65.4050205@potentialtech.com> Date: Tue, 11 Mar 2003 10:10:29 -0500 From: Bill Moran User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.2.1) Gecko/20030301 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Dan Nelson Cc: questions@FreeBSD.ORG Subject: Re: What is a practical max nujmber of files in a directory References: <3E6D6A0F.5040704@potentialtech.com> <20030311053714.GG34011@dan.emsphone.com> In-Reply-To: <20030311053714.GG34011@dan.emsphone.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG Dan Nelson wrote: > In the last episode (Mar 10), Bill Moran said: > >>Question for the gurus or anyone who has done any test on filesystem >>performance. >> >>Where is the point at which a directory has too many files in it? >>Mainly with regard to performance degredation? > > It Depends > > http://www.cnri.dit.ie/Downloads/fsopt.pdf has a nice rundown of the > performance benefits of softupdates, dirhash, dirpref, and vmiodir for > different benchmarks (including tests on directories with up to 20000 > files). vmiodir and dirpref are now on by default, but the results for > softupdates and dirhash are still useful. Thanks for the input ... that was definately an interesting article. I would have expected dirhash to do more but ... Anyway, it still wasn't exactly what I was looking for. It occurred to me that I should just describe what I'm doing .... I'm writing a web-interface to file-sharing. It's back-ended by a metadata database and a filesystem based file store. I've decided (based on my experience with software such as squid and huge directories resulting from unpacking the php documentation) to split the file store up between directories so that the directory listings never get too big. The question I need to answer is: how many files can a directory contain before the system determines that no more files should be placed in it? The database will determine where and what file to retrieve, so I don't need to worry about the listing getting too long to be easily managed by command-line tools, but I want to make sure that finding a specific file in that directory doesn't get too awfully time-consuming. -- Bill Moran Potential Technologies http://www.potentialtech.com To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message