From owner-freebsd-questions Thu Jan 2 6:13:31 2003 Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4976537B401 for ; Thu, 2 Jan 2003 06:13:30 -0800 (PST) Received: from mail.bellavista.cz (mail.bellavista.cz [62.168.44.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id 17AE643E4A for ; Thu, 2 Jan 2003 06:13:29 -0800 (PST) (envelope-from neuhauser@bellavista.cz) Received: from freepuppy.bellavista.cz (freepuppy.bellavista.cz [10.0.0.10]) by mail.bellavista.cz (Postfix) with ESMTP id 7FCDC2D3; Thu, 2 Jan 2003 15:13:24 +0100 (CET) Received: by freepuppy.bellavista.cz (Postfix, from userid 1001) id CA97F2FDB18; Thu, 2 Jan 2003 15:13:22 +0100 (CET) Date: Thu, 2 Jan 2003 15:13:22 +0100 From: Roman Neuhauser To: MikeM Cc: freebsd-questions@FreeBSD.ORG Subject: Re: Lots of files in a directory Message-ID: <20030102141322.GH348@freepuppy.bellavista.cz> Mail-Followup-To: MikeM , freebsd-questions@FreeBSD.ORG References: <200301020901270548.09C1C68B@sentry.24cl.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200301020901270548.09C1C68B@sentry.24cl.com> User-Agent: Mutt/1.5.1i Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG # MyRaQ@mgm51.com / 2003-01-02 09:01:27 -0500: > I'm considering setting up my server as a mirror site for the freedb.org > lookup database. Unfortunately, I've seem to have run into a stumbling > block. The server app requires over 250,000 files in a single directory. > Each file is about 2k in size. > > It was a surprise to me (but probably not to those on this list) that the > file system does not handle that many files in an expeditious manner (I'm > being kind here). > > Is there anything I can do so that the file system works faster with such a > large number of files? I'm looking for an increase in the area of 5 to 1. > For example, the command "rm -rf misc" where "misc" is the directory > containing the 250,000 files takes a couple of hours to run. If "misc" is > my current working directory, and I type "rm *" I get the message that > there are too many arguments being passed into rm. > > Some details: > FreeBSD 4.5, dmesg below. > Filesystem containing the files: /dev/da0s1f on /usr (ufs, local, with > quotas) would this help? (I don't know if this was available in 4.5, look in your /sys/i386/conf/LINT) roman@freepuppy ~ 1003:0 > grep -B2 DIRHASH /sys/i386/conf/LINT # Directory hashing improves the speed of operations on very large # directories at the expense of some memory. options UFS_DIRHASH -- If you cc me or remove the list(s) completely I'll most likely ignore your message. see http://www.eyrie.org./~eagle/faqs/questions.html To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message