From owner-freebsd-hackers Fri Dec 10 8:19:39 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id C6CD4152E9; Fri, 10 Dec 1999 08:19:32 -0800 (PST) (envelope-from robert@cyrus.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id LAA34614; Fri, 10 Dec 1999 11:19:15 -0500 (EST) (envelope-from robert@cyrus.watson.org) Date: Fri, 10 Dec 1999 11:19:15 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org Reply-To: Robert Watson To: David Malone Cc: Zhihui Zhang , freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org Subject: Re: Why VMIO directory is a bad idea? In-Reply-To: <19991210154459.A1034@hamilton.maths.tcd.ie> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Fri, 10 Dec 1999, David Malone wrote: > On Fri, Dec 10, 1999 at 08:56:47AM -0500, Zhihui Zhang wrote: > > > Can anyone give me an idea on how big a directory could be in some > > environment? > > Our inn's /news/spool/control/cancel directory is almost 300k. If > we were a significantly larger news site we probably wouldn't be > running inn though. > I use the CMU cyrus server to manage mail for my system, and also to handle mailing lists archives. I have an archive of the freebsd-current mailing list that contains around 33,000 messages, and with one file per message the directory size is about 530k. The big hit I see is Cyrus's use of dbm to manage index information, not the cost of directory operations. Typically, due to the way IMAP is usually used, no one reads in all the files, etc. However, performing an ls is non-trivially expensive: # time ls -af > /dev/null 1.018u 0.241s 0:01.25 100.0% 225+4365k 0+0io 0pf+0w 1.012u 0.288s 0:01.29 100.0% 225+4352k 0+0io 0pf+0w 1.053u 0.280s 0:01.33 100.0% 223+4273k 0+0io 0pf+0w 1.076u 0.253s 0:01.32 100.0% 219+4238k 0+0io 0pf+0w 1.091u 0.235s 0:01.33 99.2% 224+4273k 0+0io 0pf+0w # ls -af | wc -l 33323 This is under 2.2-STABLE, although I hope to push it to a 3.3-STABLE machine in the near future. This machine is currently a 486 dx2 66 w/24 mb of ram. It will become a Pentium sometime soon, with more memory. It should be observed, for the benefit of critics, that storing a mailbox in this format is far better from a performance perspective than storing all the messages in a single file :-). But it goes through a bunch of inodes (it almost justifies the default inode allocate on large disks :-), and does have drawbacks in terms of directory size. Because messages are hardly ever removed from this directory, my guess is that it's use of the directory space is fairly compact and unfragmented. Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message