Date: Fri, 10 Dec 1999 11:19:15 -0500 (EST) From: Robert Watson <robert@cyrus.watson.org> To: David Malone <dwmalone@maths.tcd.ie> Cc: Zhihui Zhang <zzhang@cs.binghamton.edu>, freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org Subject: Re: Why VMIO directory is a bad idea? Message-ID: <Pine.BSF.3.96.991210111043.34185E-100000@fledge.watson.org> In-Reply-To: <19991210154459.A1034@hamilton.maths.tcd.ie>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 10 Dec 1999, David Malone wrote: > On Fri, Dec 10, 1999 at 08:56:47AM -0500, Zhihui Zhang wrote: > > > Can anyone give me an idea on how big a directory could be in some > > environment? > > Our inn's /news/spool/control/cancel directory is almost 300k. If > we were a significantly larger news site we probably wouldn't be > running inn though. > I use the CMU cyrus server to manage mail for my system, and also to handle mailing lists archives. I have an archive of the freebsd-current mailing list that contains around 33,000 messages, and with one file per message the directory size is about 530k. The big hit I see is Cyrus's use of dbm to manage index information, not the cost of directory operations. Typically, due to the way IMAP is usually used, no one reads in all the files, etc. However, performing an ls is non-trivially expensive: # time ls -af > /dev/null 1.018u 0.241s 0:01.25 100.0% 225+4365k 0+0io 0pf+0w 1.012u 0.288s 0:01.29 100.0% 225+4352k 0+0io 0pf+0w 1.053u 0.280s 0:01.33 100.0% 223+4273k 0+0io 0pf+0w 1.076u 0.253s 0:01.32 100.0% 219+4238k 0+0io 0pf+0w 1.091u 0.235s 0:01.33 99.2% 224+4273k 0+0io 0pf+0w # ls -af | wc -l 33323 This is under 2.2-STABLE, although I hope to push it to a 3.3-STABLE machine in the near future. This machine is currently a 486 dx2 66 w/24 mb of ram. It will become a Pentium sometime soon, with more memory. It should be observed, for the benefit of critics, that storing a mailbox in this format is far better from a performance perspective than storing all the messages in a single file :-). But it goes through a bunch of inodes (it almost justifies the default inode allocate on large disks :-), and does have drawbacks in terms of directory size. Because messages are hardly ever removed from this directory, my guess is that it's use of the directory space is fairly compact and unfragmented. Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-fs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.3.96.991210111043.34185E-100000>