From owner-freebsd-hackers Mon Nov 22 10:30:26 1999 Delivered-To: freebsd-hackers@freebsd.org Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by hub.freebsd.org (Postfix) with ESMTP id B484015951 for ; Mon, 22 Nov 1999 10:30:13 -0800 (PST) (envelope-from robert@cyrus.watson.org) Received: from fledge.watson.org (robert@fledge.pr.watson.org [192.0.2.3]) by fledge.watson.org (8.9.3/8.9.3) with SMTP id NAA30443; Mon, 22 Nov 1999 13:29:44 -0500 (EST) (envelope-from robert@cyrus.watson.org) Date: Mon, 22 Nov 1999 13:29:44 -0500 (EST) From: Robert Watson X-Sender: robert@fledge.watson.org Reply-To: Robert Watson To: Alan Judge Cc: freebsd-hackers@freebsd.org Subject: Re: vmpfw in pine via NFS In-Reply-To: <19991122153656.DD5F614BD2@hub.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: owner-freebsd-hackers@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG On Mon, 22 Nov 1999, Alan Judge wrote: > Daniel> /me shivers at the thought of my (easily) 500+ new messages a day > Daniel> and hundreds of thousands of messages being stored one file for each > Daniel> message... > > Works OK for us (and a number of even larger ISPs using Maildirs). > Though we use NetApps for the file storage and they have a much better > system for storing large numbers of files in a directory, so it > doesn't get quadratically slower. Largest single FS we have at the > moment has about 3.5 million files in it; I don't know what the > largest number of files in a single directory is, but I've seen 10s of > thousands on occasion without problems. It works much better than a > large file per user and NFS file locking, which makes me shiver. I agree that the one-file-per-message things scales fine--I have a fairly dinky 486 w/24mb of RAM running my cyrus server, and it does just fine with a lot of files for a fair number of messages: Filesystem 1K-blocks Used Avail Capacity iused ifree %iused Mounted on /dev/wd1s1g 316999 45502 246138 16% 7224 69574 9% /usr/var/spool/imap2 /dev/wd1s1f 496367 163672 292986 36% 27949 94929 23% /usr/var/spool/imap /dev/wd2s1e 19403838 838047 17013484 5% 179066 4513412 4% /usr/var/spool/imap3 In fact, the whole thing scales a *lot* better than a single file per folder, as a lot less time is spent seeking through large mailfiles looking for arbitrarily located string boundaries, scanning through attachments, etc. I'm about to upgrade my cyrus server, but not because of performance problems :-). Robert N M Watson robert@fledge.watson.org http://www.watson.org/~robert/ PGP key fingerprint: AF B5 5F FF A6 4A 79 37 ED 5F 55 E9 58 04 6A B1 TIS Labs at Network Associates, Safeport Network Services To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message