From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 29 02:10:23 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E236C16A41F for ; Thu, 29 Sep 2005 02:10:22 +0000 (GMT) (envelope-from mwm-dated-1128823868.97bd69@mired.org) Received: from delight.idiom.com (outbound.idiom.com [216.240.47.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8E1BB43D49 for ; Thu, 29 Sep 2005 02:10:22 +0000 (GMT) (envelope-from mwm-dated-1128823868.97bd69@mired.org) Received: from idiom.com (idiom.com [216.240.32.1]) by delight.idiom.com (Postfix) with ESMTP id ACD031F9489 for ; Wed, 28 Sep 2005 19:10:21 -0700 (PDT) Received: from mired.org (mwm@idiom [216.240.32.1]) by idiom.com (8.12.11/8.12.11) with SMTP id j8T2AJ2T072337 for ; Wed, 28 Sep 2005 19:10:20 -0700 (PDT) (envelope-from mwm-dated-1128823868.97bd69@mired.org) Received: (qmail 35465 invoked by uid 1001); 29 Sep 2005 02:11:09 -0000 Received: by localhost.mired.org (tmda-sendmail, from uid 1001); Wed, 28 Sep 2005 22:11:08 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17211.19772.562587.908715@bhuda.mired.org> Date: Wed, 28 Sep 2005 22:11:08 -0400 To: aanton@smtpx.spintech.ro In-Reply-To: <433B3F41.8060004@spintech.ro> References: <433B3F41.8060004@spintech.ro> X-Mailer: VM 7.17 under 21.4 (patch 17) "Jumbo Shrimp" XEmacs Lucid X-Primary-Address: mwm@mired.org X-face: "5Mnwy%?j>IIV\)A=):rjWL~NB2aH[}Yq8Z=u~vJ`"(,&SiLvbbz2W`; h9L,Yg`+vb1>RG% *h+%X^n0EZd>TM8_IB;a8F?(Fb"lw'IgCoyM.[Lg#r\ X-Delivery-Agent: TMDA/1.0.3 (Seattle Slew) From: Mike Meyer X-Mailman-Approved-At: Thu, 29 Sep 2005 11:54:53 +0000 Cc: freebsd-hackers@freebsd.org Subject: Re: journaling fs and large mailbox format X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2005 02:10:23 -0000 In <433B3F41.8060004@spintech.ro>, Alin-Adrian Anton typed: > I am in the position to implement a large-scale mail server and I will > never go for anything else but FreeBSD (fixation?). How are users going to get to the mail? Web browsers? IMAP? POP? > I don't know if the mbox format can handle this, and I know Maildir > cannot handle this on UFS2 standard install, no matter of soft-updates. > (because it exhaustes the free nodes) So I currently have no solution > for this stuff. Large mailboxes in mbox format are a loss, because you have to rewrite all/most of the mailbox to make changes in it. In particular, a fetchmail process - read and delete the messages in chronological order - is an O(n**2) process. This applies to pretty much any mail format that stores all the messages in one file. The alternatives are things like mh and Mailder. Yes, those will have problems on UFS file systems that you build with the default parameters. That's because mail messages tend to be very small files - typically a few k - so a file system full of them is will have a very odd distribution of files sizes when compared to more usual file systems. The solution isn't to avoid Maildir/mh - the solution is to tune the file system for the expected usage. FreeBSD (and Unix in general) gives you lots of knobs to deal with things like this. Learn to use them. The default block and frag size are relatively large - 2K and 16K appear to be the defaults on 5.x. A quick look at my mail for 2005 shows 32,267 messages ranging from 280 bytes to 6+ meg, with a mean size of less than 8K. I'd go with 4k blocks and a 512 byte frag size - because that will give you four times as many inodes as the default parameters. 8K/1K is also tempting, but you'll probably want to specify -i 2048 to get the same number of inodes as you get with a 4K/512b file system. > I was wondering what is the status of Journaling File Systems on > FreeBSD? Any which is usable and mature, with write access? XFS would > fit amazingly well with Maildir, but.. I doubt it's anything else but > readonly. Neither journaling nor soft updates would solve the problem of running out of inodes. The only solution there is more inodes. XFS may be flexible enough to deal with file systems that far from the norm - but I wouldn't write a business plan based on it without checking first. http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information.