From owner-freebsd-hackers@FreeBSD.ORG Thu Sep 29 21:28:53 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81A0316A41F for ; Thu, 29 Sep 2005 21:28:53 +0000 (GMT) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mail2.fluidhosting.com [204.14.90.62]) by mx1.FreeBSD.org (Postfix) with SMTP id 08DAD43D49 for ; Thu, 29 Sep 2005 21:28:52 +0000 (GMT) (envelope-from dougb@FreeBSD.org) Received: (qmail 11469 invoked by uid 399); 29 Sep 2005 21:28:52 -0000 Received: from mail1.fluidhosting.com (204.14.90.61) by mail2.fluidhosting.com with SMTP; 29 Sep 2005 21:28:52 -0000 Received: (qmail 49763 invoked by uid 399); 29 Sep 2005 21:28:51 -0000 Received: from localhost (HELO ?192.168.1.100?) (dougb@dougbarton.net@127.0.0.1) by localhost with SMTP; 29 Sep 2005 21:28:51 -0000 Message-ID: <433C5C8B.6000003@FreeBSD.org> Date: Thu, 29 Sep 2005 14:28:43 -0700 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Mozilla Thunderbird 1.0.6 (X11/20050908) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Mike Meyer References: <433B3F41.8060004@spintech.ro> <17211.19772.562587.908715@bhuda.mired.org> In-Reply-To: <17211.19772.562587.908715@bhuda.mired.org> X-Enigmail-Version: 0.92.0.0 Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-hackers@freebsd.org, aanton@smtpx.spintech.ro Subject: Re: journaling fs and large mailbox format X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 29 Sep 2005 21:28:53 -0000 Mike Meyer wrote: > The solution isn't to avoid Maildir/mh - the solution is to tune the > file system for the expected usage. FreeBSD (and Unix in general) > gives you lots of knobs to deal with things like this. Learn to use > them. > > The default block and frag size are relatively large - 2K and 16K > appear to be the defaults on 5.x. A quick look at my mail for 2005 > shows 32,267 messages ranging from 280 bytes to 6+ meg, with a mean > size of less than 8K. I'd go with 4k blocks and a 512 byte frag size - > because that will give you four times as many inodes as the default > parameters. 8K/1K is also tempting, but you'll probably want to > specify -i 2048 to get the same number of inodes as you get with a > 4K/512b file system. I agree generally with your thinking here, and wanted to add a little more analysis based on my experience. When I was facing a similar problem with a large authoritative name server installation, I got the advice to find the median file size, and tune the file system so that the block size was 2x the median file size (with the fragment size being 1/8th the block size of course). The reasoning behind this was that because the files I was working with could grow, it made sense to make sure that even if it grew the file could stay within one block. This is slightly wasteful of space (very slightly), but resulted in a much more efficient operation. In this situation, since any given file in a maildir directory is very unlikely to grow, it seems to me to make sense that the right way to tune the fs would be to find the median file size and make the block size large enough to handle files of that size. That should give you the right tradeoff between speed and efficiency. hth, Doug -- This .signature sanitized for your protection