From owner-freebsd-hackers@FreeBSD.ORG Fri Sep 30 03:13:11 2005 Return-Path: X-Original-To: freebsd-hackers@freebsd.org Delivered-To: freebsd-hackers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E4CD316A41F for ; Fri, 30 Sep 2005 03:13:10 +0000 (GMT) (envelope-from mwm-keyword-hackers.e471b2@mired.org) Received: from delight.idiom.com (outbound.idiom.com [216.240.47.194]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8CAF043D48 for ; Fri, 30 Sep 2005 03:13:06 +0000 (GMT) (envelope-from mwm-keyword-hackers.e471b2@mired.org) Received: from idiom.com (idiom.com [216.240.32.1]) by delight.idiom.com (Postfix) with ESMTP id 923032257A7 for ; Thu, 29 Sep 2005 20:13:05 -0700 (PDT) Received: from mired.org (mwm@idiom [216.240.32.1]) by idiom.com (8.12.11/8.12.11) with SMTP id j8U3D37L055319 for ; Thu, 29 Sep 2005 20:13:05 -0700 (PDT) (envelope-from mwm-keyword-hackers.e471b2@mired.org) Received: (qmail 89325 invoked by uid 1001); 30 Sep 2005 03:13:48 -0000 Received: by localhost.mired.org (tmda-sendmail, from uid 1001); Thu, 29 Sep 2005 23:13:48 -0400 (EDT) MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Message-ID: <17212.44395.869528.437031@bhuda.mired.org> Date: Thu, 29 Sep 2005 23:13:47 -0400 To: aanton@spintech.ro In-Reply-To: <433C9A64.3030602@spintech.ro> References: <433B3F41.8060004@spintech.ro> <433B60EE.4090207@centtech.com> <433C9A64.3030602@spintech.ro> X-Mailer: VM 7.17 under 21.4 (patch 17) "Jumbo Shrimp" XEmacs Lucid X-Primary-Address: mwm@mired.org X-face: "5Mnwy%?j>IIV\)A=):rjWL~NB2aH[}Yq8Z=u~vJ`"(,&SiLvbbz2W`; h9L,Yg`+vb1>RG% *h+%X^n0EZd>TM8_IB;a8F?(Fb"lw'IgCoyM.[Lg#r\ X-Delivery-Agent: TMDA/1.0.3 (Seattle Slew) From: Mike Meyer Cc: freebsd-hackers@freebsd.org Subject: Re: journaling fs and large mailbox format X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 Sep 2005 03:13:11 -0000 In <433C9A64.3030602@spintech.ro>, Alin-Adrian Anton typed: > I run out of inodes with Maildir, and there were just a few hundred > accounts. Outlook ppl tend to "leave their messages on server if they > are not 7 days old" and this brings Christmas every day. How many files was that, and on how big a file system? Something seems out of kilter. > Mike Meyer wrote: > > The solution isn't to avoid Maildir/mh - the solution is to tune the > file system for the expected usage. > > Well, I dislike throwing up my problems to a superior level, and act > like it was brilliant. It was just running away from the issue, instead > of dealing with it. More exactly, storage problems are database theory. > Storing the mail is a classic database problem. Throwing this up to the > filesystem level is not an elegant way of dealing with it, because now > the filesystem must solve it, and this imposes new restrictions to the > filesystem. I hate to tell you this, but a file system *is* a database. Unix file systems tend to be pretty simple databases, but that's not true on all systems. Using the file system in lieue of a more complicated database - if it will work - is a time-honored unix technic. I keep a couple of gig of mail archived, and let the file system deal with sorting it out by date. Someone's got to solve the problems. If you can find an existing tool to do it for you, that's brilliant, whether the tool is a file system, a database, or a custom application. But there are tradeoffs to each such solution, and you're the only one who can decide if a specific solution is right for you or not. > I agree, B-trees are for database index problems, and not only, however, > just imagine what would happen if mySQL or PostgreSQL would throw away > their database indexing/locking issue up to the filesystem level? It > would be a total hoax, one would need separate filesystem tuning for > mysql, one for postresql, one for mail, one for apache, etc.. This just > brings headaches and unnecessarry restrictions to the partitioning schema. That depends on the underlying file system, and how flexible it is. Apache, mail, etc tend to work ok with a standard Unix file system. Database have more stringent requirements - including performance constraints. I remember commercial databases recommending that you hand them raw disk devices, and skip the OS file system manipulations completely. File systems have gotten a lot better since then, so they may not do that any longer. > This is why something like dbmail seems more appropiate in my opinion > (conceptually). Well, it's more appropriate for some uses. I punted on mbox format in the 80s, when I realized that I could use stock unix commands for manipulating single messages if I used mh mailboxes. This was a major win, as there weren't very good tools for manipulating single messages in an mbox. If your usage is restricted to people doing POP/IMAP, then dbmail would certainly work better. The downside is that you can't use Unix tools to manipulate messages. The upside (?) is that you can use SQL to manipulate messages, which may be a major win. I'm certainly going to check it out. Thanks, http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information.