Date: 22 Mar 2003 08:16:02 -0500 From: Chris Shenton <chris@shenton.org> To: Len Conrad <LConrad@Go2France.com> Cc: freebsd-isp@freebsd.org Subject: Re: Maximum recommended user limits on mail server Message-ID: <87znnn5yp9.fsf@Pectopah.shenton.org> In-Reply-To: <5.2.0.9.0.20030319184158.02da2ba8@mail.go2france.com> References: <5.2.0.9.0.20030319184158.02da2ba8@mail.go2france.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Len Conrad <LConrad@Go2France.com> writes: > >A person I know is looking to have a mail server with 10K-15K > >accounts and 50 virtual domains. Shouldn't be a big deal so long as your architecture's sane. > You've got some good pointers on the mailbox servers, but nobody > talked about an multi-box architecture, which is the key to > scaleability and reliability. Absolutely agreed. If you've got a bunch of users, you can't just take the mailserver(s) down to upgrade some HW or SW. Design a system which has no single points of failure. I've learned this at a couple places the hard way and am doing it the "right" way on a project I'm working on. We'll be using a redundant pair of load balancers with a few 1U servers behind them offering SMTP, APOP (historical), IMAP, WebMail and an LDAP replica. Behind those will be a big NFS box (NetApp if I get the budget) and an LDAP master. With this, we'll be able to take down any of the data-less 1U servers if we need to, no interruption of service. To make this work, a shared store for mailboxes is key -- in this case NFS. There are a couple of catches tho. NFS does not always provide reliable locking so multiple writers can corrupt a file. I've also seen that UNIX mbox-style single-file-for-all-mail will lead to disk i/o starvation if mboxes get above 20MB or so -- check the qpopper list for lots of war stories. A solution to both problems is to use a mailstore format that does not put all of a user's messages in a single file. DJB's "Maildir" format does this -- each message gets its own file. No file locking issues, and stuff that's expensive on mbox (e.g., delete a message) is trivial and fast with Maildir. (There are other techniques if your mail servers do not share mailstore; use a directory to route incoming mail to the appropriate box for each users, and likewise proxy POP/IMAP connections. qmail-ldap can do this. You don't get the redundancy of the shared store approach, however). I'm using qmail + vpopmail + courier-imap + sqwebmail at one ISP. It does virtual domains very nicely and the mailstore is Maildir format. This small site uses a single box so has no redundancy, but with an NFS backend it could be fault-tolerant. I'm building another using qmail-ldap (and courier-imap and sqwebmail) and rather enjoying the flexibility of qmail using LDAP to lookup user address variations, mailbox location, quotas, active/disabled status, etc. I also find it rather fast. In some benchmarking I was doing yesterday, I ran 1000 sequential sessions each injecting 3 messages and saw a rate of 22 msg/sec -- on a P2-500 with 256MB RAM and an old IDE disk; this was fully delivered to local Maildir mailboxes, not sitting in a queue trying to be delivered. I was running this on FreeBSD-4.5 or so. I need to do benchmarks on POP/IMAP retrieval, since typically you'll have more reading (users) than writing (delivery to users). If you're religiously set on sendmail, you can use procmail or maildrop as a local mail delivery agent to deliver mail to Maildir format mailboxes. I've done this at one site, and used a maildir-aware POP daemon to serve users. Works OK but sendmail isn't nearly as fast as qmail. I understand Postfix can deliver to Maildir but have no experience with it. > 1. dedicated box as SMTP gateway. > 2. dedicated AV box > 3. mailbox server, muchly liberated because no anti-mail abuse, no DNS > query delay, no retrying of delayed mail, no AV scanning, and 50+% of > the inbound mail rejected at the envelope by 1. I may partition services on my 1U servers to act like the above. As long as I offer the services on each box, I can steer traffic to any I desire based on load balancer rules. (I might have more POP/IMAP services than SMTP, for example). If you don't have a balancer, you can leverage DNS MX for SMTP delivery and DNS round-robin for POP/IMAP access (but you'll have to manually tweak DNS if you take a machine out of POP/IMAP service). I haven't yet investigated anti-spam/anti-virus stuff except to look at Postini and Brightmail services -- pricey but can be cost-justified at corporate sites. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?87znnn5yp9.fsf>