From owner-freebsd-chat Fri Feb 14 18: 0: 0 2003 Delivered-To: freebsd-chat@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7972037B401 for ; Fri, 14 Feb 2003 17:59:54 -0800 (PST) Received: from excalibur.skynet.be (excalibur.skynet.be [195.238.3.90]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B24143F75 for ; Fri, 14 Feb 2003 17:59:53 -0800 (PST) (envelope-from brad.knowles@skynet.be) Received: from [10.0.1.4] (ip-26.shub-internet.org [194.78.144.26] (may be forged)) by excalibur.skynet.be (8.12.7/8.12.7/Skynet-OUT-2.21) with ESMTP id h1F1xitT024533; Sat, 15 Feb 2003 02:59:44 +0100 (MET) (envelope-from ) Mime-Version: 1.0 X-Sender: bs663385@pop.skynet.be Message-Id: In-Reply-To: <3E4D7702.8EE51F54@mindspring.com> References: <20030211032932.GA1253@papagena.rockefeller.edu> <3E498175.295FC389@mindspring.com> <3E49C2BC.F164F19A@mindspring.com> <3E4A81A3.A8626F3D@mindspring.com> <3E4B11BA.A060AEFD@mindspring.com> <3E4BC32A.713AB0C4@mindspring.com> <3E4CB9A5.645EC9C@mindspring.com> <3E4D7702.8EE51F54@mindspring.com> Date: Sat, 15 Feb 2003 02:55:54 +0100 To: Terry Lambert From: Brad Knowles Subject: Re: Email push and pull (was Re: matthew dillon) Cc: Brad Knowles , Rahul Siddharthan , freebsd-chat@freebsd.org Content-Type: text/plain; charset="us-ascii" ; format="flowed" Sender: owner-freebsd-chat@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.org At 3:08 PM -0800 2003/02/14, Terry Lambert wrote: > I've got to say that on any mail server I've ever worked on, the > limitation on what it could handle was *never* disk I/O, unless > it was before trivial tuning had been done. It was always network > I/O, bus bandwidth, or CPU. I have yet to see a mail system that had limitations in any of these areas, and other than the ones you've seen, I have yet to hear of one such a mail system that any other mail systems expert has ever seen that I have talked to -- including the AOL mail system. > As far as I'm concerned, for most applications, "disks are fast > enough"; even an IDE disk doing thermal recalibration can keep > up with full frame rate digitized video for simultaneous record > and playback. 5 of those, and you're at the limit of 100Mbit > ethernet in and out, and you need 5 for RAID. If all you ever care about is bandwidth and not latency, and you really do get the bandwidth numbers claimed by the manufacturer as opposed to the ones that we tend to see in the real world, I might possibly believe this. Of course, I have yet to hear of a theoretical application where this would be the case, but I am willing to concede the point that such a thing might perhaps exist. > FWIW, since you really don't give a damn about timestamps on the > queue files in question, etc., you can relax POSIX guarantees on > certain metadata updates that were put there to make the DEC VMS > engineers happy by slowing down UNIX, relative to VMS, so they > would not file lawsuit from POSIX being required for gvernment > contracts. In what way don't you care about timestamps? And which timestamps don't we care about? Are you talking about noatime, or something else? Note that noatime doesn't exist for NFS mounts, at least it's not one I have been able to specify on our systems. > Because those are not magically a consequence of increased complexity. > Complexity can be managed. At what cost? > And they are back to transmitting 1 copy each, If they're putting the recipient name into the body of the e-mail message, then they're doing that anyway. Since they don't care about whether any of their spam is lost, they can run from memory-based filesystems. They can generate orders of magnitude more traffic than you could handle on the same hardware, simply because they don't have to worry what happens if the system crashes. Moreover, they can use open relays and high-volume spam-sending networks to further increase their amplitude. > Only if the messages came in on seperate sessions, and they are back > to transmitting 1 copy each, and they lose their amplification effect > in any attack. See above. Using SIS hurts you far more than it could possibly hurt them. >> So, you're right back where you started, and yet you've paid such >> a very high price. > > It's a price you have to pay anyway. No, you don't. > You mean "simply because we, the provider, failed to protect them > from a DOS". On user's DOS is another user's normal level of e-mail. It's impossible to protect them from DOS at that level, because you cannot possibly know, a priori, what is a DOS for which person. At higher levels, you can detect a DOS and at least delay it by having circuit breakers, such as quotas. > OK, that's a 25% reduction in the metadata overhead required, > which is what you claim is the bottleneck. That doesn't look > insignificant to me. Read the slides again. It doesn't reduce the meta-data overhead at all, only the data bandwidth required. Using ln to create a hard link to another file requires just as much synchronous meta-data overhead as it does to create the file in the first place -- the only difference is that you didn't have to also store another copy of the file contents. However, as we said before, storing a copy of the file contents is cheap -- what kills us is the synchronous meta-data overhead. > My argument would be that this should be handled out-of-band > via a feedback mechanism, rather than in-band via an EQUOTA, > using the quota as a ffeedback mechanism. What kind of out-of-band mechanism did you have in mind? Are we re-inventing the entirety of Internet e-mail yet once again? > You're going to do that to the user anyway. Worse, you are going > to give them a mailbox full of DOS crap, and drop good messages > in the toilet (you've taken responsibility for the delivery, so > the sender may not even have them any more, so when you drop them > after the 4 days, they are screwed; As soon as the user notices the overflowing mailbox, they can call the helpdesk and the people on the helpdesk have tools available to them to do mass cleanup, and avoid the problem for the user to deal with this problem. That gives them seven days to notice the problem and fix it, before things might start bouncing. We will likewise have daily monitoring processes that will set off alarms if a mailbox overflows, so that we can go take a look at it immediately. >> Bait not taken. The customer is paying me to implement quotas. >> This is a basic requirement. > > This is likely the source of the disconnect. I view the person > whose mail I'm taking responsibility for, as the customer. The users don't pay my salary. The customer does. I do everything I can to help the users in every way possible, but when it comes down to a choice of whether to do A or B, the customer decides -- not the users. > You wouldn't implement an out-of-band mechanism instead? Not at the price of re-inventing the entirety of Internet e-mail, no. > You'd > insist on the in-band mechanism of a MDA error, after you've > already accepted responsibility for the message you aren't going > to be able to deliver? The message was accepted and delivered to stable storage, and we would have done the best we could possibly do to actually deliver it to the user's mailbox. However, that's a gateway function -- the users mailbox doesn't speak SMTP, and therefore we would have fulfilled all of our required duties, to the best of our ability. No one has any right to expect any better. > You *will* drop stuff in /dev/null. Any queue entries you remove > are dropped in /dev/null. They're not removed or dropped in /dev/null. I don't know where you pulled that out of your hat, but on our real-world mail systems we would generate a bounce message. > My recommendation would be to use an inode FS as a variable > granularity block store, and use that for storing messages. It must be nice to be in a place where you can afford the luxury of contemplating completely re-writing the filesystem code, or even the entire OS. > Not if you constrain yourself to NFS, there isn't, I agree. Not my decision. I wasn't given a choice. > If you're convinced, then you should be doing something else. 8-(. I wish I could be. Not my decision. I wasn't given a choice. > This is an artifact of using the new Sleepycat code. You can > actually compile it to use the older code, which can be made to > not use mmap. As of what version is this still possible? How far back do you have to go? And are you sure that Cyrus would still work with that? Certainly, when it comes to SAMS, all this stuff is pre-compiled and you don't get the option of building Berkeley DB in a different manner, etc.... > I like sendmail, and I like their people. In general, though, I > would say that they are still looking for their commercial market, > so this is less impressive to me than it would be otherwise. They're going after the large ISP/ASP and the large corporate/Enterprise markets. However, their marketing & public relations could use some improvement. > That's all to the good: by pushing it from 40 seconds to ~8 minutes, > you favor my argument that the operation is network bound. Indirectly, perhaps. The real limitations is in the NFS implementation on the server, including how it handles synchronous meta-data updates. A major secondary factor is the client NFS implementation. > Yes, it is. If you read previous postings, I suggested that the > bastion SMTP server would forward the messages to the IMAP server > that will in the future serve them, in order to permit local > delivery. There will be a designated primary server for a give mailbox, but any of the other back-end servers could potentially also receive a request for delivery or access to the same mailbox. Our hope is that 99% of all requests will go through the designated primary (for obvious performance reasons), but we cannot currently design the system so that *only* the designated back-end server is allowed to serve that particular mailbox. -- Brad Knowles, "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -Benjamin Franklin, Historical Review of Pennsylvania. GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+ !w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++) tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message