Date: Sat, 15 Feb 2003 02:55:54 +0100 From: Brad Knowles <brad.knowles@skynet.be> To: Terry Lambert <tlambert2@mindspring.com> Cc: Brad Knowles <brad.knowles@skynet.be>, Rahul Siddharthan <rsidd@online.fr>, freebsd-chat@freebsd.org Subject: Re: Email push and pull (was Re: matthew dillon) Message-ID: <a05200f01ba734065f08e@[10.0.1.4]> In-Reply-To: <3E4D7702.8EE51F54@mindspring.com> References: <20030211032932.GA1253@papagena.rockefeller.edu> <a05200f2bba6e8fc03a0f@[10.0.1.2]> <3E498175.295FC389@mindspring.com> <a05200f37ba6f50bfc705@[10.0.1.2]> <3E49C2BC.F164F19A@mindspring.com> <a05200f43ba6fe1a9f4d8@[10.0.1.2]> <3E4A81A3.A8626F3D@mindspring.com> <a05200f4cba70710ad3f1@[10.0.1.2]> <3E4B11BA.A060AEFD@mindspring.com> <a05200f5bba7128081b43@[10.0.1.2]> <3E4BC32A.713AB0C4@mindspring.com> <a05200f07ba71ee8ee0b6@[10.0.1.2]> <3E4CB9A5.645EC9C@mindspring.com> <a05200f14ba72aae77b18@[10.0.1.2]> <3E4D7702.8EE51F54@mindspring.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 3:08 PM -0800 2003/02/14, Terry Lambert wrote:
> I've got to say that on any mail server I've ever worked on, the
> limitation on what it could handle was *never* disk I/O, unless
> it was before trivial tuning had been done. It was always network
> I/O, bus bandwidth, or CPU.
I have yet to see a mail system that had limitations in any of
these areas, and other than the ones you've seen, I have yet to hear
of one such a mail system that any other mail systems expert has ever
seen that I have talked to -- including the AOL mail system.
> As far as I'm concerned, for most applications, "disks are fast
> enough"; even an IDE disk doing thermal recalibration can keep
> up with full frame rate digitized video for simultaneous record
> and playback. 5 of those, and you're at the limit of 100Mbit
> ethernet in and out, and you need 5 for RAID.
If all you ever care about is bandwidth and not latency, and you
really do get the bandwidth numbers claimed by the manufacturer as
opposed to the ones that we tend to see in the real world, I might
possibly believe this. Of course, I have yet to hear of a
theoretical application where this would be the case, but I am
willing to concede the point that such a thing might perhaps exist.
> FWIW, since you really don't give a damn about timestamps on the
> queue files in question, etc., you can relax POSIX guarantees on
> certain metadata updates that were put there to make the DEC VMS
> engineers happy by slowing down UNIX, relative to VMS, so they
> would not file lawsuit from POSIX being required for gvernment
> contracts.
In what way don't you care about timestamps? And which
timestamps don't we care about? Are you talking about noatime, or
something else? Note that noatime doesn't exist for NFS mounts, at
least it's not one I have been able to specify on our systems.
> Because those are not magically a consequence of increased complexity.
> Complexity can be managed.
At what cost?
> And they are back to transmitting 1 copy each,
If they're putting the recipient name into the body of the e-mail
message, then they're doing that anyway. Since they don't care about
whether any of their spam is lost, they can run from memory-based
filesystems. They can generate orders of magnitude more traffic than
you could handle on the same hardware, simply because they don't have
to worry what happens if the system crashes. Moreover, they can use
open relays and high-volume spam-sending networks to further increase
their amplitude.
> Only if the messages came in on seperate sessions, and they are back
> to transmitting 1 copy each, and they lose their amplification effect
> in any attack.
See above. Using SIS hurts you far more than it could possibly hurt them.
>> So, you're right back where you started, and yet you've paid such
>> a very high price.
>
> It's a price you have to pay anyway.
No, you don't.
> You mean "simply because we, the provider, failed to protect them
> from a DOS".
On user's DOS is another user's normal level of e-mail. It's
impossible to protect them from DOS at that level, because you cannot
possibly know, a priori, what is a DOS for which person. At higher
levels, you can detect a DOS and at least delay it by having circuit
breakers, such as quotas.
> OK, that's a 25% reduction in the metadata overhead required,
> which is what you claim is the bottleneck. That doesn't look
> insignificant to me.
Read the slides again. It doesn't reduce the meta-data overhead
at all, only the data bandwidth required. Using ln to create a hard
link to another file requires just as much synchronous meta-data
overhead as it does to create the file in the first place -- the only
difference is that you didn't have to also store another copy of the
file contents.
However, as we said before, storing a copy of the file contents
is cheap -- what kills us is the synchronous meta-data overhead.
> My argument would be that this should be handled out-of-band
> via a feedback mechanism, rather than in-band via an EQUOTA,
> using the quota as a ffeedback mechanism.
What kind of out-of-band mechanism did you have in mind? Are we
re-inventing the entirety of Internet e-mail yet once again?
> You're going to do that to the user anyway. Worse, you are going
> to give them a mailbox full of DOS crap, and drop good messages
> in the toilet (you've taken responsibility for the delivery, so
> the sender may not even have them any more, so when you drop them
> after the 4 days, they are screwed;
As soon as the user notices the overflowing mailbox, they can
call the helpdesk and the people on the helpdesk have tools available
to them to do mass cleanup, and avoid the problem for the user to
deal with this problem. That gives them seven days to notice the
problem and fix it, before things might start bouncing. We will
likewise have daily monitoring processes that will set off alarms if
a mailbox overflows, so that we can go take a look at it immediately.
>> Bait not taken. The customer is paying me to implement quotas.
>> This is a basic requirement.
>
> This is likely the source of the disconnect. I view the person
> whose mail I'm taking responsibility for, as the customer.
The users don't pay my salary. The customer does. I do
everything I can to help the users in every way possible, but when it
comes down to a choice of whether to do A or B, the customer decides
-- not the users.
> You wouldn't implement an out-of-band mechanism instead?
Not at the price of re-inventing the entirety of Internet e-mail, no.
> You'd
> insist on the in-band mechanism of a MDA error, after you've
> already accepted responsibility for the message you aren't going
> to be able to deliver?
The message was accepted and delivered to stable storage, and we
would have done the best we could possibly do to actually deliver it
to the user's mailbox. However, that's a gateway function -- the
users mailbox doesn't speak SMTP, and therefore we would have
fulfilled all of our required duties, to the best of our ability. No
one has any right to expect any better.
> You *will* drop stuff in /dev/null. Any queue entries you remove
> are dropped in /dev/null.
They're not removed or dropped in /dev/null. I don't know where
you pulled that out of your hat, but on our real-world mail systems
we would generate a bounce message.
> My recommendation would be to use an inode FS as a variable
> granularity block store, and use that for storing messages.
It must be nice to be in a place where you can afford the luxury
of contemplating completely re-writing the filesystem code, or even
the entire OS.
> Not if you constrain yourself to NFS, there isn't, I agree.
Not my decision. I wasn't given a choice.
> If you're convinced, then you should be doing something else. 8-(.
I wish I could be. Not my decision. I wasn't given a choice.
> This is an artifact of using the new Sleepycat code. You can
> actually compile it to use the older code, which can be made to
> not use mmap.
As of what version is this still possible? How far back do you
have to go? And are you sure that Cyrus would still work with that?
Certainly, when it comes to SAMS, all this stuff is pre-compiled
and you don't get the option of building Berkeley DB in a different
manner, etc....
> I like sendmail, and I like their people. In general, though, I
> would say that they are still looking for their commercial market,
> so this is less impressive to me than it would be otherwise.
They're going after the large ISP/ASP and the large
corporate/Enterprise markets. However, their marketing & public
relations could use some improvement.
> That's all to the good: by pushing it from 40 seconds to ~8 minutes,
> you favor my argument that the operation is network bound.
Indirectly, perhaps. The real limitations is in the NFS
implementation on the server, including how it handles synchronous
meta-data updates. A major secondary factor is the client NFS
implementation.
> Yes, it is. If you read previous postings, I suggested that the
> bastion SMTP server would forward the messages to the IMAP server
> that will in the future serve them, in order to permit local
> delivery.
There will be a designated primary server for a give mailbox, but
any of the other back-end servers could potentially also receive a
request for delivery or access to the same mailbox. Our hope is that
99% of all requests will go through the designated primary (for
obvious performance reasons), but we cannot currently design the
system so that *only* the designated back-end server is allowed to
serve that particular mailbox.
--
Brad Knowles, <brad.knowles@skynet.be>
"They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety."
-Benjamin Franklin, Historical Review of Pennsylvania.
GCS/IT d+(-) s:+(++)>: a C++(+++)$ UMBSHI++++$ P+>++ L+ !E-(---) W+++(--) N+
!w--- O- M++ V PS++(+++) PE- Y+(++) PGP>+++ t+(+++) 5++(+++) X++(+++) R+(+++)
tv+(+++) b+(++++) DI+(++++) D+(++) G+(++++) e++>++++ h--- r---(+++)* z(+++)
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-chat" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a05200f01ba734065f08e>