Date: Fri, 17 Sep 1999 22:26:18 +0200 From: Brad Knowles <blk@skynet.be> To: Matthew Dillon <dillon@apollo.backplane.com> Cc: current@FreeBSD.ORG Subject: Re: 2xPIIIx450 results & NFS results (was More benchmarking stuff...) Message-ID: <v04205548b40851d564ab@[195.238.1.121]> In-Reply-To: <199909172002.NAA54939@apollo.backplane.com> References: <XFMail.990917112639.lh@aus.org> <199909171658.JAA53751@apollo.backplane.com> <v0420553bb40826e849a4@[195.238.1.121]> <199909171856.LAA54721@apollo.backplane.com> <v04205547b40842e2de12@[195.238.1.121]> <199909172002.NAA54939@apollo.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
At 1:02 PM -0700 1999/9/17, Matthew Dillon wrote: > Sendmail does not get into trouble with queue files it is able to retire > quickly. Where sendmail gets into trouble is with queue files it ISN'T > able to retire quickly. This is why you *see* 10,000+ files in mqueue > at times. These files build up because a small percentage of mail > destinations cannot be delivered to immediately. In my experience, sendmail almost always accepts messages a lot faster than it can process them, regardless of the mode in which sendmail is running. Foreground, queue-only, background, in my experience it doesn't make a whole lot of difference. So the input number of files can grow very quickly, with deliveries (and mqueue file removals) suffering. > The reason sendmail tends to break down with large queue directories has > little to do with directory overhead and a lot to do with sendmail's own > algorithms. If you have 50 sendmails running a 10,000 file queue, each > of those sendmail processes is essentially scanning the entire queue. Yup, that's another very real problem with sendmail, and another reason why I really like postfix. > If not controlled, this eventually leads to a cascade failure. The > potential for a cascade failure is, in fact, the number one reason for > *NOT* running sendmail with background queueing mode turned on. The > best way to avoid a cascade failure is to run the sendmail daemon in > queue-only mode with a set fork limit: > > sendmail -bd -OMaxDaemonChildren=X -ODeliveryMode=q > > And run the sendmail queue runner separately: > > sendmail -q1m -OMaxDaemonChildren=Y -OMinQueueAge=1h In my experience, you still get messages being accepted a lot faster than they're being pushed out. This is usually made even worse when you're delivering to mbox-style mailboxes, where a single large message may come in addressed to dozens of recipients, and now you might have megabytes of data being read but gigabytes of data being written. This is another of my pet peeves, where I believe that a database-oriented solution that writes just one copy of the message and then gives all recipients pointers to it, would help a great deal. Of course, there's not anyone like Eric or Wietse to pick on when it comes to writing database-oriented local delivery agents. > The key issue with any mail server is that bandwidth and transaction > useage tends to be low relatively speaking. A USENET news system > almost always has much higher transactional overhead, especially if it > is taking several feeds. A million news messages a day translates to > around 10 million protocol transactions for a news box taking 4 feeds. However, most of those transactions should be looking up message-ids in history or precommit cache databases and then refusing the article without it actually being transmitted. High transactional rates, yes. But the actual number of articles being received would be on par with a very busy mail server. If you've got a lot of outbound feeds, of course the outbound data rate could be a very real problem, but that's a separate issue. > What you cannot afford to spend time > doing in a mail server is scanning the same queue file over and over > again, so what you want to optimize for are the 5% of email messages > that wind up stuck in the queue for more then a few minutes but usually > less then an hour, and then make sure the 1% that stick around past > that do not interfere with the processing of those that stick around > less. You can't really fix sendmail in this regard, although you could replace it with a different MTA. I guess you could change the implementation methods of the underlying filesystem so as to speed up those constant linear sweeps of the entire mqueue directory by the queue runners (and by every sendmail process that goes to create a file in the mqueue, since they have to guarantee that the filename they're creating is unique). How you would actually do this is totally beyond me, however. -- These are my opinions -- not to be taken as official Skynet policy ____________________________________________________________________ |o| Brad Knowles, <blk@skynet.be> Belgacom Skynet NV/SA |o| |o| Systems Architect, News & FTP Admin Rue Col. Bourg, 124 |o| |o| Phone/Fax: +32-2-706.11.11/12.49 B-1140 Brussels |o| |o| http://www.skynet.be Belgium |o| \/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/ Unix is like a wigwam -- no Gates, no Windows, and an Apache inside. Unix is very user-friendly. It's just picky who its friends are. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-current" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?v04205548b40851d564ab>