Date: Tue, 14 May 2002 21:25:21 -0700 (PDT) From: Doug White <dwhite@resnet.uoregon.edu> To: Omar Thameen <omar@clifford.inch.com> Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: tuning a CPU bound server Message-ID: <20020514211907.W70761-100000@resnet.uoregon.edu> In-Reply-To: <20020514142441.A73151@clifford.inch.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Mmm, mail server tuning, something I have some experience in :-) First off, what are the specs of the server? Cpu? Disk? Memory? Network? You mention it's a dual 800MHz. What kind of NIC does it have? What is the speed and duplex set to on it? Secondly, what period was your vmstat run over? The vmstat shows most of the time spent in system. I'm thinking it might be network-based but without the specs its hard to tell. While you're getting specs run netstat -m every so often and collect the mbuf and mbuf cluster utilization numbers. qmail is also very inefficient when it comes to large delivery -- the fork per message and the qmail-remote trigger-hitting will eventually bottleneck you. It's probable you've run into it. My sympathies. :) You might try *dropping* concurrencyremote somewhat to reduce the context-switch thrash (although your context-switch numbers aren't too high, I've seen worse and the machine wasn't taxed too heavily). It might be interesting to run top and find what the major culprit of cpu usage is. Somehow I think it'll be qmail-send. On Tue, 14 May 2002, Omar Thameen wrote: > I've got a pretty heavily loaded server which is doing mostly > mailing list delivery. It happens to be running qmail with a large > number of concurrencyremotes (about 1000), meaning that there is > one smtp delivery process spawned for each message. I've hit a > plateau with regards to the amount of bandwidth that the server > can push out - I see similar performance with half the concurrencyremotes. > > As best as I can tell, the server is CPU bound. My main question > is whether there are any kernel parameters I can tune to improve > performance, or whether I just need to get more powerful processors > (Xeons?). The current system is a dual PIII/800. > > Here is a typical vmstat: > > procs memory page disks faults cpu > r b w avm fre flt re pi po fr sr da0 da1 in sy cs us sy id > 18 10 0 490660 232216 4913 0 0 0 3443 0 5 1 1328 36613 5010 6 90 3 > 22 10 0 490948 232336 4962 0 0 0 3703 0 21 2 1448 41551 5804 7 92 1 > 17 11 0 489872 232336 4791 0 0 0 3509 0 1 1 1483 36746 5176 9 88 3 > 13 11 0 490248 232004 5285 0 0 0 3817 0 6 0 1541 38059 5323 6 92 2 > 23 11 0 492356 232072 5315 0 0 0 4012 0 0 1 1561 37586 5206 8 90 3 > 13 11 0 495272 231644 4902 0 0 0 3498 0 0 0 1308 36375 5118 7 87 6 > 28 11 0 495008 231440 4041 0 0 0 2977 0 119 0 1679 42109 5956 10 86 4 > 22 11 0 493708 232108 5751 0 0 0 4408 0 0 1 1653 37760 5440 6 93 1 > 8 11 0 495796 231140 4816 0 0 0 3299 0 0 0 1482 34807 4866 6 86 8 > > The run queue and runnable-but-blocked queue are obviously problematic. > The other thing I noticed when comparing the vmstat output to a > heavily hit webserver, is that my context switching is an order of > magnitude higher. I'm assuming this is necessary because of the number > of remote processes that need to be spawned for message delivery. > > Any ideas would be much appreciated. Previously, I tweaked some > of the variables available via sysctl based on an article for tuning > the Lyris ListManager, and have tried other parameters that appear > to be related (the sysctl documentation for non-kernel hackers is > a bit sparse). > > Omar > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-hackers" in the body of the message > Doug White | FreeBSD: The Power to Serve dwhite@resnet.uoregon.edu | www.FreeBSD.org To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020514211907.W70761-100000>