Date: Wed, 15 May 2002 03:24:25 -0400 From: Omar Thameen <omar@clifford.inch.com> To: Doug White <dwhite@resnet.uoregon.edu> Cc: freebsd-hackers@FreeBSD.ORG Subject: Re: tuning a CPU bound server Message-ID: <20020515032425.A23491@clifford.inch.com> In-Reply-To: <20020514211907.W70761-100000@resnet.uoregon.edu>; from dwhite@resnet.uoregon.edu on Tue, May 14, 2002 at 09:25:21PM -0700 References: <20020514142441.A73151@clifford.inch.com> <20020514211907.W70761-100000@resnet.uoregon.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Tue, May 14, 2002 at 09:25:21PM -0700, Doug White wrote: > Mmm, mail server tuning, something I have some experience in :-) Just what I was hoping to hear! > First off, what are the specs of the server? Cpu? Disk? Memory? Network? > You mention it's a dual 800MHz. What kind of NIC does it have? What is the > speed and duplex set to on it? Dual PIII/800 2G SDRAM 2x18G IBM 10,000 rpm SCSI drives, running vinum RAID0 because I originally thought disk accesses would be a bottleneck (but they aren't) Tyan Thunder LE S2510 motherboard: sym0,sym1 SCSI controller (on board, one drive on each channel) fxp0 network (on-board), 100M, full-duplex > Secondly, what period was your vmstat run over? The vmstat shows most of > the time spent in system. I'm thinking it might be network-based but > without the specs its hard to tell. Updating every second. It has about the same network I/O as the heavily hit webserver (same # of interrupts/second), but smtp is a lengthier negotiation, right? > While you're getting specs run netstat -m every so often and collect the > mbuf and mbuf cluster utilization numbers. Will do. Since there's no big delivery happening at this moment, I'll follow up later. Last I checked, it wasn't running out of mbufs (kern.ipc.nmbufs: 34816). > qmail is also very inefficient when it comes to large delivery -- the fork > per message and the qmail-remote trigger-hitting will eventually > bottleneck you. It's probable you've run into it. My sympathies. :) You > might try *dropping* concurrencyremote somewhat to reduce the > context-switch thrash (although your context-switch numbers aren't too > high, I've seen worse and the machine wasn't taxed too heavily). I've grown to this concurrencyremote fairly gradually. There was improvement up to about 600 concurrencyremote. I added a 2nd identical qmail queue so the deliveries wouldn't be so serial, but didn't see anywhere near a doubling of the delivery capacity. When only one of the queues is delivering, I see very similar vmstat and throughput values. > It might be interesting to run top and find what the major culprit of cpu > usage is. Somehow I think it'll be qmail-send. It's dnscache (~30%), then the 2 qmail-rspawn processes (~20-25% each). I thought about running dnscache on a separate server, but don't think that will give the performance improvement I was hoping for. Omar > On Tue, 14 May 2002, Omar Thameen wrote: > > > I've got a pretty heavily loaded server which is doing mostly > > mailing list delivery. It happens to be running qmail with a large > > number of concurrencyremotes (about 1000), meaning that there is > > one smtp delivery process spawned for each message. I've hit a > > plateau with regards to the amount of bandwidth that the server > > can push out - I see similar performance with half the concurrencyremotes. > > > > As best as I can tell, the server is CPU bound. My main question > > is whether there are any kernel parameters I can tune to improve > > performance, or whether I just need to get more powerful processors > > (Xeons?). The current system is a dual PIII/800. [...] To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020515032425.A23491>