Date: Tue, 16 Mar 2004 09:42:25 +0100 From: Bogdan TARU <bgd@icomag.de> To: Peter Jeremy <PeterJeremy@optushome.com.au> Cc: freebsd-hackers@freebsd.org Subject: Re: kernel activity Message-ID: <20040316084225.GA55231@icomag.de> In-Reply-To: <20040316070601.GK56509@cirb503493.alcatel.com.au> References: <20040315150438.GA48241@icomag.de> <20040316070601.GK56509@cirb503493.alcatel.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Peter and all, Thanks for the mails & advices... The box is a dual xeon @3GHz, with 4GB of ram and raid 5 on board (scsi HDDs), with a 4.9 on it. The 'tuning' includes removing all the unnecessary stuff from the kernel, activating the ACCEPT_FILTERS and tuning some sysctl values, especially the ones dealing with network (net.inet.tcp.msl etc.), and raising the values for maxfiles and somaxconn, etc. The box has two NICs, one of them is a fxp with link0 activated (cannot use polling because I don't want to give up SMP -- the userland activity is already 40%, so giving up one CPU as to reduce sys load it's just gonna leave the bottleneck where it is -- CPU, that is), and the other one is an em, but cannot use it since i don't have a gb switch. Before activating link0 on fxp, the level of interrups/sec on this interface peaked 6k, but after activating link0 it was reduced to 2k. Still, a lot of sys activity... As a webserver, I run apache, stripped down from the modules that I don't need, and compiled in php and some other modules (statically). Most of the content that I serve is static, there are only a few php scripts and they don't get much hits. I don't run any other 'intensive computing' application on the server, no firewall software or so. And no, I haven't tried turning HTT on and off, should I do that? I am also considering trussing one of the apaches, to see what system calls it's doing... Anyways, thanks for your help, bogdan On Tue, Mar 16, 2004 at 06:06:01PM +1100, Peter Jeremy wrote: > On Mon, Mar 15, 2004 at 04:04:38PM +0100, Bogdan TARU wrote: > > I'm running a pretty busy webserver, and right now I can see it's > > CPU-bound: > > A few more details would be useful: > What version of FreeBSD? > What hardware are you using (CPU and NIC in particular)? > What application(s) are you running? You mention a webserver - which one? > Is it just serving static pages or is there lots of dynamic content > (CGI or servlet etc)? Any other applications? > Do you have firewall software running? If so, which? Is it simple (a dozen > or so rules) or complex (thousands of rules)? > What FreeBSD tuning have you done? > How does your kernel config compare to GENERIC? > If the CPU supports HTT, have you tried turning HTT on and off? > > >84 2 0 1135836 142692 82 0 0 0 405 0 35 0 3144 246325 1739 42 58 0 > > You have very high interrupt and system call rate - both of these > contribute to system time. > > I presume most of the interrupts are from your NIC. Have you considered > using polling mode (see polling(4))? Some NICs are more efficient than > others - fxp(4) is one of the best, rl(4) is probably the worst. > > It's difficult to say whether the syscall rate is excessive or not. > The number of system calls is generally up to the application - you > need to tune or redesign it to reduce the number of system calls it > makes per unit of work. That said, I've noticed that threading on > -STABLE adds quite a significant overhead - via high system call rate > and system time. In one case, I improved the throughput of a graphics > manipulation process by about 10% by removing the threading. > > It's difficult to get much visibility on where system time is going > (though "systat -v" will split out interrupt time). In theory, you > could build a profiling kernel but this is non-trivial and may or may > not be functional at present. > > Peter > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040316084225.GA55231>