Date: Thu, 11 Sep 2003 18:35:21 -0500 From: gv-list-freebsdquestions@mygirlfriday.info To: freebsd-questions@freebsd.org Subject: Re: `top` process memory usage: SIZE vs RES Message-ID: <1232387734.20030911183521@mygirlfriday.info> In-Reply-To: <bjqs64$6i9$1@sea.gmane.org> References: <bjqs64$6i9$1@sea.gmane.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Hello Jesse, Thursday, September 11, 2003, 5:15:31 PM, you wrote: J> I am stress testing a FreeBSD 4.7-RELEASE server (it's a pre- J> production test server) by sending huge email messages to it J> via SMTP. I too am putting together a 4.8 box as we speak for a client. J> I'm running qmail-1.03 built from source, with the QMAILQUEUE J> patch, qmail-qfilter, a custom Python script that runs under J> qmail-qfilter, and qmail-scanner with ClamAV. J> I test the server by sending a 59M or a 99M email from a J> remote machine (connected via fxp0). J> As I watch the email travel down the qmail-smtpd->qmail-qfilter-> custom-python->>qmail-scanner->clamd->qmail-queue "pipeline", J> I watch the memory usage with `top`. J> Memory is critical in this type of application, since I run my J> qmail-smtpd "pipeline" under DJB's softlimit program. I MUST J> know how much memory to allocate for the upper limit of each J> pipeline, otherwise qmail-smtpd will terminate the transfer J> with a 451 SMTP error. right. J> Anyway, as I watch `top`, I never see more than 15M being used J> by the various pipeline programs at any given point in time, J> but my Free Memory constantly declines until it reaches about J> 526k. J> The Questions: J> ========================== J> 1.) Where is my Free memory going? given what you say custom-python->>>qmail-scanner->clamd->qmail-queue This whole scenario is very memory intensive. First you have each email "pythonized" and then qmail-scanner is *very* memory intensive, as it has initially a very heavy duty perl script for each email before being passed off to clamd. Multiply this with the default SMTP concurrency limit of 20, and you have a lot of memory usage. Clamd is known to be a memory piggie too. I switched from that to using fprot which uses less memory and is faster processing. J> I can't account for it J> in the SIZE and RES columns of the various processes. J> These are relatively constant. Maybe running vmstat -w 1 would give you a different perspective also. -- Best regards, Gary
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1232387734.20030911183521>