From owner-freebsd-isp Thu May 20 17:17: 1 1999 Delivered-To: freebsd-isp@freebsd.org Received: from metva.com.au (metva.metva.com.au [202.0.82.1]) by hub.freebsd.org (Postfix) with ESMTP id E8AF314DDF for ; Thu, 20 May 1999 17:16:41 -0700 (PDT) (envelope-from enno.davids@metva.com.au) Received: (from enno@localhost) by metva.com.au id KAA08044 for freebsd-isp@FreeBSD.ORG; Fri, 21 May 1999 10:16:30 +1000 (EST) From: Enno Davids Message-Id: <199905210016.KAA08044@metva.com.au> Subject: Re: Web Statistics break up program. In-Reply-To: from Leif Neland at "May 20, 99 11:02:40 pm" To: freebsd-isp@FreeBSD.ORG Date: Fri, 21 May 1999 10:16:29 +1000 (EST) X-Mailer: ELM [version 2.4ME+ PL39 (25)] MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-isp@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.org Various people have written: | > > > This will create an archived logfile (http.site.May_1999) and erase | > > > the original without needing to kill -1 the httpd. | > > > | > > > #!/bin/sh - | > > > LOGDIR=/var/log | > > > ARCDIR=/var/log/oldlogs | > > > DAY=`date | awk '{ OFS="_" ;print $2,$6}' ` | > > > for log in $LOGDIR/http* ; do | > > > cp $log $ARCDIR/${log}.${DAY} | > > > chmod 440 $ARCDIR/${log}.${DAY} | > > > cp /dev/null $log | > > > done | > > | > > Egads!! | > > That's a pretty vicious race condition there, you'll lose records on busy | > > servers. | > | > In theory perhaps, in reality it doesn't. I've never seen this algorithm | > fail, even when used on log files that grow by several megabytes per day. | > | How do you know? How do you know if you are throwing some lines away which | happened between you started copying, and you nulled the logfile? Guys, can we end this thread. The correct solution, is to 'mv' your logfile to another temporary name in the same filesystem, then null the real name & reset owners and permissions, then signal apache and then move the logfile away to the archived name. The initial move inside the same filesystem ensures that its out of the way when you create the nulled file, but being in the same filesystem apache can still write it (under its tmp name) with its open file descriptor. The signal cuts over the web server to the new (nulled) file without losing data and then you can do whatever you want/need to the saved logfile. So, the middle of the loop above wants to be something like... mv ${log} ${log}"_tmp" cp /dev/null ${log} chown www ${log} apachectl restart mv ${log}"_tmp" ${ARCDIR}/${log}.${DAY} In the presence of lots of virtual hosts you need to repeat the first stanza for each one before signalling apache and then mv each file after the signal, but thats about all the subtlety there is. I'll note that I suspect from the loop itself that you're trying to pick up the virtual hosts that way, which means to do this solution properly you want two loops with the restart in the middle of them (outside/between both of them). A short sleep after the restart to let apache flush buffers and cut over is also in order usually. Cheers, Enno. ------- Enno Davids Metva P/L, P.O.Box 2669, Phone: +61 3 9583 5474 enno.davids@metva.com.au Cheltenham 3192, Australia Mobile: +61 15 316 522 To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message