Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 21 May 1999 10:16:29 +1000 (EST)
From:      Enno Davids <enno.davids@metva.com.au>
To:        freebsd-isp@FreeBSD.ORG
Subject:   Re: Web Statistics break up program.
Message-ID:  <199905210016.KAA08044@metva.com.au>
In-Reply-To: <Pine.BSF.4.05.9905202300420.37469-100000@arnold.neland.dk> from Leif Neland at "May 20, 99 11:02:40 pm"

next in thread | previous in thread | raw e-mail | index | archive | help
Various people have written:
| > > > This will create an archived logfile (http.site.May_1999) and erase
| > > > the original without needing to kill -1 the httpd.
| > > > 
| > > > 	#!/bin/sh -
| > > > 	LOGDIR=/var/log
| > > > 	ARCDIR=/var/log/oldlogs
| > > > 	DAY=`date | awk '{ OFS="_" ;print $2,$6}' `
| > > > 	for log in $LOGDIR/http* ; do
| > > > 		cp $log $ARCDIR/${log}.${DAY}
| > > > 		chmod 440 $ARCDIR/${log}.${DAY}
| > > > 		cp /dev/null $log
| > > > 	done
| > > 
| > > Egads!!
| > > That's a pretty vicious race condition there, you'll lose records on busy
| > > servers.
| > 
| > In theory perhaps, in reality it doesn't.  I've never seen this algorithm 
| > fail, even when used on log files that grow by several megabytes per day.
| > 
| How do you know? How do you know if you are throwing some lines away which
| happened between you started copying, and you nulled the logfile?

Guys, can we end this thread. The correct solution, is to 'mv' your logfile
to another temporary name in the same filesystem, then null the real name &
reset owners and permissions, then signal apache and then move the logfile
away to the archived name.

The initial move inside the same filesystem ensures that its out of the way
when you create the nulled file, but being in the same filesystem apache can
still write it (under its tmp name) with its open file descriptor. The signal
cuts over the web server to the new (nulled) file without losing data and
then you can do whatever you want/need to the saved logfile.

So, the middle of the loop above wants to be something like...
		
		mv ${log} ${log}"_tmp"
		cp /dev/null ${log}
		chown www ${log}
		apachectl restart
		mv ${log}"_tmp" ${ARCDIR}/${log}.${DAY}

In the presence of lots of virtual hosts you need to repeat the first stanza
for each one before signalling apache and then mv each file after the signal,
but thats about all the subtlety there is. I'll note that I suspect from the
loop itself that you're trying to pick up the virtual hosts that way, which
means to do this solution properly you want two loops with the restart in the
middle of them (outside/between both of them). A short sleep after the restart
to let apache flush buffers and cut over is also in order usually.


Cheers,

Enno.


-------
Enno Davids	            Metva P/L, P.O.Box 2669,     Phone: +61 3 9583 5474
enno.davids@metva.com.au   Cheltenham 3192, Australia  	 Mobile: +61 15 316 522



To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-isp" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199905210016.KAA08044>