Date: Tue, 18 May 1999 08:09:46 -0700 From: Graeme Tait <graeme@echidna.com> To: Carol Deihl <carol@tinker.com> Cc: ndear@areti.net, freebsd-isp@freebsd.org Subject: Re: Web Statistics break up program. Message-ID: <374182BA.6ED@echidna.com> References: <199905172017.VAA08078@post.mail.areti.net> <3740B3C5.947395D0@tinker.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Carol Deihl wrote: > > FreeBSD already has the parts to do a nice job, just requires > a little scripting to get it going. Here's what we do. > > I wrote a little perl script called /etc/rotatelogs that > rotates the logs for our virtual domains just after midnight. > It uses the "newsyslog" program to do this, which takes care > of removing aged log files. (The script is below.) I leave the > just-rotated access log un-gzipped, since the stats program will > be looking at it soon. I use a separate newsyslog configuration > file for each virtual domain, in case I want to do something > special for a particular domain. Below I've included an > example config file. The scripts assume that these config files > are in a directory /etc/newsyslog.confs, named with the > domain name. <snip> Thanks for the detailed answer - this is most helpful. I was about to try to figure out how to do this sort of thing in a more automated way. A few questions: I'm not sure what web server you use, but it looks like (by default) you are having newsyslog issue it a SIGHUP after rotating each log. With Apache, as I understand it, that kills off all the Apache children abruptly, interrupting transfers in progress, and requiring it to restart from zero children. It also zeros out the server statistics. Interrupting transfers in progress is not a great idea for our server, as it's taking orders worldwide, and even though it's less busy in the wee hours (US time), it's not idle. Also, some files being downloaded are large, and we wouldn't want to interrupt those transfers unnecessarily. OTOH, if you use SIGUSR1 with Apache (http://www.apache.org/docs-1.2/stopping.html), which lets existing transfers complete, there is now way of knowing when they will complete, although for the logging task you can readily wait an hour or two before using the file. The problem I see, and please correct me if I'm wrong, is that newsyslog is going to change the mode/ownership of the log files right away, which will (depending on the new mode/ownership) prevent Apache from writing the files. The other problem in this case is that the rotated log may contain a few (possibly quite) delayed entries, which results in some disorder when you concatenate successive rotated logs for analysis. Also I notice you use mode 664 for the logs - wouldn't you normally want log files private to the owner (600)? The other thing that puzzled me is that you seem to have a server process for each domain (path-to-httpd-pid-file-for-mydomain.com) and that you list that after each log file. Doesn't that mean the server gets SIGHUP'd twice? In a typical situation with Apache, there would be one parent process (one PID to be signalled), but there could be many log files (for many virtual domains, say) being generated by this process. Does that mean SIGHUP'ping it as many times as there are log files? -- Graeme Tait - Echidna To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?374182BA.6ED>