Date: Tue, 18 May 1999 16:35:02 -0500 From: Carol Deihl <carol@tinker.com> To: Graeme Tait <graeme@echidna.com> Cc: ndear@areti.net, freebsd-isp@freebsd.org Subject: Re: Web Statistics break up program. Message-ID: <3741DD06.CF1C48B4@tinker.com> References: <199905172017.VAA08078@post.mail.areti.net> <3740B3C5.947395D0@tinker.com> <374182BA.6ED@echidna.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi Graeme, Graeme Tait wrote: > Thanks for the detailed answer - this is most helpful. Glad to help... > A few questions: > > I'm not sure what web server you use, but it looks like (by default) > you are having newsyslog issue it a SIGHUP after rotating each log. > With Apache, as I understand it, that kills off all the Apache > children abruptly, interrupting transfers in progress, and requiring > it to restart from zero children. It also zeros out the server > statistics. Interrupting transfers in progress is not a great idea > for our server, as it's taking orders worldwide, and even though it's > less busy in the wee hours (US time), it's not idle. Also, some files > being downloaded are large, and we wouldn't want to interrupt those > transfers unnecessarily. Good point. Yes, I use Apache. In our case, our domains generally don't transfer very large files, so the likelihood of interrupting a transfer is pretty small (the HUP executes very fast). For a client with a worldwide intranet, we actually rotate the logs on Friday at 4:00 pm, since that's the lowest usage time... > OTOH, if you use SIGUSR1 with Apache > (http://www.apache.org/docs-1.2/stopping.html), which lets existing > transfers complete, there is now way of knowing when they will > complete, although for the logging task you can readily wait an hour > or two before using the file. The problem I see, and please correct > me if I'm wrong, is that newsyslog is going to change the > mode/ownership of the log files right away, which will (depending on > the new mode/ownership) prevent Apache from writing the files. The mode/ownership in my script just duplicates how the log files are setup anyway, so there isn't a change, so Apache could still keep writing them, I believe. I'll have to try the SIGUSR1 approach - I've been using Apache since before the USR1 approach was stable. > The > other problem in this case is that the rotated log may contain a few > (possibly quite) delayed entries, which results in some disorder when > you concatenate successive rotated logs for analysis. I presume this is true, but in our case we config wusage to look at the relevant log files, and it seems to figure out everything correctly. Since the logs aren't being processed in real time, you could also run sort on them, if analog requires them to be in time order. > Also I notice you use mode 664 for the logs - wouldn't you normally > want log files private to the owner (600)? > > The other thing that puzzled me is that you seem to have a server > process for each domain (path-to-httpd-pid-file-for-mydomain.com) and > that you list that after each log file. Doesn't that mean the server > gets SIGHUP'd twice? > > In a typical situation with Apache, there would be one parent process > (one PID to be signalled), but there could be many log files (for > many virtual domains, say) being generated by this process. Does that > mean SIGHUP'ping it as many times as there are log files? Oops, I forgot to mention that we run a separate server within a chroot'd area for each domain. Our original clients wanted more security than average, and this way one client can't affect another client's files at all, even with a buggy (or malicious cgi). So each parent server only gets HUP'd once. And the ownership of the log files isn't an issue for us. If you're using the Apache virtual domain directives and separate log files for each domain... hmmmm... seems like it wouldn't make sense to use SIGUSR1 after each rotate, but you could SIGHUP, with the problems you already mentioned. Sounds like just using one giant log file with SIGUSR1 might work in your case; then you'd (I presume) split the log file into separate domains before passing to the stats program. -- Carol Deihl - carol@tinker.com Shrier and Deihl - Unix Network Admin and Internet Software Development To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3741DD06.CF1C48B4>