Date: Sun, 31 Jul 2011 10:31:29 -0700 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Eugene Grosbein <egrosbein@rdtc.ru> Cc: stable@freebsd.org Subject: Re: running newsyslog fiveminly Message-ID: <20110731173129.GA53635@icarus.home.lan> In-Reply-To: <4E35881C.2010505@rdtc.ru> References: <4E35881C.2010505@rdtc.ru>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Jul 31, 2011 at 11:51:40PM +0700, Eugene Grosbein wrote:
> Hi!
>
> Suppose, there is a machine which writes two kinds of log files through syslogd:
> quickly-growing that need to be rotated based on their size (hourly is too seldom)
> and other that should be rotated once a day, at midnight only.
>
> For first kind of logs we have to run newsyslog once every 5 minutes using cron:
>
> */5 * * * * root newsyslog
>
> For second kind of logs we have lines in newsyslog.conf such as following:
>
> /var/log/mpd.log 640 16 * @T0000 JC
>
> This must ensure that /var/log/mpd.log is rotated and compressed at midnigt only.
> Note, that compressing the file takes 8 minutes.
>
> However, every night at 00:05 I get an error:
>
> bzip2: I/O or other error, bailing out. Possible reason follows.
> bzip2: No such file or directory
> Input file = /var/log/mpd.log.0, output file = /var/log/mpd.log.0.bz2
> newsyslog: `bzip2 -f /var/log/mpd.log.0' terminated with a non-zero status (1)
>
> It seems, newsyslog still wants to process my file at 00:05 despite @T0000
> time specification. Is it broken?
I have three things to say on the matter, all of which are somewhat
independent of one another so please keep that in mind. I imagine #1
below is your problem.
1) The newsyslog.conf(5) man page has this clause in it, for the "when"
field (in your case, @T0000):
when ... If the when field contains an asterisk (`*'), log rotation
will solely depend on the contents of the size field. Otherwise,
the when field consists of an optional interval in hours, usually
followed by an `@'-sign and a time in restricted ISO 8601 format.
If a time is specified, the log file will only be trimmed if
newsyslog(8) is run within one hour of the specified time. If an
interval is specified, the log file will be trimmed if that many
hours have passed since the last rotation. ...
You might think that "one hour of the specified time" value/clause
correlates with the interval that newsyslog is run at via cron, but that
would be wrong. newsyslog REALLY DOES have hard-coded values for 3600
seconds (1 hour) in it (grep -r 3600 /usr/src/usr.sbin/newsyslog). I
have not looked at the code, but the fact of the matter is, 1 hour
appears to be a "special" value. I would heed that as a warning.
2) Are you absolutely sure mpd.log is being rotated AND compressed within
the 5 minute window? If mpd.log is extremely large and your disks are
slow, this could take a long time. If possible, try (temporarily)
removing bzip2 from the picture (remove J flag).
3) mpd(8) logs via syslog(3). When newsyslog(8), are you aware that it
sends a SIGHUP to syslogd(8)? As such, are you absolutely certain when
this happen (every 5 minutes!) that the new log files are getting
created correctly and promptly?
4) To debug this, you're probably going to need to run some cronjobs or
daemons that keep a very close eye on /var/log/mpd.log* when the log
rotation runs, in combination with running syslogd(8) in debug mode
and/or verbose mode.
5) Why do you need to rotate logs every 5 minutes? Why do you need such
extreme levels of granularity in your rotated logs? Just how much data
are you logging via syslog? If a lot, why so much? It might be more
effective to consider expanding your logging infrastructure to multiple
machines if this the case.
--
| Jeremy Chadwick jdc at parodius.com |
| Parodius Networking http://www.parodius.com/ |
| UNIX Systems Administrator Mountain View, CA, US |
| Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110731173129.GA53635>