From owner-svn-src-stable-8@FreeBSD.ORG Mon Feb 21 23:06:19 2011 Return-Path: Delivered-To: svn-src-stable-8@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50111106564A; Mon, 21 Feb 2011 23:06:19 +0000 (UTC) (envelope-from freebsd@nagilum.org) Received: from moutng.kundenserver.de (moutng.kundenserver.de [212.227.126.187]) by mx1.freebsd.org (Postfix) with ESMTP id C55A08FC14; Mon, 21 Feb 2011 23:06:18 +0000 (UTC) Received: from cakebox.homeunix.net (ip-109-85-227-42.web.vodafone.de [109.85.227.42]) by mrelayeu.kundenserver.de (node=mrbap2) with ESMTP (Nemesis) id 0M7VWZ-1QEHqs3TZ1-00wonO; Mon, 21 Feb 2011 23:53:42 +0100 Received: from cakebox.homeunix.net (localhost [127.0.0.1]) by cakebox.homeunix.net (Postfix) with ESMTP id 55A9F6793D; Mon, 21 Feb 2011 23:53:16 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.3.1 (2010-03-16) on cakebox.tis X-Spam-Level: X-Spam-Status: No, score=-2.9 required=3.0 tests=ALL_TRUSTED,BAYES_00 autolearn=ham version=3.3.1 Received: by cakebox.homeunix.net (Postfix, from userid 80) id EC2AB67934; Mon, 21 Feb 2011 23:53:14 +0100 (CET) Received: from cakebox.tis (cakebox.tis [10.1.1.1]) by 10.1.1.1 (Horde Framework) with HTTP; Mon, 21 Feb 2011 23:53:14 +0100 Message-ID: <20110221235314.69722w383i8eex44@10.1.1.1> Date: Mon, 21 Feb 2011 23:53:14 +0100 From: Alexander To: svn-src-stable-8@freebsd.org References: <201102211008.p1LA8lmt064969@svn.freebsd.org> In-Reply-To: <201102211008.p1LA8lmt064969@svn.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-2; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: 7bit User-Agent: Dynamic Internet Messaging Program (DIMP) H3 (1.1.6) X-Provags-ID: V02:K0:u8tFYLXEhZnz/5ULn/H9GoBKw4vcWKs9/vPrj/lrhm1 Z/sw1CIgtP+TibyqkoOE6qD8/6iUI5vR7HPOvagQgKBnHHQreM Fdldlh+SC7+M6yob4wGNyI3c5SyQJiovNg088LInvobWoSB4oZ ksvsIidSUfm09a2qgtP/K/AVPt97ShlxlpMnX69cJ9kQrm4LL8 mE4moFkxLp1Dyl1SZ3626Q/4bgyqRdMSr1JBLUv2C4= Cc: Martin Matuska Subject: Re: svn commit: r218911 - stable/8/usr.sbin/newsyslog X-BeenThere: svn-src-stable-8@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for only the 8-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Feb 2011 23:06:19 -0000 *Yay* But wouldn't it better to simply have a configurable postprocessor? That way one could also plug in custom scripts to upload old logs or whatever.. Alex. ----- Message from mm@FreeBSD.org --------- Date: Mon, 21 Feb 2011 10:08:47 +0000 (UTC) From: Martin Matuska Subject: svn commit: r218911 - stable/8/usr.sbin/newsyslog To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-8@freebsd.org > Author: mm > Date: Mon Feb 21 10:08:47 2011 > New Revision: 218911 > URL: http://svn.freebsd.org/changeset/base/218911 > > Log: > MFC r218127, r218128: > > Add xz(1) support to newsyslog. > Rewrite and simplify logfile compression code. > > Approved by: gad > > Modified: > stable/8/usr.sbin/newsyslog/newsyslog.8 > stable/8/usr.sbin/newsyslog/newsyslog.c > stable/8/usr.sbin/newsyslog/newsyslog.conf.5 > stable/8/usr.sbin/newsyslog/pathnames.h > Directory Properties: > stable/8/usr.sbin/newsyslog/ (props changed) > > Modified: stable/8/usr.sbin/newsyslog/newsyslog.8 > ============================================================================== > --- stable/8/usr.sbin/newsyslog/newsyslog.8 Mon Feb 21 09:56:08 > 2011 (r218910) > +++ stable/8/usr.sbin/newsyslog/newsyslog.8 Mon Feb 21 10:08:47 > 2011 (r218911) > @@ -17,7 +17,7 @@ > .\" the suitability of this software for any purpose. It is > .\" provided "as is" without express or implied warranty. > .\" > -.Dd July 23, 2010 > +.Dd January 31, 2011 > .Dt NEWSYSLOG 8 > .Os > .Sh NAME > @@ -279,6 +279,7 @@ accepted for backwards compatibility. > .Sh SEE ALSO > .Xr bzip2 1 , > .Xr gzip 1 , > +.Xr xz 1 , > .Xr syslog 3 , > .Xr newsyslog.conf 5 , > .Xr chown 8 , > > Modified: stable/8/usr.sbin/newsyslog/newsyslog.c > ============================================================================== > --- stable/8/usr.sbin/newsyslog/newsyslog.c Mon Feb 21 09:56:08 > 2011 (r218910) > +++ stable/8/usr.sbin/newsyslog/newsyslog.c Mon Feb 21 10:08:47 > 2011 (r218911) > @@ -57,12 +57,6 @@ > __FBSDID("$FreeBSD$"); > > #define OSF > -#ifndef COMPRESS_POSTFIX > -#define COMPRESS_POSTFIX ".gz" > -#endif > -#ifndef BZCOMPRESS_POSTFIX > -#define BZCOMPRESS_POSTFIX ".bz2" > -#endif > > #include > #include > @@ -92,10 +86,35 @@ __FBSDID("$FreeBSD$"); > #include "extern.h" > > /* > + * Compression suffixes > + */ > +#ifndef COMPRESS_SUFFIX_GZ > +#define COMPRESS_SUFFIX_GZ ".gz" > +#endif > + > +#ifndef COMPRESS_SUFFIX_BZ2 > +#define COMPRESS_SUFFIX_BZ2 ".bz2" > +#endif > + > +#ifndef COMPRESS_SUFFIX_XZ > +#define COMPRESS_SUFFIX_XZ ".xz" > +#endif > + > +#define COMPRESS_SUFFIX_MAXLEN MAX(MAX(sizeof(COMPRESS_SUFFIX_GZ),sizeof(COMPRESS_SUFFIX_BZ2)),sizeof(COMPRESS_SUFFIX_XZ)) > + > +/* > + * Compression types > + */ > +#define COMPRESS_TYPES 4 /* Number of supported compression types */ > + > +#define COMPRESS_NONE 0 > +#define COMPRESS_GZIP 1 > +#define COMPRESS_BZIP2 2 > +#define COMPRESS_XZ 3 > + > +/* > * Bit-values for the 'flags' parsed from a config-file entry. > */ > -#define CE_COMPACT 0x0001 /* Compact the archived log files with gzip. */ > -#define CE_BZCOMPACT 0x0002 /* Compact the archived log files with bzip2. */ > #define CE_BINARY 0x0008 /* Logfile is in binary, do not add status */ > /* messages to logfile(s) when rotating. */ > #define CE_NOSIGNAL 0x0010 /* There is no process to signal when */ > @@ -119,6 +138,19 @@ __FBSDID("$FreeBSD$"); > > #define MAX_OLDLOGS 65536 /* Default maximum number of old logfiles */ > > +struct compress_types { > + const char *flag; /* Flag in configuration file */ > + const char *suffix; /* Compression suffix */ > + const char *path; /* Path to compression program */ > +}; > + > +const struct compress_types compress_type[COMPRESS_TYPES] = { > + { "", "", "" }, /* no compression */ > + { "Z", COMPRESS_SUFFIX_GZ, _PATH_GZIP }, /* gzip compression */ > + { "J", COMPRESS_SUFFIX_BZ2, _PATH_BZIP2 }, /* bzip2 compression */ > + { "X", COMPRESS_SUFFIX_XZ, _PATH_XZ } /* xz compression */ > +}; > + > struct conf_entry { > STAILQ_ENTRY(conf_entry) cf_nextp; > char *log; /* Name of the log */ > @@ -134,7 +166,8 @@ struct conf_entry { > int hours; /* Hours between log trimming */ > struct ptime_data *trim_at; /* Specific time to do trimming */ > unsigned int permissions; /* File permissions on the log */ > - int flags; /* CE_COMPACT, CE_BZCOMPACT, CE_BINARY */ > + int flags; /* CE_BINARY */ > + int compress; /* Compression */ > int sig; /* Signal to send */ > int def_cfg; /* Using the rule for this file */ > }; > @@ -218,6 +251,7 @@ static int isnumberstr(const char *); > static int isglobstr(const char *); > static char *missing_field(char *p, char *errline); > static void change_attrs(const char *, const struct conf_entry *); > +static const char *get_logfile_suffix(const char *logfile); > static fk_entry do_entry(struct conf_entry *); > static fk_entry do_rotate(const struct conf_entry *); > static void do_sigwork(struct sigwork_entry *); > @@ -367,6 +401,7 @@ init_entry(const char *fname, struct con > tempwork->trim_at = ptime_init(src_entry->trim_at); > tempwork->permissions = src_entry->permissions; > tempwork->flags = src_entry->flags; > + tempwork->compress = src_entry->compress; > tempwork->sig = src_entry->sig; > tempwork->def_cfg = src_entry->def_cfg; > } else { > @@ -384,6 +419,7 @@ init_entry(const char *fname, struct con > tempwork->trim_at = NULL; > tempwork->permissions = 0; > tempwork->flags = 0; > + tempwork->compress = COMPRESS_NONE; > tempwork->sig = SIGHUP; > tempwork->def_cfg = 0; > } > @@ -448,14 +484,9 @@ do_entry(struct conf_entry * ent) > char temp_reason[REASON_MAX]; > > free_or_keep = FREE_ENT; > - if (verbose) { > - if (ent->flags & CE_COMPACT) > - printf("%s <%dZ>: ", ent->log, ent->numlogs); > - else if (ent->flags & CE_BZCOMPACT) > - printf("%s <%dJ>: ", ent->log, ent->numlogs); > - else > - printf("%s <%d>: ", ent->log, ent->numlogs); > - } > + if (verbose) > + printf("%s <%d%s>: ", ent->log, ent->numlogs, > + compress_type[ent->compress].flag); > ent->fsize = sizefile(ent->log); > modtime = age_old_log(ent->log); > ent->rotate = 0; > @@ -560,17 +591,10 @@ do_entry(struct conf_entry * ent) > ent->r_reason = strdup(temp_reason); > if (verbose) > printf("--> trimming log....\n"); > - if (noaction && !verbose) { > - if (ent->flags & CE_COMPACT) > - printf("%s <%dZ>: trimming\n", > - ent->log, ent->numlogs); > - else if (ent->flags & CE_BZCOMPACT) > - printf("%s <%dJ>: trimming\n", > - ent->log, ent->numlogs); > - else > - printf("%s <%d>: trimming\n", > - ent->log, ent->numlogs); > - } > + if (noaction && !verbose) > + printf("%s <%d%s>: trimming\n", ent->log, > + ent->numlogs, > + compress_type[ent->compress].flag); > free_or_keep = do_rotate(ent); > } else { > if (verbose) > @@ -1183,6 +1207,7 @@ parse_file(FILE *cf, struct cflist *work > } > > working->flags = 0; > + working->compress = COMPRESS_NONE; > q = parse = missing_field(sob(++parse), errline); > parse = son(parse); > eol = !*parse; > @@ -1261,7 +1286,7 @@ no_trimat: > working->flags |= CE_GLOB; > break; > case 'j': > - working->flags |= CE_BZCOMPACT; > + working->compress = COMPRESS_BZIP2; > break; > case 'n': > working->flags |= CE_NOSIGNAL; > @@ -1272,8 +1297,11 @@ no_trimat: > case 'w': > /* Depreciated flag - keep for compatibility purposes */ > break; > + case 'x': > + working->compress = COMPRESS_XZ; > + break; > case 'z': > - working->flags |= CE_COMPACT; > + working->compress = COMPRESS_GZIP; > break; > case '-': > break; > @@ -1415,7 +1443,7 @@ static void > delete_oldest_timelog(const struct conf_entry *ent, const char *archive_dir) > { > char *logfname, *s, *dir, errbuf[80]; > - int logcnt, max_logcnt, dirfd, i; > + int dirfd, i, logcnt, max_logcnt, valid; > struct oldlog_entry *oldlogs; > size_t logfname_len; > struct dirent *dp; > @@ -1485,9 +1513,12 @@ delete_oldest_timelog(const struct conf_ > "match time format\n", dp->d_name); > continue; > } > - if (*s != '\0' && !(strcmp(s, BZCOMPRESS_POSTFIX) == 0 || > - strcmp(s, COMPRESS_POSTFIX) == 0)) { > - if (verbose) > + > + for (int c = 0; c < COMPRESS_TYPES; c++) > + if (strcmp(s, compress_type[c].suffix) == 0) > + valid = 1; > + if (valid != 1) { > + if (verbose) > printf("Ignoring %s which has unexpected " > "extension '%s'\n", dp->d_name, s); > continue; > @@ -1586,13 +1617,35 @@ add_to_queue(const char *fname, struct i > STAILQ_INSERT_TAIL(inclist, inc, inc_nextp); > } > > +/* > + * Search for logfile and return its compression suffix (if supported) > + * The suffix detection is first-match in the order of compress_types > + * > + * Note: if logfile without suffix exists (uncompressed, COMPRESS_NONE) > + * a zero-length string is returned > + */ > +static const char * > +get_logfile_suffix(const char *logfile) > +{ > + struct stat st; > + char zfile[MAXPATHLEN]; > + > + for (int c = 0; c < COMPRESS_TYPES; c++) { > + (void) strlcpy(zfile, logfile, MAXPATHLEN); > + (void) strlcat(zfile, compress_type[c].suffix, MAXPATHLEN); > + if (lstat(zfile, &st) == 0) > + return (compress_type[c].suffix); > + } > + return (NULL); > +} > + > static fk_entry > do_rotate(const struct conf_entry *ent) > { > char dirpart[MAXPATHLEN], namepart[MAXPATHLEN]; > char file1[MAXPATHLEN], file2[MAXPATHLEN]; > char zfile1[MAXPATHLEN], zfile2[MAXPATHLEN]; > - char jfile1[MAXPATHLEN]; > + const char *logfile_suffix; > char datetimestr[30]; > int flags, numlogs_c; > fk_entry free_or_keep; > @@ -1650,19 +1703,13 @@ do_rotate(const struct conf_entry *ent) > delete_oldest_timelog(ent, dirpart); > else { > /* name of oldest log */ > - (void) snprintf(zfile1, sizeof(zfile1), "%s%s", file1, > - COMPRESS_POSTFIX); > - snprintf(jfile1, sizeof(jfile1), "%s%s", file1, > - BZCOMPRESS_POSTFIX); > - > - if (noaction) { > - printf("\trm -f %s\n", file1); > - printf("\trm -f %s\n", zfile1); > - printf("\trm -f %s\n", jfile1); > - } else { > - (void) unlink(file1); > - (void) unlink(zfile1); > - (void) unlink(jfile1); > + for (int c = 0; c < COMPRESS_TYPES; c++) { > + (void) snprintf(zfile1, sizeof(zfile1), "%s%s", file1, > + compress_type[c].suffix); > + if (noaction) > + printf("\trm -f %s\n", zfile1); > + else > + (void) unlink(zfile1); > } > } > > @@ -1697,24 +1744,14 @@ do_rotate(const struct conf_entry *ent) > (void) snprintf(file1, sizeof(file1), "%s.%d", > ent->log, numlogs_c); > > - (void) strlcpy(zfile1, file1, sizeof(zfile1)); > - (void) strlcpy(zfile2, file2, sizeof(zfile2)); > - if (lstat(file1, &st)) { > - (void) strlcat(zfile1, COMPRESS_POSTFIX, > - sizeof(zfile1)); > - (void) strlcat(zfile2, COMPRESS_POSTFIX, > - sizeof(zfile2)); > - if (lstat(zfile1, &st)) { > - strlcpy(zfile1, file1, sizeof(zfile1)); > - strlcpy(zfile2, file2, sizeof(zfile2)); > - strlcat(zfile1, BZCOMPRESS_POSTFIX, > - sizeof(zfile1)); > - strlcat(zfile2, BZCOMPRESS_POSTFIX, > - sizeof(zfile2)); > - if (lstat(zfile1, &st)) > - continue; > - } > - } > + logfile_suffix = get_logfile_suffix(file1); > + if (logfile_suffix == NULL) > + continue; > + (void) strlcpy(zfile1, file1, MAXPATHLEN); > + (void) strlcpy(zfile2, file2, MAXPATHLEN); > + (void) strlcat(zfile1, logfile_suffix, MAXPATHLEN); > + (void) strlcat(zfile2, logfile_suffix, MAXPATHLEN); > + > if (noaction) > printf("\tmv %s %s\n", zfile1, zfile2); > else { > @@ -1760,7 +1797,7 @@ do_rotate(const struct conf_entry *ent) > swork = NULL; > if (ent->pid_file != NULL) > swork = save_sigwork(ent); > - if (ent->numlogs > 0 && (flags & (CE_COMPACT | CE_BZCOMPACT))) { > + if (ent->numlogs > 0 && ent->compress > COMPRESS_NONE) { > /* > * The zipwork_entry will include a pointer to this > * conf_entry, so the conf_entry should not be freed. > @@ -1855,15 +1892,16 @@ do_zipwork(struct zipwork_entry *zwork) > > pgm_path = NULL; > strlcpy(zresult, zwork->zw_fname, sizeof(zresult)); > - if (zwork != NULL && zwork->zw_conf != NULL) { > - if (zwork->zw_conf->flags & CE_COMPACT) { > - pgm_path = _PATH_GZIP; > - strlcat(zresult, COMPRESS_POSTFIX, sizeof(zresult)); > - } else if (zwork->zw_conf->flags & CE_BZCOMPACT) { > - pgm_path = _PATH_BZIP2; > - strlcat(zresult, BZCOMPRESS_POSTFIX, sizeof(zresult)); > + if (zwork != NULL && zwork->zw_conf != NULL && > + zwork->zw_conf->compress > COMPRESS_NONE) > + for (int c = 1; c < COMPRESS_TYPES; c++) { > + if (zwork->zw_conf->compress == c) { > + pgm_path = compress_type[c].path; > + (void) strlcat(zresult, > + compress_type[c].suffix, sizeof(zresult)); > + break; > + } > } > - } > if (pgm_path == NULL) { > warnx("invalid entry for %s in do_zipwork", zwork->zw_fname); > return; > @@ -2141,9 +2179,8 @@ static int > age_old_log(char *file) > { > struct stat sb; > - char *endp; > - char tmp[MAXPATHLEN + sizeof(".0") + sizeof(COMPRESS_POSTFIX) + > - sizeof(BZCOMPRESS_POSTFIX) + 1]; > + const char *logfile_suffix; > + char tmp[MAXPATHLEN + sizeof(".0") + COMPRESS_SUFFIX_MAXLEN + 1]; > > if (archtodir) { > char *p; > @@ -2173,21 +2210,12 @@ age_old_log(char *file) > } > > strlcat(tmp, ".0", sizeof(tmp)); > - if (stat(tmp, &sb) < 0) { > - /* > - * A plain '.0' file does not exist. Try again, first > - * with the added suffix of '.gz', then with an added > - * suffix of '.bz2' instead of '.gz'. > - */ > - endp = strchr(tmp, '\0'); > - strlcat(tmp, COMPRESS_POSTFIX, sizeof(tmp)); > - if (stat(tmp, &sb) < 0) { > - *endp = '\0'; /* Remove .gz */ > - strlcat(tmp, BZCOMPRESS_POSTFIX, sizeof(tmp)); > - if (stat(tmp, &sb) < 0) > - return (-1); > - } > - } > + logfile_suffix = get_logfile_suffix(tmp); > + if (logfile_suffix == NULL) > + return (-1); > + (void) strlcat(tmp, logfile_suffix, sizeof(tmp)); > + if (stat(tmp, &sb) < 0) > + return (-1); > return ((int)(ptimeget_secs(timenow) - sb.st_mtime + 1800) / 3600); > } > > > Modified: stable/8/usr.sbin/newsyslog/newsyslog.conf.5 > ============================================================================== > --- stable/8/usr.sbin/newsyslog/newsyslog.conf.5 Mon Feb 21 09:56:08 > 2011 (r218910) > +++ stable/8/usr.sbin/newsyslog/newsyslog.conf.5 Mon Feb 21 10:08:47 > 2011 (r218911) > @@ -21,7 +21,7 @@ > .\" the suitability of this software for any purpose. It is > .\" provided "as is" without express or implied warranty. > .\" > -.Dd July 23, 2010 > +.Dd January 31, 2011 > .Dt NEWSYSLOG.CONF 5 > .Os > .Sh NAME > @@ -292,6 +292,12 @@ indicates that > should attempt to save disk space by compressing the rotated > log file using > .Xr bzip2 1 . > +.It Cm X > +indicates that > +.Xr newsyslog 8 > +should attempt to save disk space by compressing the rotated > +log file using > +.Xr xz 1 . > .It Cm N > indicates that there is no process which needs to be signaled > when this log file is rotated. > @@ -346,6 +352,7 @@ signal will be sent. > .Sh SEE ALSO > .Xr bzip2 1 , > .Xr gzip 1 , > +.Xr xz 1 , > .Xr syslog 3 , > .Xr chown 8 , > .Xr newsyslog 8 , > > Modified: stable/8/usr.sbin/newsyslog/pathnames.h > ============================================================================== > --- stable/8/usr.sbin/newsyslog/pathnames.h Mon Feb 21 09:56:08 > 2011 (r218910) > +++ stable/8/usr.sbin/newsyslog/pathnames.h Mon Feb 21 10:08:47 > 2011 (r218911) > @@ -22,7 +22,8 @@ provided "as is" without express or impl > > */ > > -#define _PATH_CONF "/etc/newsyslog.conf" > -#define _PATH_SYSLOGPID _PATH_VARRUN "syslog.pid" > -#define _PATH_BZIP2 "/usr/bin/bzip2" > -#define _PATH_GZIP "/usr/bin/gzip" > +#define _PATH_CONF "/etc/newsyslog.conf" > +#define _PATH_SYSLOGPID _PATH_VARRUN "syslog.pid" > +#define _PATH_BZIP2 "/usr/bin/bzip2" > +#define _PATH_GZIP "/usr/bin/gzip" > +#define _PATH_XZ "/usr/bin/xz" > _______________________________________________ > svn-src-stable-8@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/svn-src-stable-8 > To unsubscribe, send any mail to "svn-src-stable-8-unsubscribe@freebsd.org" > ----- End message from mm@FreeBSD.org ----- ---------------------------------------------------------------- cakebox.homeunix.net - all the machine one needs..