From owner-freebsd-questions@FreeBSD.ORG Sun Aug 4 00:05:18 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 46217F61 for ; Sun, 4 Aug 2013 00:05:18 +0000 (UTC) (envelope-from frank2@fjl.co.uk) Received: from bs1.fjl.org.uk (bs1.fjl.org.uk [84.45.41.196]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9E3E7228F for ; Sun, 4 Aug 2013 00:05:17 +0000 (UTC) Received: from [192.168.1.35] (mux.fjl.org.uk [62.3.120.246]) (authenticated bits=0) by bs1.fjl.org.uk (8.14.4/8.14.4) with ESMTP id r7405FCP041261 (version=TLSv1/SSLv3 cipher=DHE-DSS-CAMELLIA256-SHA bits=256 verify=NO) for ; Sun, 4 Aug 2013 01:05:15 +0100 (BST) (envelope-from frank2@fjl.co.uk) Message-ID: <51FD9ABC.1010408@fjl.co.uk> Date: Sun, 04 Aug 2013 01:05:16 +0100 From: Frank Leonhardt User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: freebsd-questions@freebsd.org Subject: Re: Archiving a log file References: <51FD8E19.90403@fjl.co.uk> <20130803232018.GA59293@neutralgood.org> In-Reply-To: <20130803232018.GA59293@neutralgood.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.14 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 04 Aug 2013 00:05:18 -0000 On 04/08/2013 00:20, kpneal@pobox.com wrote: > On Sun, Aug 04, 2013 at 12:11:21AM +0100, Frank Leonhardt wrote: >> The answer isn't (AFAIK) newsyslog >> >> As a one-off, I need to archive an old log file - say httpd-access.log - >> while its still open. I don't want this to happen automatically and I >> don't want to set up newsyslog or anything like that. And I really don't >> want to mess about with signals to whatever is writing to the file, even >> assuming the writer could respond to them. I can't just rename the file >> as it's open for writing, and there would also be a good chance that >> something will be added to the file while it's being compressed. >> >> What I actually do is: >> >> cp httpd-access.log httpd-access.log-03-Aug-13 && :> httpd-access.log && >> bzip2 httpd-access.log-03-Aug-13 >> >> Data might be lost here as something may be added between the cp being >> completed and the file being truncated. It's not the end of the world if >> this happens, but is there a better way? I could always shut down Apache >> for the duration, but I don't want to do that either, so in this case >> I'm happy to take the risk (it's not like I'm likely to miss anything >> that important). >> >> I don't know if this can be relied on as a POSIX thing, but the cp >> command simply(!) issues read() and write() calls until read() fails to >> get any more bytes, so if data is being appended to the file after cp is >> started it'll still be copied. Therefore the window where stuff could be >> written after the copy but before the truncation is shortened, but extant. >> >> So what's the magic utility I don't know about? > How about cronolog? I use it with Apache where Apache logs to cronolog > and cronolog handles the rotating of the logs. No signals. No races. > It even makes a symlink pointing at the newest log file. > > It doesn't seem to have a way to compress logs, but you could probably > script up something that wakes up every so often and compresses files > if a newer file exists. Thanks for the suggestion - I wasn't aware of cronolog. Unfortunately it doesn't work on existing log files, only stuff piped to it from the start, so it won't help here. I can see it being very handy in other situations though. httpd-access.log was just an example of such a file, but I'm looking for a general solution.