From owner-freebsd-isp@FreeBSD.ORG Wed Jan 23 18:25:55 2008 Return-Path: Delivered-To: freebsd-isp@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 09DB816A417 for ; Wed, 23 Jan 2008 18:25:55 +0000 (UTC) (envelope-from andreas@klemm.apsfilter.org) Received: from srv1.cosmo-project.de (srv1.cosmo-project.de [213.83.6.106]) by mx1.freebsd.org (Postfix) with ESMTP id 7DA2A13C459 for ; Wed, 23 Jan 2008 18:25:54 +0000 (UTC) (envelope-from andreas@klemm.apsfilter.org) Received: from srv1.cosmo-project.de (localhost [IPv6:::1]) by srv1.cosmo-project.de (8.12.10/8.12.10) with ESMTP id m0NHF6Z4035177 (version=TLSv1/SSLv3 cipher=EDH-RSA-DES-CBC3-SHA bits=168 verify=NO) for ; Wed, 23 Jan 2008 18:15:06 +0100 (CET) (envelope-from andreas@klemm.apsfilter.org) Received: (from uucp@localhost) by srv1.cosmo-project.de (8.12.10/8.12.10/Submit) with UUCP id m0NHF6tp035176 for freebsd-isp@freebsd.org; Wed, 23 Jan 2008 18:15:06 +0100 (CET) (envelope-from andreas@klemm.apsfilter.org) Received: from titan.klemm.apsfilter.org (localhost.klemm.apsfilter.org [127.0.0.1]) by klemm.apsfilter.org (8.14.2/8.13.4) with ESMTP id m0NHBS1w019562 for ; Wed, 23 Jan 2008 18:11:28 +0100 (CET) (envelope-from andreas@titan.klemm.apsfilter.org) Received: (from andreas@localhost) by titan.klemm.apsfilter.org (8.14.2/8.13.8/Submit) id m0NHBSvU019339 for freebsd-isp@freebsd.org; Wed, 23 Jan 2008 18:11:28 +0100 (CET) (envelope-from andreas) Date: Wed, 23 Jan 2008 18:11:28 +0100 From: Andreas Klemm To: freebsd-isp@freebsd.org Message-ID: <20080123171128.GA2398@titan.klemm.apsfilter.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline X-Operating-System: FreeBSD 6.3-STABLE X-Disclaimer: A free society is one where it is safe to be unpopular User-Agent: Mutt/1.5.15 (2007-04-06) Subject: central logfile / real time logging X-BeenThere: freebsd-isp@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Internet Services Providers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jan 2008 18:25:55 -0000 Hi, do you have an idea how to store application generated logfiles (i.e. apache, ... ) from 1000 Solaris "zones" running on 250 physical Sun server onto a central FreeBSD logserver ? There are 6 strong demands, which makes this setup very special and tricky. After many thinking and googeling I didn't find any OpenSource or at least "not too expensive" solution which fulfills all of these demands: - "reliability", no logmessage may get lost - on the central logserver the logfiles need to stay in separate files per virtual zone and application (can be the case the zone runs multiple services) - "real-time logging", as soon as an application logrecord gets written on one of the 1000 zones this logmessage should be written immediately onto the central logserver. So its not sufficient to make this batched on an hourly base. - applications may not write to NFS directly to prevent locking of apps if NFS server is unavailable (if solution should be NFS based). - this solution needs to be aware of logfile rotation on the application server side (the 1000 zones) - this logging needs to be done from the "global zone" of a Solaris server. Some people seem to use syslog-ng for this on client and central repository side. But I see here some problems. I need to convert apache logs from normal logfiles. This must happen from the Global Zones. Then the syslog-ng on the central side has no chance (I think) to demultiplex the logfile stream into logs separated by the different zone and application. On the Global Zones I need a process that tracks some logfiles that are from different virtual zones. I need to open one tcp session to central archive per logfile. So ~250 global zones open ~1000 tcp sessions towards the central FreeBSD server. There must be a way to tell the application on central FreeBSD Server, that this stream now should be written to a certain subdirectory and filename, so that all is separate. This application need to buffer on the Global zone, if the server is not available. Or at least keep track, what hasn't been written yet. And must be aware of logfile rotation. Any idea ? Andreas /// -- Andreas Klemm - Powered by FreeBSD 6 Need a magic printfilter today ? -> http://www.apsfilter.org/