From owner-freebsd-questions Wed May 1 18:25:38 2002 Delivered-To: freebsd-questions@freebsd.org Received: from pintail.mail.pas.earthlink.net (pintail.mail.pas.earthlink.net [207.217.120.122]) by hub.freebsd.org (Postfix) with ESMTP id 290E937B404 for ; Wed, 1 May 2002 18:25:35 -0700 (PDT) Received: from user-119a7q7.biz.mindspring.com ([66.149.31.71] helo=athlon.storm2k.com) by pintail.mail.pas.earthlink.net with esmtp (Exim 3.33 #2) id 1735Li-00031h-00 for questions@FreeBSD.ORG; Wed, 01 May 2002 18:25:34 -0700 Message-Id: <5.1.0.14.0.20020501192418.02cea050@pop.wsonline.net> X-Sender: storm2k@pop.wsonline.net X-Mailer: QUALCOMM Windows Eudora Version 5.1 Date: Wed, 01 May 2002 19:29:29 -0600 To: questions@FreeBSD.ORG From: RichardH Subject: Parsing Log Files Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG This was posted to webhostingtalk.com and the Apacheusers mail list and still no answers. I think most of them are running small 300 or so user systems so the idea proposed in the resonse below would be fine for them. We are looking @ 2000+ users per server (the servers can handle that with no problem due to their configurations). This will be virtual hosting. Please check the following and let me know of any ideas on doing this correctly. Original post (me): I have looked all in here and on the net for more specific info on parsing access logs for multiple user accounts. If I have 300 users on a server and want to have a "log" directory in each users home directory is there a script or a simple grep that will pull that users info from the raw access logs and split/parse the info out to the individual log directory in each users account so that it can then be viewed by the user with AWStats, Analog, etc. I know this has to be accomplished failry easily but cannot find any links or data giving the exact way of doing it. Any input would be greatly appreciated. OS is FreeBSD4.5 w/Apache Web Server 1.3.24. Response: You can just put the TransferLog directive in all of your virtualhosts. Second post (me): The only problem with using the transferlog directive, is that for each an every virtual host entry that is used in the manner, you have another open file process. In time this will overload your server and create server slow down. By parsing out the files with a script, it reduces overall server load AND permits the use of rewrite rules, that allow you to use a virtmap.txt type of setup for hosting entries (in which case the transferlog entry does not work at all). The transferlog directive is fine for a small production server, where you are hosting a few accounts, but for a large application, is not such a wise idea. The above post says 300 users as an example, actually looking at 3000+ users per server. Any input, tips, etc. would be greatly appreciated. Thanks,Richard Hutson To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message