Date: Thu, 2 May 2002 09:24:35 -0700 From: Nathan Kinkade <nkinkade@dsl-only.com> To: questions@FreeBSD.ORG Subject: Re: Parsing Log Files Message-ID: <20020502092435.4e263f34.nkinkade@dsl-only.com> In-Reply-To: <20020502150203.GA84982@web.ca> References: <5.1.0.14.0.20020501192418.02cea050@pop.wsonline.net> <20020502150203.GA84982@web.ca>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 2 May 2002 11:02:03 -0400
Rob Ellis <rob@web.ca> wrote:
> On Wed, May 01, 2002 at 07:29:29PM -0600, RichardH wrote:
> > By parsing out the files with a script, it reduces overall server
> > load AND permits the use of rewrite rules, that allow you to use a
> > virtmap.txt type of setup for hosting entries (in which case the
> > transferlog entry does not work at all).
>
> Assuming the domain name is the first thing on each log line,
> you could do something like
>
> #! /usr/bin/perl -w
> use FileCache; # opens/closes file descriptors as required
> no strict "refs"; # FileCache generates "strict refs" warnings
> $log = "/usr/local/apache/logs/access_log";
> $outdir = "/usr/local/var/weblogs";
> open(LOG, $log) || die $!;
> while (<LOG>) {
> if (/^([\w\.-]+)\s+/) {
> $domain = $1;
> $outfile = "$outdir/$domain/access_log";
> die $! unless (cacheout $outfile);
> print $outfile $_;
> }
> # do something here with junk lines
> }
> close(LOG);
> 1;
Here are some snips from a small script that I put together to parse the
apache log (/var/log/httpd-access.log) to find suspect log entries
containing lame attempts to exploit IIS vulnerabilities. If found, it
will try to send an email to "abuse" at whatever domain the user was at.
It doesn't write anything to an output file, but it does selectively
choose entries from the current date only. You could possibly modify
this to append each days activities to each users log file. Again, the
below doesn't necessarily speak to your particular problem, but maybe
some tidbits of this could be a start, along with the post from Rob
Ellis.
#!/usr/bin/perl -w
use strict;
use Mail::Sendmail;
my ($line, $host, $rcpt, $dstamp, $body); # some scalars
my @date; # an array
my (%mail, %offenders); # some hashes
@date = split(" ", `date`); # get current date into
an array$dstamp = "$date[2]/$date[1]/$date[5]"; # rearrange to
match date in apache log file
open (FILE, "/var/log/httpd-access.log"); # open log file for
reading
while ($line = <FILE>) {
# find log entries from today that also contain mischevious keywords
if ( (grep(/.*\[$dstamp:/, $line)) &&
(grep(/scripts|winnt|cmd\.exe|root\.exe|system32/, $line)) ) {
$line =~ /^(\S+).*\[(.+)\].*GET\s(\S+)/; # parse interesting line
$1=host $2=date/time $3=GET command push @{$offenders{$1}},"$2
$3\n"; # put values into a hash for later processing }
}
foreach $host (keys(%offenders)) {
if ($host !~ /\.\d+$/) { # only act if $host is an actual host
name to which we can construct an email $host =~ /^\S+\.(.*)$/; #
get domain portion of $host $rcpt = $1; # assign
$rcpt to value of previous regex $body = ( # create
the email body "Email Body"
);
%mail = ( # create some email headers
'Date' => Mail::Sendmail::time_to_date(),
'To' => "abuse\@$rcpt",
'From' => 'somebody@somewhere.org',
'Subject' => 'Notification of malicious user or system',
'Body' => "$body"
);
sendmail(%mail); # send the mail
}
}
close (FILE); # close the file log file
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020502092435.4e263f34.nkinkade>