From owner-freebsd-questions  Thu May  2  8: 2:12 2002
Delivered-To: freebsd-questions@freebsd.org
Received: from spin.web.net (spin.web.net [192.139.37.16])
	by hub.freebsd.org (Postfix) with ESMTP id 83DA637B41C
	for <questions@FreeBSD.ORG>; Thu,  2 May 2002 08:02:04 -0700 (PDT)
Received: by spin.web.net (Postfix, from userid 1000)
	id 83E0912E631; Thu,  2 May 2002 11:02:03 -0400 (EDT)
Date: Thu, 2 May 2002 11:02:03 -0400
From: Rob Ellis <rob@web.ca>
To: RichardH <rh@storm2k.com>
Cc: questions@FreeBSD.ORG
Subject: Re: Parsing Log Files
Message-ID: <20020502150203.GA84982@web.ca>
References: <5.1.0.14.0.20020501192418.02cea050@pop.wsonline.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5.1.0.14.0.20020501192418.02cea050@pop.wsonline.net>
User-Agent: Mutt/1.3.28i
Sender: owner-freebsd-questions@FreeBSD.ORG
Precedence: bulk
List-ID: <freebsd-questions.FreeBSD.ORG>
List-Archive: <http://docs.freebsd.org/mail/> (Web Archive)
List-Help: <mailto:majordomo@FreeBSD.ORG?subject=help> (List Instructions)
List-Subscribe: <mailto:majordomo@FreeBSD.ORG?subject=subscribe%20freebsd-questions>
List-Unsubscribe: <mailto:majordomo@FreeBSD.ORG?subject=unsubscribe%20freebsd-questions>
X-Loop: FreeBSD.ORG

On Wed, May 01, 2002 at 07:29:29PM -0600, RichardH wrote:
> By parsing out the files with a script, it reduces overall server load AND 
> permits the use of rewrite rules, that allow you to use a virtmap.txt type 
> of setup for hosting entries (in which case the transferlog entry does not 
> work at all).

Assuming the domain name is the first thing on each log line, 
you could do something like

   #! /usr/bin/perl -w
   use FileCache; # opens/closes file descriptors as required
   no strict "refs"; # FileCache generates "strict refs" warnings
   $log = "/usr/local/apache/logs/access_log";
   $outdir = "/usr/local/var/weblogs";
   open(LOG, $log) || die $!;
   while (<LOG>) {
	 if (/^([\w\.-]+)\s+/) {
		$domain = $1;
		$outfile = "$outdir/$domain/access_log";
		die $! unless (cacheout $outfile);
		print $outfile $_;
	 }
	 # do something here with junk lines
   } 
   close(LOG);
   1;

?

Not tested, but we do something similar.

- Rob

--
Rob Ellis <rob@web.ca>
System Administrator, Web Networks

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message