Date: Thu, 2 May 2002 01:21:54 -0400 (EDT) From: Mohsin Rahman <mtech@buffnet.net> To: Bill Moran <wmoran@potentialtech.com> Cc: RichardH <rh@storm2k.com>, questions@FreeBSD.ORG Subject: Re: Parsing Log Files Message-ID: <Pine.BSF.4.05.10205020102100.12721-100000@buffnet5.buffnet.net> In-Reply-To: <3CD0B07C.6030207@potentialtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 1 May 2002, Bill Moran wrote: > RichardH wrote: > > I have looked all in here and on the net for more specific info on > > parsing access logs for multiple user accounts. If I have 300 users on a > > server and want to have a "log" directory in each users home directory > > is there a script or a simple grep that will pull that users info from > > the raw access logs and split/parse the info out to the individual log > > directory in each users account so that it can then be viewed by the > > user with AWStats, Analog, etc. I know this has to be accomplished > > failry easily but cannot find any links or data giving the exact way of > > doing it. Any input would be greatly appreciated. OS is FreeBSD4.5 > > w/Apache Web Server 1.3.24. > > <SNIP> > > > The above post says 300 users as an example, actually looking at 3000+ > > users per server. > > > > Any input, tips, etc. would be greatly appreciated. > > Thanks,Richard Hutson > > Have you tried simply running a grep on the main log at a certain schedule > to extract the records that pertain to a specific host? It would work > around your open files concern. > Apache logs are pretty configurable, I don't remember details, but it should > be possible to ensure that each virtual host puts something in the log > file that is distinct, that can then be grepped out. > If you run the grep processes on a daily basis, you should probably stagger > them. 3000 greps trying to extract data from the same file at the same time > is going to be a HUGE load. Probably a script that simply ran all the required > greps, one after another in succession would work. > > > -- > Bill Moran > Potential Technology > http://www.potentialtech.com > > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-questions" in the body of the message > I do not claim to be an apache expert, the config below works for me. Some more experienced users can probably suggest something better. Please also note that the REQUEST_URI method will match all instances of user1, user10, user110 and so on. Might not be accurate when the users grow to a large number. What you want to do for each user is to look at the SetEnvIf and Request_URI. For example, to filter out user1's log: In httpd.conf: SetEnvIf Request_URI \user1 user1_logs SetEnvIf Request_URI \~user1 user1_logs CustomLog /home/user1/user1-access_log combined env=user1_logs I picked this syntax from the Nimda virus days and adjusted it to log my personal webpage access logs (not a vertual host). If you mean you have 300+ virtual hosts, then add this in each <VirtualHost xxx.xxx.xxx.xxx> ServerAdmin webmaster@buffnet.net DocumentRoot /home/user1 ServerName www.user1.com ErrorLog /home/user1/user1-error_log CustomLog /home/user1/user1-access_log combined </VirtualHost> Hope this helps. Mohsin AbdulRahman MTech@BuffNET.Net To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.05.10205020102100.12721-100000>