Date: Wed, 19 May 1999 01:58:38 +1200 From: Andrew McNaughton <andrew@squiz.co.nz> To: Graeme Tait <graeme@echidna.com> Cc: Carol Deihl <carol@tinker.com>, ndear@areti.net, freebsd-isp@FreeBSD.ORG Subject: Re: Web Statistics break up program. Message-ID: <199905181358.BAA09220@aniwa.sky> In-Reply-To: Your message of "Tue, 18 May 1999 08:09:46 MST." <374182BA.6ED@echidna.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Graeme Tait wrote: > Carol Deihl wrote: > > > > FreeBSD already has the parts to do a nice job, just requires > > a little scripting to get it going. Here's what we do. > > > > I wrote a little perl script called /etc/rotatelogs that > > rotates the logs for our virtual domains just after midnight. > > It uses the "newsyslog" program to do this, which takes care > > of removing aged log files. (The script is below.) I leave the > > just-rotated access log un-gzipped, since the stats program will > > be looking at it soon. I use a separate newsyslog configuration > > file for each virtual domain, in case I want to do something > > special for a particular domain. Below I've included an > > example config file. The scripts assume that these config files > > are in a directory /etc/newsyslog.confs, named with the > > domain name. > > > <snip> > > > Thanks for the detailed answer - this is most helpful. I was about to > try to figure out how to do this sort of thing in a more automated > way. > > A few questions: > > I'm not sure what web server you use, but it looks like (by default) > you are having newsyslog issue it a SIGHUP after rotating each log. > With Apache, as I understand it, that kills off all the Apache > children abruptly, interrupting transfers in progress, and requiring > it to restart from zero children. It also zeros out the server > statistics. Interrupting transfers in progress is not a great idea > for our server, as it's taking orders worldwide, and even though it's > less busy in the wee hours (US time), it's not idle. Also, some files > being downloaded are large, and we wouldn't want to interrupt those > transfers unnecessarily. If you look amongst the binaries that come with Apache you'll find one called rotatelogs. Check out it's man page, and the stuff about it under the CustomLog directive in the Apache web pages. It allows you to rotate your log files without the need to restart your server. Alternatively You could substitute any other program into the CustomLog directive to handle the splitting and perhaps also compression of your logs. rotatelogs splits the log file each time the time in seconds from the epoch goes past a multiple of a number supplied as an argument. Unfortunately this means that if you split once per day, the split happens at mignight GMT not midnight local time. You'll need to either modify rotatelogs or cater to this in your log analysis script. Andrew McNaughton -- ----------- Andrew McNaughton andrew@squiz.co.nz http://www.newsroom.co.nz/ To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-isp" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199905181358.BAA09220>