From owner-freebsd-questions Thu Feb 21 11: 2:44 2002 Delivered-To: freebsd-questions@freebsd.org Received: from pc1-dale5-0-cust136.not.cable.ntl.com (pc1-dale5-0-cust136.not.cable.ntl.com [80.1.76.136]) by hub.freebsd.org (Postfix) with SMTP id ACB9337B402 for ; Thu, 21 Feb 2002 11:02:33 -0800 (PST) Received: (qmail 22776 invoked from network); 21 Feb 2002 19:02:11 -0000 Received: from localhost (HELO matt.thebigchoice.com) (127.0.0.1) by localhost with SMTP; 21 Feb 2002 19:02:11 -0000 Date: Thu, 21 Feb 2002 19:02:05 +0000 From: Matt H To: "Scott Aitken" Cc: freebsd-questions@FreeBSD.ORG Subject: Re: Sorting Apache logs by Date and Time Message-Id: <20020221190205.05c6ee5d.matt@proweb.co.uk> In-Reply-To: <20020222050939.A43475@gandalf.scott.sh> References: <20020222050939.A43475@gandalf.scott.sh> X-Mailer: Sylpheed version 0.7.1 (GTK+ 1.2.10; i386--freebsd4.4) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: owner-freebsd-questions@FreeBSD.ORG Precedence: bulk List-ID: List-Archive: (Web Archive) List-Help: (List Instructions) List-Subscribe: List-Unsubscribe: X-Loop: FreeBSD.ORG On Fri, 22 Feb 2002 05:09:40 +1100 "Scott Aitken" wrote: > Hi All, > is there a quick way to sort apache logs by the date and time field if > the format is the following: > > 146.101.149.65 - - [22/Feb/2002:04:18:54 +1100] "GET /images/test.png > HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" > "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" > > This is a sample line. > > I have found difficulty in using the sort utility because of the textual > month names, as well as the fact that the separators are both slashes > and colons. here's a whacky way to do it start with a file called months sorted alphabetically : 587 ~ >cat months /Apr/04/ /Aug/08/ /Dec/12/ /Feb/02/ /Jan/01/ /Jul/07/ /Jun/06/ /Mar/03/ /May/05/ /Nov/11/ /Oct/10/ /Sep/09/ # then use join 588 ~ >join -t "/" -1 2 -2 2 -o 1.1 2.3 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 1.19 1.20 log months | sed 's;/*$;;' | sort +4 unfortunately the 1.3 - 1.20 is to try and catch all the directory names with a / in them!! if it's not enough add some more!! I leave the awk file that would count them and output the maximum number of / fields for you to do every blank field leaves a / so the sed on the end chops off the trailing ones the final sort will sort by the datetime (ignoring the +1100 TMZ but i leave that as an exercise!) here's my output592 ~ >cat hlogs 146.101.149.65 - - [22/Aug/2002:01:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 146.101.149.65 - - [22/Aug/2002:05:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 146.101.149.65 - - [22/Dec/2002:03:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 146.101.149.65 - - [22/Mar/2002:04:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 597 ~ >join -t "/" -1 2 -2 2 -o 1.1 2.3 1.3 1.4 1.5 1.6 1.7 1.8 1.9 1.10 1.11 1.12 1.13 1.14 1.15 1.16 1.17 1.18 1.19 1.20 hlogs months | sed 's;/*$;;' | sort +4 146.101.149.65 - - [22/03/2002:04:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 146.101.149.65 - - [22/08/2002:01:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 146.101.149.65 - - [22/08/2002:05:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 146.101.149.65 - - [22/12/2002:03:18:54 +1100] "GET /images/test.png HTTP/1.0" 200 381 "http://www-redirect.scott.sh:8080/cgi-bin/fred.pl" "Mozilla/4.0 (compatible; MSIE 5.5; Windows 98)" 598 ~ > To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message