From owner-freebsd-questions@FreeBSD.ORG Mon Oct 1 21:53:26 2007 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EAD1916A417 for ; Mon, 1 Oct 2007 21:53:26 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from pi.codefab.com (pi.codefab.com [199.103.21.227]) by mx1.freebsd.org (Postfix) with ESMTP id D0DBD13C455 for ; Mon, 1 Oct 2007 21:53:26 +0000 (UTC) (envelope-from cswiger@mac.com) Received: from localhost (localhost [127.0.0.1]) by pi.codefab.com (Postfix) with ESMTP id D3E6E5DB5; Mon, 1 Oct 2007 17:53:25 -0400 (EDT) X-Virus-Scanned: amavisd-new at codefab.com Received: from pi.codefab.com ([127.0.0.1]) by localhost (pi.codefab.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id y0APvh2Raj1r; Mon, 1 Oct 2007 17:53:23 -0400 (EDT) Received: from [199.103.21.238] (pan.codefab.com [199.103.21.238]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by pi.codefab.com (Postfix) with ESMTP id CF2B65C42; Mon, 1 Oct 2007 17:53:23 -0400 (EDT) In-Reply-To: <20071001231334.71ddc9ac@attila> References: <20071001231334.71ddc9ac@attila> Mime-Version: 1.0 (Apple Message framework v752.2) Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed Message-Id: <3EAD7F56-D4D4-4D83-B501-3DEF3C7039AE@mac.com> Content-Transfer-Encoding: 7bit From: Chuck Swiger Date: Mon, 1 Oct 2007 17:53:18 -0400 To: "Bahman M." X-Mailer: Apple Mail (2.752.2) Cc: freebsd-questions@freebsd.org Subject: Re: Is Apache rotatelogs efficienct for real world use? X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Oct 2007 21:53:27 -0000 On Oct 1, 2007, at 3:43 PM, Bahman M. wrote: > Is Apache rotatelogs suitable for handling large volumes of > access logs, i.e. around 50K requests per hour at _peak_ time which is > 1.2M requests per day. According to Apache website > (http://httpd.apache.org/docs/1.3/logs.html#rotation) this becomes > about 120MB in size per day. > > Does somebody have similar experience with rotatelogs? Absolutely-- rotatelogs did just fine with one of the sites I used to admin which was getting between 1 and 2.5 million hits a day, depending on the day of the week it was. Storing the logfiles and doing analysis on them afterwards started getting pretty challenging, however: even doing reverse DNS lookups took a fair amount of work, and once you started piling up a year or so's worth, doing things like analog or webalyzer or Unison started taking close to 24-hours to finish running and produce a report. -- -Chuck