From owner-freebsd-questions@FreeBSD.ORG  Mon Jul  3 07:59:11 2006
Return-Path: <owner-freebsd-questions@FreeBSD.ORG>
X-Original-To: questions@freebsd.org
Delivered-To: freebsd-questions@FreeBSD.ORG
Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125])
	by hub.freebsd.org (Postfix) with ESMTP id 9DD8B16A417
	for <questions@freebsd.org>; Mon,  3 Jul 2006 07:59:11 +0000 (UTC)
	(envelope-from on@cs.ait.ac.th)
Received: from mail.cs.ait.ac.th (mail.cs.ait.ac.th [192.41.170.16])
	by mx1.FreeBSD.org (Postfix) with ESMTP id C6B1E43E30
	for <questions@freebsd.org>; Mon,  3 Jul 2006 07:59:10 +0000 (GMT)
	(envelope-from on@cs.ait.ac.th)
Received: from banyan.cs.ait.ac.th (banyan.cs.ait.ac.th [192.41.170.5])
	by mail.cs.ait.ac.th (8.13.1/8.12.11) with ESMTP id k637x3A6096889
	(version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
	Mon, 3 Jul 2006 14:59:03 +0700 (ICT)
Received: (from on@localhost)
	by banyan.cs.ait.ac.th (8.13.3/8.12.11) id k637x1NH095899;
	Mon, 3 Jul 2006 14:59:01 +0700 (ICT)
Date: Mon, 3 Jul 2006 14:59:01 +0700 (ICT)
Message-Id: <200607030759.k637x1NH095899@banyan.cs.ait.ac.th>
From: Olivier Nicole <on@cs.ait.ac.th>
To: hakmi@rogers.com
In-reply-to: <200606301537.k5UFb7gH077006@mail.cs.ait.ac.th> (hakmi@rogers.com)
References: <200606301537.k5UFb7gH077006@mail.cs.ait.ac.th>
X-Virus-Scanned: on CSIM by amavisd-milter (http://www.amavis.org/)
Cc: questions@freebsd.org, xfb52@dial.pipex.com
Subject: Re: Slow server
X-BeenThere: freebsd-questions@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: User questions <freebsd-questions.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
	<mailto:freebsd-questions-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-questions>
List-Post: <mailto:freebsd-questions@freebsd.org>
List-Help: <mailto:freebsd-questions-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-questions>, 
	<mailto:freebsd-questions-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 03 Jul 2006 07:59:11 -0000

> > >2) as there are many connections comming from search engines siders
> > >   (90% of all the established connections), I'd like to limit the
> > >   ressources that spiders are using. One way would be through IPFW,
> > >   but are there better ways? Is there a way to limit/prioritize in
> > >   Apache (not that I know any).
> Lookup mod_security rules for Apache and mod_dosevasive. mod_evasive
> will help prevent the spiders from opening many pages at one time

Thanks for the idea. I looked at both. mod_evasive would be the one,
but it keeps traffic information on a per web site basis. The problem
is that I have hundred of web sites and the spider tries to access one
page at a time, but one page of each web site...

OK I have to dig that further.

Thanks,

Olivier