Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 4 Sep 2002 15:45:15 +0100
From:      Daniel Bye <dan@slightlystrange.org>
To:        questions@freebsd.org
Subject:   Re: Macros in ipfw rules
Message-ID:  <20020904144515.GC91519@catflap.home.slightlystrange.org>
In-Reply-To: <20020904133511.GD20573@pcwin002.win.tue.nl>
References:  <20020903142632.GA71601@catflap.home.slightlystrange.org> <443cspq4ll.fsf@be-well.ilk.org> <20020904133053.GA91519@catflap.home.slightlystrange.org> <20020904133511.GD20573@pcwin002.win.tue.nl>

next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, Sep 04, 2002 at 03:35:11PM +0200, Stijn Hoop wrote:
> On Wed, Sep 04, 2002 at 02:30:53PM +0100, Daniel Bye wrote:
> > However, I wanted to prevent googlebot.com et al from 
> > cataloguing the site, while allowing pretty much unrestricted access to
> > everyone else.
> 
> If that's all you want:
> 
> http://www.robotstxt.org/wc/robots.html
> 
> No need for firewall rules for that kind of thing.

Aha!  Exactly what I needed!  Thanks a lot for that, Stijn.

Dan

-- 
Daniel Bye

PGP Key: ftp://ftp.slightlystrange.org/pgpkey/dan.asc
PGP Key fingerprint: 3D73 AF47 D448 C5CA 88B4 0DCF 849C 1C33 3C48 2CDC
                                                                     _
                                              ASCII ribbon campaign ( )
                                         - against HTML, vCards and  X
                                - proprietary attachments in e-mail / \

To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-questions" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20020904144515.GC91519>