From owner-freebsd-www Thu Jul 24 21:39:57 1997 Return-Path: Received: (from root@localhost) by hub.freebsd.org (8.8.5/8.8.5) id VAA13738 for www-outgoing; Thu, 24 Jul 1997 21:39:57 -0700 (PDT) Received: from asis.com (root@red.asis.com [206.99.112.12]) by hub.freebsd.org (8.8.5/8.8.5) with ESMTP id VAA13733 for ; Thu, 24 Jul 1997 21:39:55 -0700 (PDT) Received: from megan (ppp-7.asis.com [206.99.112.23]) by asis.com (8.8.5/8.7.3) with SMTP id VAA27974 for ; Thu, 24 Jul 1997 21:39:39 -0700 (PDT) Message-Id: <3.0.32.19970724213535.008c8310@asis.com> X-Sender: meganm@asis.com X-Mailer: Windows Eudora Pro Version 3.0 (32) Date: Thu, 24 Jul 1997 21:35:37 -0700 To: www@freebsd.org From: Megan McCormack Subject: robots.txt Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Sender: owner-freebsd-www@freebsd.org X-Loop: FreeBSD.org Precedence: bulk Hello, Does it make sense to disallow stats.html in robots.txt so that mirror sites error logs don't overflow with error messages of robots trying to scoop up the links to local pages like ~jkh on the mirrors? Megan ************** Chop wood, carry water, check e-mail ************ Megan McCormack http://www.asis.com/~meganm