From owner-freebsd-questions@FreeBSD.ORG Thu Apr 17 17:56:55 2003 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4744137B401 for ; Thu, 17 Apr 2003 17:56:55 -0700 (PDT) Received: from obsecurity.dyndns.org (adsl-63-207-60-150.dsl.lsan03.pacbell.net [63.207.60.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5946A43FD7 for ; Thu, 17 Apr 2003 17:56:54 -0700 (PDT) (envelope-from kris@obsecurity.org) Received: from rot13.obsecurity.org (rot13.obsecurity.org [10.0.0.5]) by obsecurity.dyndns.org (Postfix) with ESMTP id 176AB66CFA; Thu, 17 Apr 2003 17:56:54 -0700 (PDT) Received: by rot13.obsecurity.org (Postfix, from userid 1000) id EACB41187; Thu, 17 Apr 2003 17:56:53 -0700 (PDT) Date: Thu, 17 Apr 2003 17:56:53 -0700 From: Kris Kennaway To: Bill Moran Message-ID: <20030418005653.GA93673@rot13.obsecurity.org> References: <000701c30532$0ea7fda0$6510200a@corp.google.com> <3E9F36E9.3080506@potentialtech.com> Mime-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="TB36FDmn/VVEgNH/" Content-Disposition: inline In-Reply-To: <3E9F36E9.3080506@potentialtech.com> User-Agent: Mutt/1.4i cc: Ashley Gillard cc: freebsd-questions@freebsd.org Subject: Re: Free BSD in Googles's Index X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Apr 2003 00:56:55 -0000 --TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 17, 2003 at 07:21:13PM -0400, Bill Moran wrote: > Ashley Gillard wrote: > >Dear Free BSD Representative: > > > >Our crawl team has discovered that Google is currently blocked from > >crawling your site by the robot.txt that is on your site. We would like > >to index your site at http://bento.freebsd.org to provide you better > >coverage and are interested in working with you to do this.=20 > > > >Google's mission is to deliver the best search experience on the > >Internet by making the world's information universally accessible and > >useful. To find out more about Google.com visit: > >http://www.google.com/webmasters/. > > > >I look forward to hearing from you. >=20 > First off, bento is a ports build cluster and it's probably impractical > to have google index it, as the information probably changes before it > can even reach your index. Correct, it's not appropriate for google to index bento. It contains tens of gigabytes of log files. > If you still think it would be practical to alter the robots.txt file, > it's probably best to contact Satoshi Asami (asami@freebsd.org) as he > appears to maintain that site. Nope, that's mine. Kris --TB36FDmn/VVEgNH/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+n01VWry0BWjoQKURAtwcAKDufiEkH4uI8bjDaX0ru4oFv/JT6ACg1y+z u3fzPBQLqt5ZT3glpGXAjR0= =8OJJ -----END PGP SIGNATURE----- --TB36FDmn/VVEgNH/--