Date: Thu, 17 Apr 2003 17:56:53 -0700 From: Kris Kennaway <kris@obsecurity.org> To: Bill Moran <wmoran@potentialtech.com> Cc: freebsd-questions@freebsd.org Subject: Re: Free BSD in Googles's Index Message-ID: <20030418005653.GA93673@rot13.obsecurity.org> In-Reply-To: <3E9F36E9.3080506@potentialtech.com> References: <000701c30532$0ea7fda0$6510200a@corp.google.com> <3E9F36E9.3080506@potentialtech.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--TB36FDmn/VVEgNH/ Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Thu, Apr 17, 2003 at 07:21:13PM -0400, Bill Moran wrote: > Ashley Gillard wrote: > >Dear Free BSD Representative: > > > >Our crawl team has discovered that Google is currently blocked from > >crawling your site by the robot.txt that is on your site. We would like > >to index your site at http://bento.freebsd.org to provide you better > >coverage and are interested in working with you to do this.=20 > > > >Google's mission is to deliver the best search experience on the > >Internet by making the world's information universally accessible and > >useful. To find out more about Google.com visit: > >http://www.google.com/webmasters/. > > > >I look forward to hearing from you. >=20 > First off, bento is a ports build cluster and it's probably impractical > to have google index it, as the information probably changes before it > can even reach your index. Correct, it's not appropriate for google to index bento. It contains tens of gigabytes of log files. > If you still think it would be practical to alter the robots.txt file, > it's probably best to contact Satoshi Asami (asami@freebsd.org) as he > appears to maintain that site. Nope, that's mine. Kris --TB36FDmn/VVEgNH/ Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (FreeBSD) iD8DBQE+n01VWry0BWjoQKURAtwcAKDufiEkH4uI8bjDaX0ru4oFv/JT6ACg1y+z u3fzPBQLqt5ZT3glpGXAjR0= =8OJJ -----END PGP SIGNATURE----- --TB36FDmn/VVEgNH/--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030418005653.GA93673>