From owner-svn-doc-head@FreeBSD.ORG  Fri Oct  4 18:55:55 2013
Return-Path: <owner-svn-doc-head@FreeBSD.ORG>
Delivered-To: svn-doc-head@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id E2925FE;
 Fri,  4 Oct 2013 18:55:55 +0000 (UTC) (envelope-from gjb@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org
 [IPv6:2001:1900:2254:2068::e6a:0])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (No client certificate requested)
 by mx1.freebsd.org (Postfix) with ESMTPS id CFFEF2FCE;
 Fri,  4 Oct 2013 18:55:55 +0000 (UTC)
Received: from svn.freebsd.org ([127.0.1.70])
 by svn.freebsd.org (8.14.7/8.14.7) with ESMTP id r94ItteJ082114;
 Fri, 4 Oct 2013 18:55:55 GMT (envelope-from gjb@svn.freebsd.org)
Received: (from gjb@localhost)
 by svn.freebsd.org (8.14.7/8.14.5/Submit) id r94IttLG082111;
 Fri, 4 Oct 2013 18:55:55 GMT (envelope-from gjb@svn.freebsd.org)
Message-Id: <201310041855.r94IttLG082111@svn.freebsd.org>
From: Glen Barber <gjb@FreeBSD.org>
Date: Fri, 4 Oct 2013 18:55:55 +0000 (UTC)
To: doc-committers@freebsd.org, svn-doc-all@freebsd.org,
 svn-doc-head@freebsd.org
Subject: svn commit: r42830 - head/en_US.ISO8859-1/htdocs
X-SVN-Group: doc-head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-doc-head@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: SVN commit messages for the doc tree for head
 <svn-doc-head.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-doc-head>,
 <mailto:svn-doc-head-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-doc-head>
List-Post: <mailto:svn-doc-head@freebsd.org>
List-Help: <mailto:svn-doc-head-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-doc-head>,
 <mailto:svn-doc-head-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 04 Oct 2013 18:55:56 -0000

Author: gjb
Date: Fri Oct  4 18:55:55 2013
New Revision: 42830
URL: http://svnweb.freebsd.org/changeset/doc/42830

Log:
  Remove a stray blank line from robots.txt.
  Add a commented Crawl-delay of 10 seconds, which might be useful
  after the commit that is to follow.

Modified:
  head/en_US.ISO8859-1/htdocs/robots.txt

Modified: head/en_US.ISO8859-1/htdocs/robots.txt
==============================================================================
--- head/en_US.ISO8859-1/htdocs/robots.txt	Fri Oct  4 18:43:31 2013	(r42829)
+++ head/en_US.ISO8859-1/htdocs/robots.txt	Fri Oct  4 18:55:55 2013	(r42830)
@@ -1,7 +1,7 @@
 # $FreeBSD$
 
 User-agent: *
-
+#Crawl-delay: 10
 Disallow: /cgi/
 Disallow: /statistic
 Disallow: /sites/downloads.sourceforge.net/