Date: Fri, 13 Aug 2021 22:57:40 GMT From: "Danilo G. Baio" <dbaio@FreeBSD.org> To: doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org Subject: git: 6af7769111 - main - documentation/robots.txt: Disallow Past Releases Documentation Message-ID: <202108132257.17DMveYC007559@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch main has been updated by dbaio: URL: https://cgit.FreeBSD.org/doc/commit/?id=6af77691110d19019b389fad3a0700286fdf0ef1 commit 6af77691110d19019b389fad3a0700286fdf0ef1 Author: Danilo G. Baio <dbaio@FreeBSD.org> AuthorDate: 2021-08-08 14:12:03 +0000 Commit: Danilo G. Baio <dbaio@FreeBSD.org> CommitDate: 2021-08-13 22:51:16 +0000 documentation/robots.txt: Disallow Past Releases Documentation It was noticed that Google/DDG sometimes are sending users to the past releases documentation. To avoid misinformation, disallow this archive from the search engines. This robots.txt only covers docs.FreeBSD.org and not www.FreeBSD.org, so also remove unnecessary items. Differential Revision: https://reviews.freebsd.org/D31459 --- documentation/themes/beastie/layouts/robots.txt | 19 +------------------ 1 file changed, 1 insertion(+), 18 deletions(-) diff --git a/documentation/themes/beastie/layouts/robots.txt b/documentation/themes/beastie/layouts/robots.txt index b94bcd43cf..f59c629169 100644 --- a/documentation/themes/beastie/layouts/robots.txt +++ b/documentation/themes/beastie/layouts/robots.txt @@ -1,19 +1,2 @@ -# $FreeBSD$ - User-agent: * -#Crawl-delay: 10 -Disallow: /cgi/confirm-code.cgi -Disallow: /cgi/cvsweb.cgi -Disallow: /cgi/dosendpr.cgi -Disallow: /cgi/getmsg.cgi -Disallow: /cgi/mailindex.cgi -Disallow: /cgi/man.cgi?apropos=2& -Disallow: /cgi/mid.cgi -Disallow: /cgi/mirror.cgi -Disallow: /cgi/missing_handler.cgi -Disallow: /cgi/ports.cgi -Disallow: /cgi/query-pr-summary.cgi -Disallow: /cgi/query-pr.cgi -Disallow: /cgi/search.cgi -Disallow: /cgi/url.cgi -Disallow: /statistic +Disallow: /doc/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202108132257.17DMveYC007559>