Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 7 Aug 2023 13:52:01 GMT
From:      Wolfram Schneider <wosch@FreeBSD.org>
To:        doc-committers@FreeBSD.org, dev-commits-doc-all@FreeBSD.org
Subject:   git: c048664e41 - main - block GPTBot (OpenAI) from accessing cgi scripts
Message-ID:  <202308071352.377Dq1jr074083@gitrepo.freebsd.org>

next in thread | raw e-mail | index | archive | help
The branch main has been updated by wosch:

URL: https://cgit.FreeBSD.org/doc/commit/?id=c048664e41ee40b03123d4370f89c7be217f9d39

commit c048664e41ee40b03123d4370f89c7be217f9d39
Author:     Wolfram Schneider <wosch@FreeBSD.org>
AuthorDate: 2023-08-07 13:51:53 +0000
Commit:     Wolfram Schneider <wosch@FreeBSD.org>
CommitDate: 2023-08-07 13:51:53 +0000

    block GPTBot (OpenAI) from accessing cgi scripts
---
 website/themes/beastie/layouts/robots.txt | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/website/themes/beastie/layouts/robots.txt b/website/themes/beastie/layouts/robots.txt
index b94bcd43cf..ac0950c19f 100644
--- a/website/themes/beastie/layouts/robots.txt
+++ b/website/themes/beastie/layouts/robots.txt
@@ -17,3 +17,7 @@ Disallow: /cgi/query-pr.cgi
 Disallow: /cgi/search.cgi
 Disallow: /cgi/url.cgi
 Disallow: /statistic
+
+User-agent: GPTBot
+Disallow: /cgi
+



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202308071352.377Dq1jr074083>