Date: Thu, 5 Sep 2024 14:05:42 GMT From: Pietro Cerutti <gahr@FreeBSD.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org Subject: git: ab7a79806e31 - stable/14 - libfetch: don't include fragments in HTTP requests Message-ID: <202409051405.485E5g0F099348@gitrepo.freebsd.org>
next in thread | raw e-mail | index | archive | help
The branch stable/14 has been updated by gahr: URL: https://cgit.FreeBSD.org/src/commit/?id=ab7a79806e3103accb1ba2d89ba51e977704a2e3 commit ab7a79806e3103accb1ba2d89ba51e977704a2e3 Author: Pietro Cerutti <gahr@FreeBSD.org> AuthorDate: 2024-08-21 12:35:27 +0000 Commit: Pietro Cerutti <gahr@FreeBSD.org> CommitDate: 2024-09-05 14:05:15 +0000 libfetch: don't include fragments in HTTP requests Fragments are reserved for client-side processing, see https://www.rfc-editor.org/rfc/rfc9110.html#section-7.1 Also, some servers don't like to receive HTTP requests with fragments. ``` $ fetch 'https://dropbox.com/a/b' fetch: https://dropbox.com/a/b: Not Found $ fetch 'https://dropbox.com/a/b#' fetch: https://dropbox.com/a/b#: Bad Request ``` This is a real-world scenario, where some download link from dropbox (eventually) redirects to an URL with a fragment: ``` $ fetch -v 'https://www.dropbox.com/sh/<some>/<thing>?dl=1' 2>&1 | grep requesting requesting https://www.dropbox.com/sh/<some>/<thing>?dl=1 requesting https://www.dropbox.com/scl/fo/<foo>/<bar>?rlkey=<baz>&dl=1 requesting https://<boo>.dl.dropboxusercontent.com/zip_download_get/<some-long-strig># ``` See how the last redirect ends with a `#`. Currently, libfetch includes the ending fragment and makes it impossible to download the file. Differential Revision: https://reviews.freebsd.org/D46318 MFC after: 2 weeks (cherry picked from commit 1af7d5f389536a2f391153513d95d92ffdf360e4) --- lib/libfetch/fetch.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/lib/libfetch/fetch.c b/lib/libfetch/fetch.c index 8cb3536a46c5..6e14a62ca596 100644 --- a/lib/libfetch/fetch.c +++ b/lib/libfetch/fetch.c @@ -448,7 +448,10 @@ nohost: goto ouch; } u->doc = doc; - while (*p != '\0') { + /* fragments are reserved for client-side processing, see + * https://www.rfc-editor.org/rfc/rfc9110.html#section-7.1 + */ + while (*p != '\0' && *p != '#') { if (!isspace((unsigned char)*p)) { *doc++ = *p++; } else {
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?202409051405.485E5g0F099348>