Date: Thu, 17 Feb 2022 13:05:35 +1100 From: Aristedes Maniatis <ari@ish.com.au> To: freebsd-ports@FreeBSD.org Subject: Re: pkgs contain non URL safe characters Message-ID: <e0aea756-b7c3-b9bf-9dc7-b34c91399b31@ish.com.au> In-Reply-To: <e4f8bec4-e2c7-154c-0e49-109718782725@ish.com.au> References: <e4f8bec4-e2c7-154c-0e49-109718782725@ish.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------G695vUoiivf6HvwjCgbcLrYi Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit Just to check this behaviour, I used tcpdump to see what the request looked like from pkg-fetch. 123.ish.com.au.15580 > pkg0.twn.freebsd.org.http: Flags [P.], cksum 0x80e0 (incorrect -> 0xfc82), seq 1:184, ack 1, win 1027, options [nop,nop,TS val 975600196 ecr 3136747760], length 183: HTTP, length: 183 GET /FreeBSD:13:amd64/quarterly/All/openjdk11-11.0.13+8.1.pkg HTTP/1.1 Host: pkgmir.geo.freebsd.org Accept: */* User-Agent: pkg/1.17.5 Range: bytes=6733824- Connection: close You can see in there that the + is not URL encoded. Is it expected that pkg uses URL standards for its repository? If not, any advice on how to host a repository on a commercial service like AWS cloudfront? Should we rewrite all our files with + symbols to spaces? Should pkg names only contain URL safe characters? Or should pkg-fetch be fixed to encode URLs? I took a quick look at the source for pkg.c and where it calls fetchXGet but I can't understand where any URL encoding might happen. Ari On 14/2/2022 11:18am, Aristedes Maniatis wrote: > Some packages contain "+" symbol which is a way of encoding spaces in > a URL. This means that I'm having trouble hosting our pkg repository > behind cloudfront/S3. > > I wasn't sure where to post this issue, so I put more details here: > https://github.com/freebsd/poudriere/issues/976 > > > Is there a workaround for this issue? Could pkg-fetch escape such > characters when interacting with a http repository? > > > Cheers > > Ari > > --------------G695vUoiivf6HvwjCgbcLrYi Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> </head> <body> <p>Just to check this behaviour, I used tcpdump to see what the request looked like from pkg-fetch.</p> <p><br> </p> <p> 123.ish.com.au.15580 > pkg0.twn.freebsd.org.http: Flags [P.], cksum 0x80e0 (incorrect -> 0xfc82), seq 1:184, ack 1, win 1027, options [nop,nop,TS val 975600196 ecr 3136747760], length 183: HTTP, length: 183<br> GET /FreeBSD:13:amd64/quarterly/All/openjdk11-11.0.13+8.1.pkg HTTP/1.1<br> Host: pkgmir.geo.freebsd.org<br> Accept: */*<br> User-Agent: pkg/1.17.5<br> Range: bytes=6733824-<br> Connection: close<br> </p> <p><br> </p> <p>You can see in there that the + is not URL encoded. Is it expected that pkg uses URL standards for its repository? If not, any advice on how to host a repository on a commercial service like AWS cloudfront?<br> </p> <p>Should we rewrite all our files with + symbols to spaces? Should pkg names only contain URL safe characters? Or should pkg-fetch be fixed to encode URLs?<br> </p> <p><br> </p> <p>I took a quick look at the source for pkg.c and where it calls <span class="pl-c1">fetchXGet but I can't understand where any URL encoding might happen.<br> </span></p> <p><br> </p> <p>Ari</p> <p><br> </p> <div class="moz-cite-prefix">On 14/2/2022 11:18am, Aristedes Maniatis wrote:<br> </div> <blockquote type="cite" cite="mid:e4f8bec4-e2c7-154c-0e49-109718782725@ish.com.au">Some packages contain "+" symbol which is a way of encoding spaces in a URL. This means that I'm having trouble hosting our pkg repository behind cloudfront/S3. <br> <br> I wasn't sure where to post this issue, so I put more details here: <a class="moz-txt-link-freetext" href="https://github.com/freebsd/poudriere/issues/976">https://github.com/freebsd/poudriere/issues/976</a>; <br> <br> <br> Is there a workaround for this issue? Could pkg-fetch escape such characters when interacting with a http repository? <br> <br> <br> Cheers <br> <br> Ari <br> <br> <br> </blockquote> </body> </html> --------------G695vUoiivf6HvwjCgbcLrYi--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?e0aea756-b7c3-b9bf-9dc7-b34c91399b31>