From nobody Wed Jan 14 16:35:22 2026 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4drsDJ5CMtz6NcpN for ; Wed, 14 Jan 2026 16:35:28 +0000 (UTC) (envelope-from red_M95@proton.me) Received: from mail-43167.protonmail.ch (mail-43167.protonmail.ch [185.70.43.167]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4drsDJ28WVz3HYp for ; Wed, 14 Jan 2026 16:35:28 +0000 (UTC) (envelope-from red_M95@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1768408525; x=1768667725; bh=Ziza/9YELoutA68Y+OvRPWKVJ7GZJD4wBSQxWWz5M3Y=; h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=bzgtkZ0YJRAuf+1kEt27TnZxZIrQkjNGAN19Ls792/v4CUPZyS4GFa2yaDbglBXC5 yZRtYSiHDF7uX5t0PrV7k2E4vZrmv9LPrqdMFTD93ccJMDrGyBhUi25Izfgmscccat qtIY03AxDDuFhJ6NmH/AZv/KtXHwHSXwA6nhCoGOLQvyLWSg7FRq7I/Rwb+WzwtDod To8BBC89OnhFj+Y23wxvVucSJcrS+zmSc226JznHc5cmAG6cP588ICcYeXS7hZFRG7 lMOirJjBZRUI/Q0MIDrSucceTe0PxeHm/vB35+e+BR546HocgcmoP7UYyABAlcKTVe +1heZdypwbljw== Date: Wed, 14 Jan 2026 16:35:22 +0000 To: Ronald Klop From: ruby R53 Cc: Current FreeBSD Subject: Re: Change fetch addresses for Rust crates, Python packages and whatnot? Message-ID: In-Reply-To: <1541007573.1910.1768406972044@localhost> References: <1503278086.9.1768374682910@localhost> <9Qp2LaDX-QG6aXtsRcjekPDWOjni9Cy4LLmza8jhB2cD15_VfdTY7znjCzIClM1qtEhVuK4P_cL1yKl8AMJZ3m8UnE3EEC67polveeC_Zys=@proton.me> <1541007573.1910.1768406972044@localhost> Feedback-ID: 66100208:user:proton X-Pm-Message-ID: d0f26884f347e99763d575e3361f9fcf6b64e613 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4drsDJ28WVz3HYp On Wednesday, January 14th, 2026 at 1:10 PM, Ronald Klop wrote: >=20 >=20 > Van: ruby R53 > Datum: woensdag, 14 januari 2026 15:41 > Aan: Ronald Klop , Current FreeBSD > Onderwerp: Re: Change fetch addresses for Rust crates, Python packages an= d whatnot? >=20 > > On Wednesday, January 14th, 2026 at 4:11 AM, Ronald Klop wrote: > >=20 > > > > > > > > > Van: ruby R53 > > > Datum: woensdag, 14 januari 2026 03:26 > > > Aan: freebsd-current@freebsd.org > > > Onderwerp: Change fetch addresses for Rust crates, Python packages an= d whatnot? > > > > > > > I've been trying to compile www/chromium for literal days now, all = because whenever `make` runs `fetch`, it goes to an address that's actually= supposed to redirect to another one, but `fetch` instead keeps trying for = a whole minute until it ultimately falls back to distcache.freebsd.org, whi= ch is another whole minute of waiting for the file to download. The cherry = on top is that the file is, most of the time, just hundreds of kilobytes in= size. > > > > > > > > Hell, I even had to manually download part of these individual file= s using that Links web browser=C2=B7=C2=B7=C2=B7 it'd do it a lot faster th= an `fetch`, all because it actually followed the redirection and didn't jus= t keep trying over and over. But of course I wouldn't manually download hun= dreds upon hundreds of archive files, this is the reason why such tool exis= ts. > > > > > > > > So here's an example: currently one of the missing dependencies for= www/chromium is GTK3, which depends on graphics/librsvg2-rust, which then = depends on cargo-c. It's now trying to fetch dunce-1.0.5.crate, which is on= e of the crates required for it, from https://crates.io/api/v1/crates/dunce= /1.0.5/download?dummy=3D/rust/crates/dunce-1.0.5.crate. Running `curl` on t= his link does literally nothing, no files are downloaded. But running it on= any web browser instantly downloads the crate in question, all because tha= t link actually redirects to https://static.crates.io/crates/dunce/dunce-1.= 0.5.crate. Even `curl`ing that new link does give the file I want. Same app= lies for Python packages. > > > > > > > > Now imagine this for all the remaining files=C2=B7=C2=B7=C2=B7 if i= t takes about 2 minutes to download a single file, and there's 398 crate en= tries on cargo-c's `Makefile.crates`, that means it'd take a whole 796 minu= tes to download everything, which is literally more than half a day (~13.6 = hours) :) > > > > =C2=A0 > > > > > > > > > > > > Can you copy-paste the output of the following command in a reply? > > > fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy= =3D/rust/crates/dunce-1.0.5.crate" > > > > > > For me it immediately follows the redirect and downloads the file in = half a second. See the output below. > > > > > > Regards, > > > Ronald. > > > > > > > > > --=C2=A0 > > > > > > time fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?= dummy=3D/rust/crates/dunce-1.0.5.crate" > > > scheme: =C2=A0=C2=A0"https" > > > user: =C2=A0=C2=A0=C2=A0=C2=A0"" > > > password: "" > > > host: =C2=A0=C2=A0=C2=A0=C2=A0"crates.io" > > > port: =C2=A0=C2=A0=C2=A0=C2=A0"0" > > > document: "/api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/d= unce-1.0.5.crate" > > > ---> crates.io:443 > > > resolving server address: crates.io:443 > > > SSL options: 82004850 > > > Peer verification enabled > > > Using OpenSSL default CA cert file and path > > > Verify hostname > > > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > > > Certificate subject: /CN=3Dcrates.io > > > Certificate issuer: /C=3DUS/O=3DAmazon/CN=3DAmazon RSA 2048 M01 > > > requesting https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy= =3D/rust/crates/dunce-1.0.5.crate > > > >>> GET /api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dunc= e-1.0.5.crate HTTP/1.1 > > > >>> Host: crates.io > > > >>> Accept: */* > > > >>> User-Agent: fetch libfetch/2.0 > > > >>> Connection: close > > > >>> > > > <<< HTTP/1.1 302 Moved Temporarily > > > <<< Content-Length: 0 > > > <<< Connection: close > > > content length: [0] > > > <<< Access-Control-Allow-Origin: * > > > <<< Date: Wed, 14 Jan 2026 07:10:06 GMT > > > <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate > > > <<< Nel: {"report_to":"heroku-nel","response_headers":["Via"],"max_ag= e":3600,"success_fraction":0.01,"failure_fraction":0.1} > > > 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.cra= te > > > scheme: =C2=A0=C2=A0"https" > > > user: =C2=A0=C2=A0=C2=A0=C2=A0"" > > > password: "" > > > host: =C2=A0=C2=A0=C2=A0=C2=A0"static.crates.io" > > > port: =C2=A0=C2=A0=C2=A0=C2=A0"0" > > > document: "/crates/dunce/dunce-1.0.5.crate" > > > <<< Report-To: {"group":"heroku-nel","endpoints":[{"url":"https://nel= .heroku.com/reports?s=3DVOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D\u0= 026sid=3Daf571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=3D1768374606"}],"max_= age":3600} > > > <<< Reporting-Endpoints: heroku-nel=3D"https://nel.heroku.com/reports= ?s=3DVOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D&sid=3Daf571f24-03ee-4= 6d1-9f90-ab9030c2c74c&ts=3D1768374606" > > > <<< Server: Heroku > > > <<< Strict-Transport-Security: max-age=3D31536000; includeSubDomains > > > <<< Via: 1.1 heroku-router, 1.1 c2905f891f96a0ec9c7fab16916dbb46.clou= dfront.net (CloudFront) > > > <<< X-Cache: Miss from cloudfront > > > <<< X-Amz-Cf-Pop: AMS58-P6 > > > <<< X-Amz-Cf-Id: Xa10pwbYp4MCA5eToVa0shahNpzr2epWFUuQZS63bMpmbiDLKc2d= mQ=3D=3D > > > <<< > > > ---> static.crates.io:443 > > > resolving server address: static.crates.io:443 > > > SSL options: 82004850 > > > Peer verification enabled > > > Using OpenSSL default CA cert file and path > > > Verify hostname > > > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > > > Certificate subject: /CN=3Dstatic.crates.io > > > Certificate issuer: /C=3DBE/O=3DGlobalSign nv-sa/CN=3DGlobalSign Atla= s R3 DV TLS CA 2025 Q3 > > > requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate > > > >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1 > > > >>> Host: static.crates.io > > > >>> Accept: */* > > > >>> User-Agent: fetch libfetch/2.0 > > > >>> Connection: close > > > >>> > > > <<< HTTP/1.1 200 > > > <<< Connection: close > > > <<< Content-Length: 8244 > > > <<< etag: "198d7f7c37f632bc83502b28870b6d60" > > > content length: [8244] > > > <<< cache-control: public,max-age=3D31536000,immutable > > > <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D > > > <<< x-amz-id-2: Dj7mFoaqQJUc61mB1FcouUQcy0tajnacJvU+xOR3Be5//jo3GaSmd= CwSSgtYyDMnYE4cPCiPv7PV9m7z1Jk0WazYHTFPbYhm > > > <<< x-amz-request-id: KWXPQ4JP20HSYSY2 > > > <<< date: Wed, 14 Jan 2026 00:09:56 GMT > > > <<< x-amz-server-side-encryption: AES256 > > > <<< x-amz-replication-status: COMPLETED > > > <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT > > > <<< accept-ranges: bytes > > > last modified: [2024-08-04 01:36:47] > > > <<< content-type: application/gzip > > > <<< x-served-by: cache-ams21058-AMS > > > <<< server: AmazonS3 > > > <<< age: 25212 > > > <<< x-cache: HIT > > > <<< x-cache-hits: 148 > > > <<< > > > offset 0, length -1, size -1, clength 8244 > > > local size / mtime: 8244 / 1722735407 > > > remote size / mtime: 8244 / 1722735407 > > > dunce-1.0.5.crate =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A08244 =C2=A0B =C2=A0=C2=A010 MBps =C2=A0=C2=A0=C2= =A000s > > > > > > real =C2=A0=C2=A0=C2=A00m0.429s > > > user =C2=A0=C2=A0=C2=A00m0.023s > > > sys =C2=A0=C2=A0=C2=A00m0.027s > >=20 > > ``` > > # time fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?= dummy=3D/rust/crates/dunce-1.0.5.crate" > > scheme: =C2=A0=C2=A0"https" > > user: =C2=A0=C2=A0=C2=A0=C2=A0"" > > password: "" > > host: =C2=A0=C2=A0=C2=A0=C2=A0"crates.io" > > port: =C2=A0=C2=A0=C2=A0=C2=A0"0" > > document: "/api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dun= ce-1.0.5.crate" > > ---> crates.io:443 > > resolving server address: crates.io:443 (<- it gets stuck here for like= a minute) > > SSL options: 82004850 > > Peer verification enabled > > Using OpenSSL default CA cert file and path > > Verify hostname > > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > > Certificate subject: /CN=3Dcrates.io > > Certificate issuer: /C=3DUS/O=3DAmazon/CN=3DAmazon RSA 2048 M01 > > requesting https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy= =3D/rust/crates/dunce-1.0.5.crate > > >>> GET /api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dunce-= 1.0.5.crate HTTP/1.1 > > >>> Host: crates.io > > >>> Accept: */* > > >>> User-Agent: fetch libfetch/2.0 > > >>> Connection: close > > >>> > > <<< HTTP/1.1 302 Moved Temporarily > > <<< Content-Length: 0 > > <<< Connection: close > > content length: [0] > > <<< Access-Control-Allow-Origin: * > > <<< Date: Wed, 14 Jan 2026 14:34:21 GMT > > <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate > > <<< Nel: {"report_to":"heroku-nel","response_headers":["Via"],"max_age"= :3600,"success_fraction":0.01,"failure_fraction":0.1} > > 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate > > scheme: =C2=A0=C2=A0"https" > > user: =C2=A0=C2=A0=C2=A0=C2=A0"" > > password: "" > > host: =C2=A0=C2=A0=C2=A0=C2=A0"static.crates.io" > > port: =C2=A0=C2=A0=C2=A0=C2=A0"0" > > document: "/crates/dunce/dunce-1.0.5.crate" > > <<< Report-To: {"group":"heroku-nel","endpoints":[{"url":"https://nel.h= eroku.com/reports?s=3D9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D\u0026s= id=3Daf571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=3D1768401261"}],"max_age"= :3600} > > <<< Reporting-Endpoints: heroku-nel=3D"https://nel.heroku.com/reports?s= =3D9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D&sid=3Daf571f24-03ee-46d1-= 9f90-ab9030c2c74c&ts=3D1768401261" > > <<< Server: Heroku > > <<< Strict-Transport-Security: max-age=3D31536000; includeSubDomains > > <<< Via: 1.1 heroku-router, 1.1 608555ff8069f08e821e1679cd412294.cloudf= ront.net (CloudFront) > > <<< X-Cache: Miss from cloudfront > > <<< X-Amz-Cf-Pop: GRU3-P10 > > <<< X-Amz-Cf-Id: LvtqT2cBMH7thFfdy3ktelkhg-ALuajh-ucfDhnv7l_s73okAxXp7Q= =3D=3D > > <<< > > ---> static.crates.io:443 > > resolving server address: static.crates.io:443 > > SSL options: 82004850 > > Peer verification enabled > > Using OpenSSL default CA cert file and path > > Verify hostname > > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > > Certificate subject: /CN=3Dstatic.crates.io > > Certificate issuer: /C=3DBE/O=3DGlobalSign nv-sa/CN=3DGlobalSign Atlas = R3 DV TLS CA 2025 Q3 > > requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate > > >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1 > > >>> Host: static.crates.io > > >>> Accept: */* > > >>> User-Agent: fetch libfetch/2.0 > > >>> Connection: close > > >>> > > <<< HTTP/1.1 200 > > <<< Connection: close > > <<< Content-Length: 8244 > > <<< cache-control: public,max-age=3D31536000,immutable > > content length: [8244] > > <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D > > <<< x-amz-id-2: EzDoao06Q7mYZgq0ecISLqKGCrHOV1gBvzNGaaAfOg4QCWZ/d3G4pjY= 4ZAoo2BNA0jACxYFWJQQ=3D > > <<< x-amz-request-id: 7JJDJ8X4GESHDW5P > > <<< date: Wed, 14 Jan 2026 04:21:53 GMT > > <<< x-amz-replication-status: COMPLETED > > <<< x-amz-server-side-encryption: AES256 > > <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT > > <<< etag: "198d7f7c37f632bc83502b28870b6d60" > > last modified: [2024-08-04 01:36:47] > > <<< accept-ranges: bytes > > <<< content-type: application/gzip > > <<< x-served-by: cache-gru-sbgr1930032-GRU > > <<< server: AmazonS3 > > <<< age: 36813 > > <<< x-cache: HIT > > <<< x-cache-hits: 107 > > <<< > > offset 0, length -1, size -1, clength 8244 > > fetch: transfer timed out > > fetch: dunce-1.0.5.crate appears to be truncated: 0/8244 bytes > > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0184.03 real =C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A00.02 user =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A00.00 sys > >=20 > > so it's even worse, it does follow the redirect but somehow can't get t= he file at all, and all of that actually takes 3 minutes to do >=20 >=20 >=20 > That looks weird. It is hard to diagnose from a distance. How is you gene= ral network access? Do you have a lot of package loss? >=20 > Regards, > Ronald. >=20 > ah yeah=C2=B7=C2=B7=C2=B7 just tried pinging google.com and got 100% packet= loss, even tho' i'm on ethernet and internet works blazingly fast on Linux cURL can access google in an instant however, and as i mentioned i can easi= ly download these files using Links however that one takes a lot longer to load google and any other website fo= r some reason=C2=B7=C2=B7=C2=B7 must be something wrong with the network dr= iver, this didn't happen before