From nobody Wed Jan 14 14:41:49 2026 X-Original-To: freebsd-current@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4drpjK4svsz6NVTy for ; Wed, 14 Jan 2026 14:41:57 +0000 (UTC) (envelope-from red_M95@proton.me) Received: from mail-43167.protonmail.ch (mail-43167.protonmail.ch [185.70.43.167]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "protonmail.com", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4drpjK1fjWz4HSM for ; Wed, 14 Jan 2026 14:41:57 +0000 (UTC) (envelope-from red_M95@proton.me) Authentication-Results: mx1.freebsd.org; none DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=proton.me; s=protonmail; t=1768401712; x=1768660912; bh=ZZKPQylg0cc0UFCvkKNA2uDro1EbQyi3DzcJNsHz6tU=; h=Date:To:From:Subject:Message-ID:In-Reply-To:References: Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID: Message-ID:BIMI-Selector; b=P3SNFGjBkYL/2p2VCJsZTDI9Y03MF9J5HJXWA4hTS8sjqFF2xA4mwW3UqLKpZS5OV dGwy0N1PPPtwJv/5kK97zXEey9yHZV6spYSU3B/EjlNAA8v7LhBB2FeqB8NAmuOYbC VP0kZsZUPfYuarHzDGnEy/imtd6JLYLO7bP5fwNcRvfbgTQ2Jas+maCOyJTa7w1NF/ FBYUiePvm7xUH6ZT4nrfWVQRFKGy+7Eti5wmaN8nA+7vUshHtaMuJE2pGAaX+BhYBa KLfy4ls3e5L7Nvg/8EVAb8e6BWSEDwDaGYYaWeUo+D3CGlG9Eoxuqsbd0di/Zo/jhz fvI3EteeGOdcg== Date: Wed, 14 Jan 2026 14:41:49 +0000 To: Ronald Klop , Current FreeBSD From: ruby R53 Subject: Re: Change fetch addresses for Rust crates, Python packages and whatnot? Message-ID: <9Qp2LaDX-QG6aXtsRcjekPDWOjni9Cy4LLmza8jhB2cD15_VfdTY7znjCzIClM1qtEhVuK4P_cL1yKl8AMJZ3m8UnE3EEC67polveeC_Zys=@proton.me> In-Reply-To: <1503278086.9.1768374682910@localhost> References: <1503278086.9.1768374682910@localhost> Feedback-ID: 66100208:user:proton X-Pm-Message-ID: 7d3dfa9d0ad91fc90c6346ca2bcb1a51f292fe03 List-Id: Discussions about the use of FreeBSD-current List-Archive: https://lists.freebsd.org/archives/freebsd-current List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-current@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:62371, ipnet:185.70.43.0/24, country:CH] X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Rspamd-Queue-Id: 4drpjK1fjWz4HSM On Wednesday, January 14th, 2026 at 4:11 AM, Ronald Klop wrote: >=20 >=20 > Van: ruby R53 > Datum: woensdag, 14 januari 2026 03:26 > Aan: freebsd-current@freebsd.org > Onderwerp: Change fetch addresses for Rust crates, Python packages and wh= atnot? >=20 > > I've been trying to compile www/chromium for literal days now, all beca= use whenever `make` runs `fetch`, it goes to an address that's actually sup= posed to redirect to another one, but `fetch` instead keeps trying for a wh= ole minute until it ultimately falls back to distcache.freebsd.org, which i= s another whole minute of waiting for the file to download. The cherry on t= op is that the file is, most of the time, just hundreds of kilobytes in siz= e. > >=20 > > Hell, I even had to manually download part of these individual files us= ing that Links web browser=C2=B7=C2=B7=C2=B7 it'd do it a lot faster than `= fetch`, all because it actually followed the redirection and didn't just ke= ep trying over and over. But of course I wouldn't manually download hundred= s upon hundreds of archive files, this is the reason why such tool exists. > >=20 > > So here's an example: currently one of the missing dependencies for www= /chromium is GTK3, which depends on graphics/librsvg2-rust, which then depe= nds on cargo-c. It's now trying to fetch dunce-1.0.5.crate, which is one of= the crates required for it, from https://crates.io/api/v1/crates/dunce/1.0= .5/download?dummy=3D/rust/crates/dunce-1.0.5.crate. Running `curl` on this = link does literally nothing, no files are downloaded. But running it on any= web browser instantly downloads the crate in question, all because that li= nk actually redirects to https://static.crates.io/crates/dunce/dunce-1.0.5.= crate. Even `curl`ing that new link does give the file I want. Same applies= for Python packages. > >=20 > > Now imagine this for all the remaining files=C2=B7=C2=B7=C2=B7 if it ta= kes about 2 minutes to download a single file, and there's 398 crate entrie= s on cargo-c's `Makefile.crates`, that means it'd take a whole 796 minutes = to download everything, which is literally more than half a day (~13.6 hour= s) :) > > =C2=A0 >=20 >=20 >=20 > Can you copy-paste the output of the following command in a reply? > fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=3D/= rust/crates/dunce-1.0.5.crate" >=20 > For me it immediately follows the redirect and downloads the file in half= a second. See the output below. >=20 > Regards, > Ronald. >=20 >=20 > --=C2=A0 >=20 > time fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?dumm= y=3D/rust/crates/dunce-1.0.5.crate" > scheme: "https" > user: "" > password: "" > host: "crates.io" > port: "0" > document: "/api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dunce= -1.0.5.crate" > ---> crates.io:443 > resolving server address: crates.io:443 > SSL options: 82004850 > Peer verification enabled > Using OpenSSL default CA cert file and path > Verify hostname > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > Certificate subject: /CN=3Dcrates.io > Certificate issuer: /C=3DUS/O=3DAmazon/CN=3DAmazon RSA 2048 M01 > requesting https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=3D/= rust/crates/dunce-1.0.5.crate > >>> GET /api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dunce-1.= 0.5.crate HTTP/1.1 > >>> Host: crates.io > >>> Accept: */* > >>> User-Agent: fetch libfetch/2.0 > >>> Connection: close > >>> > <<< HTTP/1.1 302 Moved Temporarily > <<< Content-Length: 0 > <<< Connection: close > content length: [0] > <<< Access-Control-Allow-Origin: * > <<< Date: Wed, 14 Jan 2026 07:10:06 GMT > <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate > <<< Nel: {"report_to":"heroku-nel","response_headers":["Via"],"max_age":3= 600,"success_fraction":0.01,"failure_fraction":0.1} > 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate > scheme: "https" > user: "" > password: "" > host: "static.crates.io" > port: "0" > document: "/crates/dunce/dunce-1.0.5.crate" > <<< Report-To: {"group":"heroku-nel","endpoints":[{"url":"https://nel.her= oku.com/reports?s=3DVOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D\u0026s= id=3Daf571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=3D1768374606"}],"max_age"= :3600} > <<< Reporting-Endpoints: heroku-nel=3D"https://nel.heroku.com/reports?s= =3DVOwdWzeBall4rSiByZHUwCe6zIbVFdxy%2FIgcGEUOIZo%3D&sid=3Daf571f24-03ee-46d= 1-9f90-ab9030c2c74c&ts=3D1768374606" > <<< Server: Heroku > <<< Strict-Transport-Security: max-age=3D31536000; includeSubDomains > <<< Via: 1.1 heroku-router, 1.1 c2905f891f96a0ec9c7fab16916dbb46.cloudfro= nt.net (CloudFront) > <<< X-Cache: Miss from cloudfront > <<< X-Amz-Cf-Pop: AMS58-P6 > <<< X-Amz-Cf-Id: Xa10pwbYp4MCA5eToVa0shahNpzr2epWFUuQZS63bMpmbiDLKc2dmQ= =3D=3D > <<< > ---> static.crates.io:443 > resolving server address: static.crates.io:443 > SSL options: 82004850 > Peer verification enabled > Using OpenSSL default CA cert file and path > Verify hostname > TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 > Certificate subject: /CN=3Dstatic.crates.io > Certificate issuer: /C=3DBE/O=3DGlobalSign nv-sa/CN=3DGlobalSign Atlas R3= DV TLS CA 2025 Q3 > requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate > >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1 > >>> Host: static.crates.io > >>> Accept: */* > >>> User-Agent: fetch libfetch/2.0 > >>> Connection: close > >>> > <<< HTTP/1.1 200 > <<< Connection: close > <<< Content-Length: 8244 > <<< etag: "198d7f7c37f632bc83502b28870b6d60" > content length: [8244] > <<< cache-control: public,max-age=3D31536000,immutable > <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D > <<< x-amz-id-2: Dj7mFoaqQJUc61mB1FcouUQcy0tajnacJvU+xOR3Be5//jo3GaSmdCwSS= gtYyDMnYE4cPCiPv7PV9m7z1Jk0WazYHTFPbYhm > <<< x-amz-request-id: KWXPQ4JP20HSYSY2 > <<< date: Wed, 14 Jan 2026 00:09:56 GMT > <<< x-amz-server-side-encryption: AES256 > <<< x-amz-replication-status: COMPLETED > <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT > <<< accept-ranges: bytes > last modified: [2024-08-04 01:36:47] > <<< content-type: application/gzip > <<< x-served-by: cache-ams21058-AMS > <<< server: AmazonS3 > <<< age: 25212 > <<< x-cache: HIT > <<< x-cache-hits: 148 > <<< > offset 0, length -1, size -1, clength 8244 > local size / mtime: 8244 / 1722735407 > remote size / mtime: 8244 / 1722735407 > dunce-1.0.5.crate 8244 B 10 MBps = 00s >=20 > real 0m0.429s > user 0m0.023s > sys 0m0.027s ``` # time fetch -vv "https://crates.io/api/v1/crates/dunce/1.0.5/download?dumm= y=3D/rust/crates/dunce-1.0.5.crate" scheme: "https" user: "" password: "" host: "crates.io" port: "0" document: "/api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dunce-1= .0.5.crate" ---> crates.io:443 resolving server address: crates.io:443 (<- it gets stuck here for like a m= inute) SSL options: 82004850 Peer verification enabled Using OpenSSL default CA cert file and path Verify hostname TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 Certificate subject: /CN=3Dcrates.io Certificate issuer: /C=3DUS/O=3DAmazon/CN=3DAmazon RSA 2048 M01 requesting https://crates.io/api/v1/crates/dunce/1.0.5/download?dummy=3D/ru= st/crates/dunce-1.0.5.crate >>> GET /api/v1/crates/dunce/1.0.5/download?dummy=3D/rust/crates/dunce-1.0.= 5.crate HTTP/1.1 >>> Host: crates.io >>> Accept: */* >>> User-Agent: fetch libfetch/2.0 >>> Connection: close >>> <<< HTTP/1.1 302 Moved Temporarily <<< Content-Length: 0 <<< Connection: close content length: [0] <<< Access-Control-Allow-Origin: * <<< Date: Wed, 14 Jan 2026 14:34:21 GMT <<< Location: https://static.crates.io/crates/dunce/dunce-1.0.5.crate <<< Nel: {"report_to":"heroku-nel","response_headers":["Via"],"max_age":360= 0,"success_fraction":0.01,"failure_fraction":0.1} 302 redirect to https://static.crates.io/crates/dunce/dunce-1.0.5.crate scheme: "https" user: "" password: "" host: "static.crates.io" port: "0" document: "/crates/dunce/dunce-1.0.5.crate" <<< Report-To: {"group":"heroku-nel","endpoints":[{"url":"https://nel.herok= u.com/reports?s=3D9t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D\u0026sid= =3Daf571f24-03ee-46d1-9f90-ab9030c2c74c\u0026ts=3D1768401261"}],"max_age":3= 600} <<< Reporting-Endpoints: heroku-nel=3D"https://nel.heroku.com/reports?s=3D9= t0rqIH4lMlfE9ncVibRPy8hCUlMqBmippMeWunG5M0%3D&sid=3Daf571f24-03ee-46d1-9f90= -ab9030c2c74c&ts=3D1768401261" <<< Server: Heroku <<< Strict-Transport-Security: max-age=3D31536000; includeSubDomains <<< Via: 1.1 heroku-router, 1.1 608555ff8069f08e821e1679cd412294.cloudfront= .net (CloudFront) <<< X-Cache: Miss from cloudfront <<< X-Amz-Cf-Pop: GRU3-P10 <<< X-Amz-Cf-Id: LvtqT2cBMH7thFfdy3ktelkhg-ALuajh-ucfDhnv7l_s73okAxXp7Q=3D= =3D <<< ---> static.crates.io:443 resolving server address: static.crates.io:443 SSL options: 82004850 Peer verification enabled Using OpenSSL default CA cert file and path Verify hostname TLSv1.3 connection established using TLS_AES_128_GCM_SHA256 Certificate subject: /CN=3Dstatic.crates.io Certificate issuer: /C=3DBE/O=3DGlobalSign nv-sa/CN=3DGlobalSign Atlas R3 D= V TLS CA 2025 Q3 requesting https://static.crates.io/crates/dunce/dunce-1.0.5.crate >>> GET /crates/dunce/dunce-1.0.5.crate HTTP/1.1 >>> Host: static.crates.io >>> Accept: */* >>> User-Agent: fetch libfetch/2.0 >>> Connection: close >>> <<< HTTP/1.1 200 <<< Connection: close <<< Content-Length: 8244 <<< cache-control: public,max-age=3D31536000,immutable content length: [8244] <<< x-amz-version-id: PO7xZdimM08F9npIqG8PYtpqpEuWdV9D <<< x-amz-id-2: EzDoao06Q7mYZgq0ecISLqKGCrHOV1gBvzNGaaAfOg4QCWZ/d3G4pjY4ZAo= o2BNA0jACxYFWJQQ=3D <<< x-amz-request-id: 7JJDJ8X4GESHDW5P <<< date: Wed, 14 Jan 2026 04:21:53 GMT <<< x-amz-replication-status: COMPLETED <<< x-amz-server-side-encryption: AES256 <<< last-modified: Sun, 04 Aug 2024 01:36:47 GMT <<< etag: "198d7f7c37f632bc83502b28870b6d60" last modified: [2024-08-04 01:36:47] <<< accept-ranges: bytes <<< content-type: application/gzip <<< x-served-by: cache-gru-sbgr1930032-GRU <<< server: AmazonS3 <<< age: 36813 <<< x-cache: HIT <<< x-cache-hits: 107 <<< offset 0, length -1, size -1, clength 8244 fetch: transfer timed out fetch: dunce-1.0.5.crate appears to be truncated: 0/8244 bytes 184.03 real 0.02 user 0.00 sys so it's even worse, it does follow the redirect but somehow can't get the f= ile at all, and all of that actually takes 3 minutes to do