Skip site navigation (1)Skip section navigation (2)
Date:      Fri, 3 Apr 2020 19:36:19 +0200
From:      Dimitry Andric <dim@FreeBSD.org>
To:        Kurt Jaeger <pi@freebsd.org>
Cc:        Lev Serebryakov <lev@freebsd.org>, ports@freebsd.org
Subject:   Re: fetch is tarpitted by Texas Instruments and/or Akamai and can not download distfiles for TI-related ports.
Message-ID:  <88F7E459-73D5-444C-A473-85C78A4484C5@FreeBSD.org>
In-Reply-To: <20200403130821.GB37073@home.opsec.eu>
References:  <04228bf1-5d14-8458-37d3-66ec28645763@FreeBSD.org> <20200403130821.GB37073@home.opsec.eu>

next in thread | previous in thread | raw e-mail | index | archive | help

--Apple-Mail=_C0974ACC-C957-4724-9C50-0F69FBFABA09
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=us-ascii

On 3 Apr 2020, at 15:08, Kurt Jaeger <pi@freebsd.org> wrote:
>=20
> Hi!
>=20
>> It is true for both IPv4 and IPv6, and for Russia and Germany (I can
>> not test from USA, though).
>=20
> I tried it from freefall, same problem.

Typically one of those "Endpoint Protection" products, they have some =
sort of whitelist or blacklist of bad user agents. Using "curl/7.68.0" =
as user agent works:

$ fetch -vvv --user-agent=3D"curl/7.68.0" =
http://www.ti.com/lit/ug/slau646e/slau646e.pdf
scheme:   "http"
user:     ""
password: ""
host:     "www.ti.com"
port:     "0"
document: "/lit/ug/slau646e/slau646e.pdf"
---> www.ti.com:80
resolving server address: www.ti.com:80
requesting http://www.ti.com/lit/ug/slau646e/slau646e.pdf
>>> GET /lit/ug/slau646e/slau646e.pdf HTTP/1.1
>>> Host: www.ti.com
>>> Accept: */*
>>> User-Agent: curl/7.68.0
>>> Connection: close
>>>
<<< HTTP/1.1 200 OK
<<< Server: Apache
<<< Last-Modified: Wed, 26 Jun 2019 21:34:37 GMT
<<< ETag: "1551ab-58c40cfeb6c6f"
last modified: [2019-06-26 21:34:37]
<<< Accept-Ranges: bytes
<<< Content-Length: 1397163
<<< Cache-Control: max-age=3D60
content length: [1397163]
<<< Expires: Thu, 30 Jan 2020 09:27:51 GMT
<<< Content-Type: application/pdf
<<< Date: Fri, 03 Apr 2020 17:30:20 GMT
<<< Connection: close
<<< Set-Cookie: =
ti_geo=3Dcountry=3DXX|city=3DYYYYYYY|continent=3DEU|tc_ip=3Dn.m.o.p; =
path=3D/; domain=3D.ti.com
<<< Set-Cookie: ti_rid=3D3110b500; path=3D/; domain=3D.ti.com
<<< Set-Cookie: ti_ua=3Dcurl%2f7.68.0; path=3D/; domain=3D.ti.com
<<< Set-Cookie: ti_ak_id=3D67fba42067fba42067fba4205e8772ac3110b500; =
expires=3DSat, 03-Apr-2021 17:30:20 GMT; path=3D/; domain=3D.ti.com
<<< Set-Cookie: ti_ridh=3Dexpired; path=3D/; domain=3Dti.com; =
expires=3DThu, 1-Aug-2019 06:14:04 GMT
<<< Set-Cookie: ti_bm=3D; path=3D/; domain=3D.ti.com
<<<
offset 0, length -1, size -1, clength 1397163
local size / mtime: 1397163 / 1585934997
remote size / mtime: 1397163 / 1561584877
slau646e.pdf                                          1364 kB   15 MBps  =
  00s

You can immediately see they're trying to violate cookie laws. ;-)  The =
"ti_ua" cookie stores the user agent, and I guess "tk_ak_id" is the =
Akamai ID (a.k.a. personally identifiable information).

-Dimitry


--Apple-Mail=_C0974ACC-C957-4724-9C50-0F69FBFABA09
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
	filename=signature.asc
Content-Type: application/pgp-signature;
	name=signature.asc
Content-Description: Message signed with OpenPGP

-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.2

iF0EARECAB0WIQR6tGLSzjX8bUI5T82wXqMKLiCWowUCXod0EwAKCRCwXqMKLiCW
o3pKAKDYUj8CokLhlH9Cn2oEpgHi+94KdwCfUaoV642XozaxpsDkCUQB3WIs2FA=
=TS4h
-----END PGP SIGNATURE-----

--Apple-Mail=_C0974ACC-C957-4724-9C50-0F69FBFABA09--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?88F7E459-73D5-444C-A473-85C78A4484C5>