From owner-freebsd-python@freebsd.org  Wed May  4 05:29:02 2016
Return-Path: <owner-freebsd-python@freebsd.org>
Delivered-To: freebsd-python@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C1C7B2C280
 for <freebsd-python@mailman.ysv.freebsd.org>;
 Wed,  4 May 2016 05:29:02 +0000 (UTC)
 (envelope-from bogorodskiy@gmail.com)
Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3])
 by mx1.freebsd.org (Postfix) with ESMTP id 5DD5C1F75
 for <freebsd-python@freebsd.org>; Wed,  4 May 2016 05:29:02 +0000 (UTC)
 (envelope-from bogorodskiy@gmail.com)
Received: by mailman.ysv.freebsd.org (Postfix)
 id 597D1B2C27F; Wed,  4 May 2016 05:29:02 +0000 (UTC)
Delivered-To: python@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 569A6B2C27E
 for <python@mailman.ysv.freebsd.org>; Wed,  4 May 2016 05:29:02 +0000 (UTC)
 (envelope-from bogorodskiy@gmail.com)
Received: from mail-lf0-x22b.google.com (mail-lf0-x22b.google.com
 [IPv6:2a00:1450:4010:c07::22b])
 (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id D21CD1F73;
 Wed,  4 May 2016 05:29:01 +0000 (UTC)
 (envelope-from bogorodskiy@gmail.com)
Received: by mail-lf0-x22b.google.com with SMTP id u64so46605494lff.3;
 Tue, 03 May 2016 22:29:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=date:from:to:cc:subject:message-id:references:mime-version
 :content-disposition:in-reply-to:user-agent;
 bh=3IhBlmBjMjSN/xW1Uqd16kRMbDe1kxb3pbGqnnkkMU0=;
 b=udm21PUvtN9KfhxS/KBOJ5mwOB2O3oSaq2xe/xP03siY3ACIaN/IrR+1RgAhQxkS2J
 gXkiuf54TNivw2t7GRr+4TV6H5hiAmYjVqQazy9c3p2dvnIOxR7JF4/4YBH3vcxO60nG
 RX6odXYaZDbJ0UDchrq45xeEdy6OF72MmYapbT6H9KPg1MvDvFwa/tWkVFVDWA9n1NeX
 xllbyhdzhk9I4I4HmX0BdlbS0VzgT1VFo5ly27p1+UdeJ1N4DIAwZlP8WcJ/vxQErXId
 NKg+j7loIeAjIrfB+ZYNbdCXcaC8d3E3mDo0dEIl89kFSYBLIAS5T05+qou2cMF+uc6x
 5tMg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:date:from:to:cc:subject:message-id:references
 :mime-version:content-disposition:in-reply-to:user-agent;
 bh=3IhBlmBjMjSN/xW1Uqd16kRMbDe1kxb3pbGqnnkkMU0=;
 b=Zs+Qp/8S+LUf7Ld8tVzxt4HSkGGksXJdL+WjeQWbnxtNAkM58GlOy/ufWf9gBaoAMS
 wjXnkAphIVHc73e1ZFUjSnh1dtkZ8KinpGzlQZ2zspqjRMh8g91WahO3ha7G8iZSbeRj
 NylJQiKINe29J0r+C5QP3KcKHXYDW9UsQp0IFpvUY5e2NWzTmqPNU8dTor9ZtcMLlXRw
 mpxCJS0/YNz3NCQ8yeXLL32/i+uNT44nGQVcLHq86GicZBxC77NA0JM7NYjuREPhlD16
 unBmuPKrvyLU82+J2ZbXeC17nOnB8sxF+qhtJE/xRrDgtj4/XA53736GW/5ncgV1TV/r
 U9RQ==
X-Gm-Message-State: AOPr4FX/M1A+GKbWFi1hZa+AN49H8WZln2KlwADi0eHjCqQYgtR4Db2DpwMsOqcGPtlRyQ==
X-Received: by 10.112.53.168 with SMTP id c8mr3096712lbp.10.1462339739486;
 Tue, 03 May 2016 22:28:59 -0700 (PDT)
Received: from dev.san.ru ([88.147.129.60])
 by smtp.gmail.com with ESMTPSA id d1sm309437lbm.3.2016.05.03.22.28.58
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 03 May 2016 22:28:58 -0700 (PDT)
Date: Wed, 4 May 2016 08:28:56 +0300
From: Roman Bogorodskiy <bogorodskiy@gmail.com>
To: Roman Bogorodskiy <novel@FreeBSD.org>
Cc: python@FreeBSD.org
Subject: Re: pypi URL schema change
Message-ID: <20160504052854.GE20427@dev.san.ru>
References: <20160501053945.GA2709@kloomba>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20160501053945.GA2709@kloomba>
User-Agent: Mutt/1.5.24 (2015-08-30)
X-BeenThere: freebsd-python@freebsd.org
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: FreeBSD-specific Python issues <freebsd-python.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-python>, 
 <mailto:freebsd-python-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-python/>
List-Post: <mailto:freebsd-python@freebsd.org>
List-Help: <mailto:freebsd-python-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-python>,
 <mailto:freebsd-python-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 04 May 2016 05:29:02 -0000

  Roman Bogorodskiy wrote:

> Hi,
> 
> Recently pypi has changed its URL schema for distfiles. Specifically,
> it was:
> 
>   /packages/{python version}/{name[0]}/{name}/{filename}
> 
> and now it is:
> 
>   /packages/{hash[:2]}/{hash[2:4]}/{hash[4:]}/{filename} Where hash is
>   blake2b(file_content, digest_size=32).hexdigest().lower()
> 
> (copy/pasted from [1]).
> 
> That means that if you have MASTER_SITES= CHEESESHOP it won't be able to
> fetch newer packages (however, it looks like it's still fine for the
> previously uploaded packages).
> 
> As per [1], there's a fix applied on pypi.io that supports the old
> schema by redirection to a proper URL. However, it looks like fetch(1)
> is not quite happy about such a direct:
> 
> => Attempting to fetch
> https://pypi.io/packages/source/p/positional/positional-1.1.0.tar.gz
> fetch:
> https://pypi.io/packages/source/p/positional/positional-1.1.0.tar.gz:
> Temporary Redirect

^^^

In additional to that, it looks like a bug in fetch(1) to me that I've
tried to address here:

https://reviews.freebsd.org/D6167

> => Attempting to fetch
> http://distcache.FreeBSD.org/ports-distfiles/positional-1.1.0.tar.gz
> fetch:
> http://distcache.FreeBSD.org/ports-distfiles/positional-1.1.0.tar.gz:
> Not Found
> 
> For now I have to redefine SUBDIR manually with the hash, e.g.:
> 
> MASTER_SITES=   CHEESESHOP
> MASTER_SITE_SUBDIR=
> bf/a8/bc656a556a60b76c32830b57279f51714ab7c6366fd243d6ea86b6fcad46
> 
> However, it feels a little awkward to keep a long hash like that in the
> Makefile.
> 
> Any thoughts if there's a way to avoid that?
> 
> 1: https://bitbucket.org/pypa/pypi/issues/438/backwards-compatible-un-hashed-package
> 
> Roman Bogorodskiy



Roman Bogorodskiy