Date: Sat, 02 Nov 2013 11:15:07 +0100 From: Matthias Andree <matthias.andree@gmx.de> To: Matthew Seaman <matthew@FreeBSD.org>,freebsd-current@freebsd.org Subject: Re: Official FreeBSD Binary Packages now available for pkgng Message-ID: <1680682c-dc77-4ee3-8e59-ee7356f307a3@email.android.com> In-Reply-To: <5274B947.7030607@FreeBSD.org> References: <5271BC11.1010303@FreeBSD.org> <CACeEFf4Hif3WHufC=i08gbkXb6oC=4sxbyvO4FQnTkRWA7ZwnA@mail.gmail.com> <5272D0DE.4080209@FreeBSD.org> <CACeEFf79RZskefh6RzBhxkHuAWnGjPWRDr_JBTRUWcGH4ZNVTg@mail.gmail.com> <CAOjFWZ7BbdXYi3gQtTvCa3jqTmjOC-tt5bwV1GR8Jf=tOanT%2BQ@mail.gmail.com> <52745B7F.2080608@vangyzen.net> <5274B947.7030607@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Matthew Seaman <matthew@FreeBSD.org> schrieb: >On 02/11/2013 01:55, Eric van Gyzen wrote: >> This kind of proxy configuration is not uncommon. It would be >awesome >> if this would Just Work. It would remove an impediment to adoption, >> which is especially important in the kind of environments that have >this >> kind of proxy configuration. >> >> Simply adding the mirrors' A (and AAAA) records to pkg.freebsd.org >might >> suffice. > >You seem hung up on the idea that pkg.freebsd.org should resolve to a >list of IP addresses. It doesn't and for very good reasons. >Admittedly, using eg. 'http://' as the URL scheme for PACKAGESITE URLs >was an error -- it contravenes RFC 2616 -- which is why we will be >switching to a new 'pkg+http://' (or 'pkg+https://', 'pkg+ftp://', >etc.) >set of URL schemes with pkg-1.2.x > >There certainly are all of the necessary A and AAAA records in the DNS >for the real servers that host the repositories. > >If I understand what you're complaining about is that you see behavious >like the following: > > * You download package foo-1.2.3.txz from pkg.freebsd.org > > * Internally, that gets resolved to an HTTP request to eg. > pkg0.isc.freebsd.org > > * Your web proxy caches this package > > * On another host, you also want to download foo-1.2.3.txz > > * This time the SRV record gets resolved to a different mirror, > say pkg1.nyi.freebsd.org > > * Your proxy has no way of knowing that foo-1.2.3.txz from pkg1.nyi > is exactly the same file as foo-1.2.3.txz from pkg0.isc so it > downloads the whole package all over again. > >Yes, this is certainly undesirable behaviour. I need to run some tests >to determine if this is actually what does happen in practice. If so, >I've an idea about how this problem might be addressed, but it will >require some changes to the repository configuration. > >In the mean time, I suggest just choosing which ever of the >pkg.freebsd.org repositories is closest to you and using it directly -- >eg. > >cat <<EOF > /usr/local/etc/pkg/repos/myrepo.conf >pkg0.isc { > url: http://pkg0.isc.freebsd.org/${ABI}/latest > enabled: yes > mirror_type: none >} >EOF > >Obviously, substitute which ever one of > > pkg0.isc.freebsd.org (US West) > pkg1.nyi.freebsd.org (US East) > pkg0.bme.freebsd.org (Europe) > >is appropriate. And be prepared to deal with that specific mirror >being >down or replaced by some other server. > >> Alternatively, running an HTTP-redirection service on a host named >> pkg.freebsd.org would offer as much flexibility as the SRV records, >if >> not more. However, it would require maintenance of yet another >central >> service. > >This is already supported in pkg when using the HTTP mirror type. This >would entail significantly more administrative effort and hardware >requirement to maintain and keep consistent in the specific case of >pkg.freebsd.org which is exactly why the SRV mirror type was selected. > > Cheers, > > Matthew > > >-- >Dr Matthew J Seaman MA, D.Phil. >PGP: http://www.infracaninophile.co.uk/pgpkey I understand from Eric's pist that the issue is that through his limiting proxies, the SRV are not available at all so he does not even get to the point where he could get the pkgN.nyi.freebsd.org name back. From owner-freebsd-current@FreeBSD.ORG Sat Nov 2 10:51:08 2013 Return-Path: <owner-freebsd-current@FreeBSD.ORG> Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3A6045A5 for <freebsd-current@freebsd.org>; Sat, 2 Nov 2013 10:51:08 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CCCF42E37 for <freebsd-current@freebsd.org>; Sat, 2 Nov 2013 10:51:07 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.2.117.99]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.7/8.14.7) with ESMTP id rA2Ap3nv075228 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits%6 verify=NO); Sat, 2 Nov 2013 10:51:03 GMT (envelope-from matthew@FreeBSD.org) DKIM-Filter: OpenDKIM Filter v2.8.3 smtp.infracaninophile.co.uk rA2Ap3nv075228 Authentication-Results: smtp.infracaninophile.co.uk/rA2Ap3nv075228; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy) Message-ID: <5274D90D.8040508@FreeBSD.org> Date: Sat, 02 Nov 2013 10:50:53 +0000 From: Matthew Seaman <matthew@FreeBSD.org> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Matthias Andree <matthias.andree@gmx.de>, freebsd-current@freebsd.org Subject: Re: Official FreeBSD Binary Packages now available for pkgng References: <5271BC11.1010303@FreeBSD.org> <CACeEFf4Hif3WHufC=i08gbkXb6oC=4sxbyvO4FQnTkRWA7ZwnA@mail.gmail.com> <5272D0DE.4080209@FreeBSD.org> <CACeEFf79RZskefh6RzBhxkHuAWnGjPWRDr_JBTRUWcGH4ZNVTg@mail.gmail.com> <CAOjFWZ7BbdXYi3gQtTvCa3jqTmjOC-tt5bwV1GR8Jf=tOanT+Q@mail.gmail.com> <52745B7F.2080608@vangyzen.net> <5274B947.7030607@FreeBSD.org> <1680682c-dc77-4ee3-8e59-ee7356f307a3@email.android.com> In-Reply-To: <1680682c-dc77-4ee3-8e59-ee7356f307a3@email.android.com> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b" X-Virus-Scanned: clamav-milter 0.97.8 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 02 Nov 2013 10:51:08 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 02/11/2013 10:15, Matthias Andree wrote: > I understand from Eric's pist that the issue is that through his > limiting proxies, the SRV are not available at all so he does not even > get to the point where he could get the pkgN.nyi.freebsd.org > <http://pkgN.nyi.freebsd.org>; name back. That doesn't make sense. All the DNS SRV lookups on pkg.freebsd.org are done internally to pkg(8), which then issues an HTTP GET to the specific mirror selected by its internal algorithms. The web cache won't see literal 'pkg.freebsd.org' anywhere in the HTTP traffic -- as far as it is concerned, it's a simple HTTP request to a specific mirror 'pkg1.nyi.freebsd.org', and can be cached using the usual processes. What makes it cache unfriendly is that as far as the web cache is concerned each of the different mirrors appears to be completely independent of the others. So at the moment the chance of getting a cache hit is reduced by a factor of three because of the traffic distribution across the three mirrors. Cheers, Matthew -- Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJ8BAEBCgBmBQJSdNkWXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATgCUP/2Ir45H/CTYBxADUYWNCQp74 2mfCvUdmMfbWMOVooKee7MWYknrx5oVvHgHGcR8Gqge+IylhbotYvjL2ahR+Hyh8 y9bm6liUhYJTp5ACqJistduwjiFmLaQ7+OPewjOOXk5vssQzLyejAZFj4TOZGCE5 8FVTUgfGf//C9NGTEtRxRmLn/V32NWsJiyBGf4mz9CpJglJGeUJeMq7VTeM9AmIY F2n80K32FqcLWp26rPWjjuRHoh7T4c9FuEqcbw5Z2vpiJLTd0drg4kBt9bcU3gry 2pfDRQOq8K8BJ4jGPbkvsjkmp4wyttKd3YO2Gd3hocP3pKRzsGIhPPaiBVGq1QcE nlmF4IB0ntNu1OSa/X6mFNZsnKIUPlPgCucnIobaCZwxS2ScfiITl4XvF9PxExx5 VQyZ2/jhL3Rz3X4AS++CrOwZgA+zxbmyVT4Zvc+y2m5+JjcEVZu8d0ARWlNlUoER swE1FclEBqIaMPNoofitihrtuBcefaAz5ECBCTVG8JFPxWV1BeZiWkBDegSA5Gez UoRSyLh/ymRnio6shbsmrYjnM8/Jv4juXlUG5MVT/IICwlZJBei7Q4c0K7fMtgvP NqBQB+K8R3Sr16a0HPOhIw1+eoNSA62B51UP4qbC1tHuvTTOHp4b85ngOBMl5UXs 5BHVJ+aRPCnBKevemUpR =1WKR -----END PGP SIGNATURE----- --u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1680682c-dc77-4ee3-8e59-ee7356f307a3>