Date: Sat, 02 Nov 2013 11:15:07 +0100 From: Matthias Andree <matthias.andree@gmx.de> To: Matthew Seaman <matthew@FreeBSD.org>,freebsd-current@freebsd.org Subject: Re: Official FreeBSD Binary Packages now available for pkgng Message-ID: <1680682c-dc77-4ee3-8e59-ee7356f307a3@email.android.com> In-Reply-To: <5274B947.7030607@FreeBSD.org> References: <5271BC11.1010303@FreeBSD.org> <CACeEFf4Hif3WHufC=i08gbkXb6oC=4sxbyvO4FQnTkRWA7ZwnA@mail.gmail.com> <5272D0DE.4080209@FreeBSD.org> <CACeEFf79RZskefh6RzBhxkHuAWnGjPWRDr_JBTRUWcGH4ZNVTg@mail.gmail.com> <CAOjFWZ7BbdXYi3gQtTvCa3jqTmjOC-tt5bwV1GR8Jf=tOanT%2BQ@mail.gmail.com> <52745B7F.2080608@vangyzen.net> <5274B947.7030607@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Matthew Seaman <matthew@FreeBSD=2Eorg> schrieb: >On 02/11/2013 01:55, Eri= c van Gyzen wrote: >> This kind of proxy configuration is not uncommon=2E = It would be >awesome >> if this would Just Work=2E It would remove an impe= diment to adoption, >> which is especially important in the kind of environ= ments that have >this >> kind of proxy configuration=2E >> >> Simply addin= g the mirrors' A (and AAAA) records to pkg=2Efreebsd=2Eorg >might >> suffic= e=2E > >You seem hung up on the idea that pkg=2Efreebsd=2Eorg should resolv= e to a >list of IP addresses=2E It doesn't and for very good reasons=2E >A= dmittedly, using eg=2E 'http://' as the URL scheme for PACKAGESITE URLs >wa= s an error -- it contravenes RFC 2616 -- which is why we will be >switching= to a new 'pkg+http://' (or 'pkg+https://', 'pkg+ftp://', >etc=2E) >set of = URL schemes with pkg-1=2E2=2Ex > >There certainly are all of the necessary = A and AAAA records in the DNS >for the real servers that host the repositor= ies=2E > >If I understand what you're complaining about is that you see beh= avious >like the following: > > * You download package foo-1=2E2=2E3=2Etx= z from pkg=2Efreebsd=2Eorg > > * Internally, that gets resolved to an HTT= P request to eg=2E > pkg0=2Eisc=2Efreebsd=2Eorg > > * Your web proxy = caches this package > > * On another host, you also want to download foo-= 1=2E2=2E3=2Etxz > > * This time the SRV record gets resolved to a differe= nt mirror, > say pkg1=2Enyi=2Efreebsd=2Eorg > > * Your proxy has no w= ay of knowing that foo-1=2E2=2E3=2Etxz from pkg1=2Enyi > is exactly the= same file as foo-1=2E2=2E3=2Etxz from pkg0=2Eisc so it > downloads the= whole package all over again=2E > >Yes, this is certainly undesirable beha= viour=2E I need to run some tests >to determine if this is actually what d= oes happen in practice=2E If so, >I've an idea about how this problem migh= t be addressed, but it will >require some changes to the repository configu= ration=2E > >In the mean time, I suggest just choosing which ever of the >p= kg=2Efreebsd=2Eorg repositories is closest to you and using it directly -- = >eg=2E > >cat <<EOF > /usr/local/etc/pkg/repos/myrepo=2Econf >pkg0=2Eisc { = > url: http://pkg0=2Eisc=2Efreebsd=2Eorg/${ABI}/latest > enabled: yes= > mirror_type: none >} >EOF > >Obviously, substitute which ever one of = > > pkg0=2Eisc=2Efreebsd=2Eorg (US West) > pkg1=2Enyi=2Efreebsd=2Eorg= (US East) > pkg0=2Ebme=2Efreebsd=2Eorg (Europe) > >is appropriate=2E= And be prepared to deal with that specific mirror >being >down or replace= d by some other server=2E > >> Alternatively, running an HTTP-redirection s= ervice on a host named >> pkg=2Efreebsd=2Eorg would offer as much flexibili= ty as the SRV records, >if >> not more=2E However, it would require mainte= nance of yet another >central >> service=2E > >This is already supported in= pkg when using the HTTP mirror type=2E This >would entail significantly m= ore administrative effort and hardware >requirement to maintain and keep co= nsistent in the specific case of >pkg=2Efreebsd=2Eorg which is exactly why= the SRV mirror type was selected=2E > > Cheers, > > Matthew > > >-- >Dr M= atthew J Seaman MA, D=2EPhil=2E >PGP: http://www=2Einfracaninophile=2Eco=2E= uk/pgpkey I understand from Eric's pist that the issue is that through his= limiting proxies, the SRV are not available at all so he does not even get= to the point where he could get the pkgN=2Enyi=2Efreebsd=2Eorg name back= =2E From owner-freebsd-current@FreeBSD.ORG Sat Nov 2 10:51:08 2013 Return-Path: <owner-freebsd-current@FreeBSD.ORG> Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 3A6045A5 for <freebsd-current@freebsd.org>; Sat, 2 Nov 2013 10:51:08 +0000 (UTC) (envelope-from matthew@FreeBSD.org) Received: from smtp.infracaninophile.co.uk (smtp6.infracaninophile.co.uk [IPv6:2001:8b0:151:1:3cd3:cd67:fafa:3d78]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id CCCF42E37 for <freebsd-current@freebsd.org>; Sat, 2 Nov 2013 10:51:07 +0000 (UTC) Received: from seedling.black-earth.co.uk (seedling.black-earth.co.uk [81.2.117.99]) (authenticated bits=0) by smtp.infracaninophile.co.uk (8.14.7/8.14.7) with ESMTP id rA2Ap3nv075228 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sat, 2 Nov 2013 10:51:03 GMT (envelope-from matthew@FreeBSD.org) DKIM-Filter: OpenDKIM Filter v2.8.3 smtp.infracaninophile.co.uk rA2Ap3nv075228 Authentication-Results: smtp.infracaninophile.co.uk/rA2Ap3nv075228; dkim=none reason="no signature"; dkim-adsp=none (unprotected policy) Message-ID: <5274D90D.8040508@FreeBSD.org> Date: Sat, 02 Nov 2013 10:50:53 +0000 From: Matthew Seaman <matthew@FreeBSD.org> User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:24.0) Gecko/20100101 Thunderbird/24.1.0 MIME-Version: 1.0 To: Matthias Andree <matthias.andree@gmx.de>, freebsd-current@freebsd.org Subject: Re: Official FreeBSD Binary Packages now available for pkgng References: <5271BC11.1010303@FreeBSD.org> <CACeEFf4Hif3WHufC=i08gbkXb6oC=4sxbyvO4FQnTkRWA7ZwnA@mail.gmail.com> <5272D0DE.4080209@FreeBSD.org> <CACeEFf79RZskefh6RzBhxkHuAWnGjPWRDr_JBTRUWcGH4ZNVTg@mail.gmail.com> <CAOjFWZ7BbdXYi3gQtTvCa3jqTmjOC-tt5bwV1GR8Jf=tOanT+Q@mail.gmail.com> <52745B7F.2080608@vangyzen.net> <5274B947.7030607@FreeBSD.org> <1680682c-dc77-4ee3-8e59-ee7356f307a3@email.android.com> In-Reply-To: <1680682c-dc77-4ee3-8e59-ee7356f307a3@email.android.com> X-Enigmail-Version: 1.6 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b" X-Virus-Scanned: clamav-milter 0.97.8 at lucid-nonsense.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.3.2 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on lucid-nonsense.infracaninophile.co.uk X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussions about the use of FreeBSD-current <freebsd-current.freebsd.org> List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current>; List-Post: <mailto:freebsd-current@freebsd.org> List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help> List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>, <mailto:freebsd-current-request@freebsd.org?subject=subscribe> X-List-Received-Date: Sat, 02 Nov 2013 10:51:08 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable On 02/11/2013 10:15, Matthias Andree wrote: > I understand from Eric's pist that the issue is that through his > limiting proxies, the SRV are not available at all so he does not even > get to the point where he could get the pkgN.nyi.freebsd.org > <http://pkgN.nyi.freebsd.org>; name back. That doesn't make sense. All the DNS SRV lookups on pkg.freebsd.org are done internally to pkg(8), which then issues an HTTP GET to the specific mirror selected by its internal algorithms. The web cache won't see literal 'pkg.freebsd.org' anywhere in the HTTP traffic -- as far as it is concerned, it's a simple HTTP request to a specific mirror 'pkg1.nyi.freebsd.org', and can be cached using the usual processes. What makes it cache unfriendly is that as far as the web cache is concerned each of the different mirrors appears to be completely independent of the others. So at the moment the chance of getting a cache hit is reduced by a factor of three because of the traffic distribution across the three mirrors. Cheers, Matthew --=20 Dr Matthew J Seaman MA, D.Phil. PGP: http://www.infracaninophile.co.uk/pgpkey --u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG/MacGPG2 v2.0.20 (Darwin) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQJ8BAEBCgBmBQJSdNkWXxSAAAAAAC4AKGlzc3Vlci1mcHJAbm90YXRpb25zLm9w ZW5wZ3AuZmlmdGhob3JzZW1hbi5uZXQ2NTNBNjhCOTEzQTRFNkNGM0UxRTEzMjZC QjIzQUY1MThFMUE0MDEzAAoJELsjr1GOGkATgCUP/2Ir45H/CTYBxADUYWNCQp74 2mfCvUdmMfbWMOVooKee7MWYknrx5oVvHgHGcR8Gqge+IylhbotYvjL2ahR+Hyh8 y9bm6liUhYJTp5ACqJistduwjiFmLaQ7+OPewjOOXk5vssQzLyejAZFj4TOZGCE5 8FVTUgfGf//C9NGTEtRxRmLn/V32NWsJiyBGf4mz9CpJglJGeUJeMq7VTeM9AmIY F2n80K32FqcLWp26rPWjjuRHoh7T4c9FuEqcbw5Z2vpiJLTd0drg4kBt9bcU3gry 2pfDRQOq8K8BJ4jGPbkvsjkmp4wyttKd3YO2Gd3hocP3pKRzsGIhPPaiBVGq1QcE nlmF4IB0ntNu1OSa/X6mFNZsnKIUPlPgCucnIobaCZwxS2ScfiITl4XvF9PxExx5 VQyZ2/jhL3Rz3X4AS++CrOwZgA+zxbmyVT4Zvc+y2m5+JjcEVZu8d0ARWlNlUoER swE1FclEBqIaMPNoofitihrtuBcefaAz5ECBCTVG8JFPxWV1BeZiWkBDegSA5Gez UoRSyLh/ymRnio6shbsmrYjnM8/Jv4juXlUG5MVT/IICwlZJBei7Q4c0K7fMtgvP NqBQB+K8R3Sr16a0HPOhIw1+eoNSA62B51UP4qbC1tHuvTTOHp4b85ngOBMl5UXs 5BHVJ+aRPCnBKevemUpR =1WKR -----END PGP SIGNATURE----- --u2MIcrVwJlrnrpbUEJMl7a9VUWWCqAT4b--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1680682c-dc77-4ee3-8e59-ee7356f307a3>