From owner-freebsd-hubs@FreeBSD.ORG Tue Apr 8 14:16:02 2008 Return-Path: Delivered-To: hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 332031065741; Tue, 8 Apr 2008 14:16:02 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (cl-162.ewr-01.us.sixxs.net [IPv6:2001:4830:1200:a1::2]) by mx1.freebsd.org (Postfix) with ESMTP id A3C498FC2A; Tue, 8 Apr 2008 14:16:01 +0000 (UTC) (envelope-from brooks@lor.one-eyed-alien.net) Received: from lor.one-eyed-alien.net (localhost [127.0.0.1]) by lor.one-eyed-alien.net (8.14.2/8.14.2) with ESMTP id m38EG4Un053821; Tue, 8 Apr 2008 09:16:04 -0500 (CDT) (envelope-from brooks@lor.one-eyed-alien.net) Received: (from brooks@localhost) by lor.one-eyed-alien.net (8.14.2/8.14.2/Submit) id m38EG4Vi053820; Tue, 8 Apr 2008 09:16:04 -0500 (CDT) (envelope-from brooks) Date: Tue, 8 Apr 2008 09:16:04 -0500 From: Brooks Davis To: Pav Lucistnik Message-ID: <20080408141604.GA53370@lor.one-eyed-alien.net> References: <1207605059.1031.38.camel@ikaros.oook.cz> <20080408002005.W15502@fledge.watson.org> <1207642548.84150.10.camel@pav.hide.vol.cz> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="h31gzZEtNLTqOjlF" Content-Disposition: inline In-Reply-To: <1207642548.84150.10.camel@pav.hide.vol.cz> User-Agent: Mutt/1.5.17 (2007-11-01) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (lor.one-eyed-alien.net [127.0.0.1]); Tue, 08 Apr 2008 09:16:04 -0500 (CDT) Cc: hubs@freebsd.org Subject: Re: package distribution crisis - CDN needed X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2008 14:16:02 -0000 --h31gzZEtNLTqOjlF Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Apr 08, 2008 at 10:15:48AM +0200, Pav Lucistnik wrote: > Eberhard Moenkeberg p??e v ?t 08. 04. 2008 v 01:54 +0200: >=20 > > > > I do propose a creation of a CDN (Content Delivery Network), having= these > > > > features: > > >=20 > > > The success of a CDN is premised on the working set of active files b= eing > > > [significantly] smaller than the set of available files. While I don= 't doubt > > > that only a few packages are of interest to most users, for a CDN to = provide a > > > real benefit it needs to be the case that no users require most files= =2E The > > > reason being that, if you have to transfer all or even most files eve= ry time, > > > a CDN may well be a significantly less efficient way to do it than rs= ync. Has > > > a working set analysis for our main mirrors been done that demonstrat= es that > > > the actual working set is sufficiently smaller than the total set tha= t a CDN > > > would be more, rather than less, efficient? >=20 > I don't have these statistics, but I'd like to see them very much. > Can I ask some of the mirror admins to compile me statistics on > downloads of files from /pub/FreeBSD/ports/*packages*, say for March > 2008 ? >=20 > I'd also like to have these numbers from ftp.FreeBSD.org so I have a > comparision of volume. >=20 > My suspicion is that 99% of users get their packages from > ftp.FreeBSD.org, because that's what pkg_add -r uses without PACKAGEROOT > specification. I suspect that like virtually everything else on the internet package access is heavy-tailed. If so, no from of cooperative caching will be of use. For example, this paper killed all research on cooperative caching web proxies by demostrating that its was fundamentally pointless: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=3D10.1.1.22.9583 -- Brooks --h31gzZEtNLTqOjlF Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.8 (FreeBSD) iD8DBQFH+34jXY6L6fI4GtQRAhwDAJ9h3df8cg+ZxEsesdQIJJl4xJC5qACfTO8s eR4JbbyHsdmJTCF9Qr18jec= =kmyi -----END PGP SIGNATURE----- --h31gzZEtNLTqOjlF--