From owner-freebsd-hubs@FreeBSD.ORG Tue Apr 8 00:41:16 2008 Return-Path: Delivered-To: hubs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AF091065675 for ; Tue, 8 Apr 2008 00:41:16 +0000 (UTC) (envelope-from emoenke@gwdg.de) Received: from mailer.gwdg.de (mailer.gwdg.de [134.76.10.26]) by mx1.freebsd.org (Postfix) with ESMTP id 0A51F8FC19 for ; Tue, 8 Apr 2008 00:41:15 +0000 (UTC) (envelope-from emoenke@gwdg.de) Received: from gwdu05.gwdg.de ([134.76.8.5]) by mailer.gwdg.de with esmtps (TLSv1:AES256-SHA:256) (Exim 4.68) (envelope-from ) id 1Jj1Ah-0002Bd-O7 for hubs@FreeBSD.org; Tue, 08 Apr 2008 01:54:43 +0200 Date: Tue, 8 Apr 2008 01:54:43 +0200 (CEST) From: Eberhard Moenkeberg To: hubs@FreeBSD.org In-Reply-To: <20080408002005.W15502@fledge.watson.org> Message-ID: References: <1207605059.1031.38.camel@ikaros.oook.cz> <20080408002005.W15502@fledge.watson.org> MIME-Version: 1.0 Content-Type: MULTIPART/MIXED; BOUNDARY="1283851528-1837400255-1207612483=:10070" X-Spam-Level: - X-Virus-Scanned: (clean) by exiscan+sophie Cc: Subject: Re: package distribution crisis - CDN needed X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Apr 2008 00:41:16 -0000 This message is in MIME format. The first part should be readable text, while the remaining parts are likely unreadable without MIME-aware tools. --1283851528-1837400255-1207612483=:10070 Content-Type: TEXT/PLAIN; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Hi, On Tue, 8 Apr 2008, Robert Watson wrote: > On Mon, 7 Apr 2008, Pav Lucistnik wrote: =20 > > Okay the situation recently was that the mirrors had no chance keeping = up > > with all the package sets I've been uploading to ftp-master. > > > > We clearly need to move beyond rsync/cvsup synced ftp mirrors. This doe= s not > > scale. > > > > I do propose a creation of a CDN (Content Delivery Network), having the= se > > features: >=20 > The success of a CDN is premised on the working set of active files being > [significantly] smaller than the set of available files. While I don't d= oubt > that only a few packages are of interest to most users, for a CDN to prov= ide a > real benefit it needs to be the case that no users require most files. T= he > reason being that, if you have to transfer all or even most files every t= ime, > a CDN may well be a significantly less efficient way to do it than rsync.= Has > a working set analysis for our main mirrors been done that demonstrates t= hat > the actual working set is sufficiently smaller than the total set that a = CDN > would be more, rather than less, efficient? Maybe it is interesting for you to have a look at=20 http://www.poeml.de/~poeml/talks/redirector/ The SUSE team has a redirection mechanism at work on single file basis,=20 and they also have experienced that it would be "enough" for the users if= =20 there are lots of mirrors with the "most requested" files plus some=20 mirrors with all files. It is all GPL, so you (even you) should be able to use the software. Viele Gr=FC=DFe Eberhard M=F6nkeberg (emoenke@gwdg.de, em@kki.org) --1283851528-1837400255-1207612483=:10070--