From owner-freebsd-hubs@FreeBSD.ORG Wed Apr 9 07:35:30 2008 Return-Path: Delivered-To: freebsd-hubs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 35322106567A for ; Wed, 9 Apr 2008 07:35:30 +0000 (UTC) (envelope-from anthonybryan@gmail.com) Received: from hs-out-0708.google.com (hs-out-0708.google.com [64.233.178.240]) by mx1.freebsd.org (Postfix) with ESMTP id E49128FC1D for ; Wed, 9 Apr 2008 07:35:29 +0000 (UTC) (envelope-from anthonybryan@gmail.com) Received: by hs-out-0708.google.com with SMTP id m63so1820550hsc.11 for ; Wed, 09 Apr 2008 00:35:29 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; bh=Id25dxhvZmO0+4XK0Z+R4mpuKelTLNbFQOfHr3SKO0A=; b=fsqClni/iHaPHukyNXrlRo72cz7JjBpsxdMAIemsjU/nS4WebiszM6OBdXS8+19QOO5Vq9vc8chZBNXzz0rRpKDU4TF7kFbuy665PsVcFfJKaODUTwY39n/igQ+IcD0aJhA/vQEzwH+nL27spF7p4o0jV97B7wkP6YG3H2iWCUM= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=drP3I/j840VJ1m2/mPgwwJy5c5DZpz5vy+PEquE/03b4ntXs0ErzHt7394v3tdl7/72DOKflo11aWvD43FEfSy/yH9H85U1jUrOuynD9Fb4uGULpb8oyygtwdiuZ23zjkj3ECRTNfJ/VrooUYSMhdpwIRwibREXdOakyMZQ98Ok= Received: by 10.141.203.7 with SMTP id f7mr2527160rvq.7.1207724815628; Wed, 09 Apr 2008 00:06:55 -0700 (PDT) Received: by 10.141.48.20 with HTTP; Wed, 9 Apr 2008 00:06:55 -0700 (PDT) Message-ID: Date: Wed, 9 Apr 2008 03:06:55 -0400 From: "Anthony Bryan" To: freebsd-hubs@freebsd.org, dimitar.vassilev@gmail.com MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Cc: Subject: Re: package distribution crisis - CDN needed X-BeenThere: freebsd-hubs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "FreeBSD Distributions Hubs: mail sup ftp" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Apr 2008 07:35:30 -0000 I hope you will investigate the MirrorBrain mentioned earlier and in the presentation at http://www.poeml.de/~poeml/talks/redirector/ to see if it meets your needs. obviously, it works well for openSUSE but will function just as well for other projects in similar circumstances. the author will be receptive to FreeBSD (& other organization's) needs as he is actively improving it for everyone, not just openSUSE. http://www.mirrorbrain.org/ replies to other messages follow: Francois Petillon wrote: > Dimitar Vasilev wrote: > > Once upon a time a guy suggested that all FreeBSD mirrors join > > http://www.metalinker.org/, but someone from the French mirrors rejecte= d the > > idea. > > The frenchy did not reject the idea, the frenchy just told segmented > downloads is fucking up IOs on big servers. And metalinker is mainly > aimed at multisources segmented downloads. ahh, Fran=E7ois, my dear colleague, you wound me! Metalink is mainly aimed at providing mirror lists and checksums for automated integrity verification and error repair. If you want to completely disable segmented downloads in metalinks, there is that option in the .metalink. a 2006 reply https://lists.ubuntu.com/archives/ubuntu-mirrors/2007-February= /000042.html Thanks for bringing this up! These are all excellent reasons for using metalink - by putting your preferences into a machine readable format that courteous users can take advantage of. (Along with configuring a server correctly and to your liking). Metalink isn't focused on segmented downloading, although many of the clients do support that, some of them don't. The increased availability from multiple links, integrity/reliability from checksums, being able to add multiple files to a transfer queue and other automated features are also found useful by others. Ubuntu, OpenOffice.org, or other organizations can choose to limit (in the metalink, according to the spec) the number of connections used for downloading a file, or for specific servers. That is, you can say in the metalink that you don't want programs doing segmented downloading, or restrict them to a certain number of connections to a server. Of course, servers still need to be correctly configured because, as you say, other programs will not respect your wishes and use up resources. Dimitar Vassilev, I'm sorry I didn't reply to your message from Nov 2006 ( http://lists.freebsd.org/mailman/htdig/freebsd-hubs/2006-November/00= 1544.html ), I didn't see it until now or I would have replied. I believe some of the server side metalink generators do what you are asking about. --=20 (( Anthony Bryan ... Metalink [ http://www.metalinker.org ] )) Easier, More Reliable, Self Healing Downloads