Date: Mon, 10 Mar 2003 11:25:25 +1000 (EST) From: jason andrade <jason@rtfmconsult.com> To: David O'Brien <obrien@FreeBSD.ORG> Cc: hubs@FreeBSD.ORG Subject: Re: Poor state of some top-level FTP mirrors Message-ID: <Pine.GSO.4.50.0303101116080.41-100000@luna.rtfmconsult.com> In-Reply-To: <20030309215448.GB30033@dragon.nuxi.com> References: <20030309215448.GB30033@dragon.nuxi.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 9 Mar 2003, David O'Brien wrote: > In my never ending quest to find an alternate ftp site for "pkg_add -r", > I've tried many of our ftpX.freebsd.org mirrors. The problem is the > mirrors only seem to be consistent for short periods of time. > > As a simple test, the documented mirrors was polled for the file > ftp://${SITE}/pub/FreeBSD/ports/i386/packages-4-stable/All/bonnie++-1.93.03.tgz > which was updated on ftp.freebsd.org 2-March-2003. the methodology of the way the packages tree is set out makes it very difficult to provide a stable and uptodate mirror. the freebsd archive is one of the few that has a system which causes an entire package tree to be rebuilt (and hence refetched) over and over again with no ability to only fetch updated or changed packages and the packages trees make up the bulk of the freebsd archive. keeping that in mind, as well as the load this puts on bandwidth (how many freebsd mirrors are there ? say 135 from below's stats) where at 6G for this tree, that is almost a Terabyte of data that is pushed out in various directions almost weekly for one tree. would anyone say that there is even half a terabyte of (compressed) code that is changing weekly ? or monthly.. i'd be happy to be proven wrong on this but i suspect it is one of the major issues behind the impossibility of keeping an up to date mirror of the packages tree, and this is made worse by having multiple architectures and releases now, e.g packages-4-stable, packages-5-current for i386, alpha, and eventually sparc64 and others. here at ftp.au.freebsd.org i try and keep the packages-4-stable tree updated once a week to once a fortnight. it still takes most of a day - sometimes 3 or 4 days, to update one tree. regards, -jason To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hubs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.GSO.4.50.0303101116080.41-100000>