Date: Wed, 10 Mar 2004 22:08:49 +0200 From: Ion-Mihai Tetcu <itetcu@apropo.ro> To: Brooks Davis <brooks@one-eyed-alien.net> Cc: rofug@rofug.ro Subject: Re: [RFC] suport for fetching from local mirrors Message-ID: <20040310220849.78247c5b@it.buh.cameradicommercio.ro> In-Reply-To: <20040310183555.GC14892@Odin.AC.HMC.Edu> References: <20040310190422.43ac46c9@it.buh.cameradicommercio.ro> <20040310183555.GC14892@Odin.AC.HMC.Edu>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 10 Mar 2004 10:35:55 -0800
Brooks Davis <brooks@one-eyed-alien.net> wrote:
> On Wed, Mar 10, 2004 at 07:04:22PM +0200, Ion-Mihai Tetcu wrote:
> > Hi,
> >
> >
> > I wonder if there is any way to convince globally ``make fetch'' to try
> > use first other site that those in bsd.sites.mk (not with
> > MASTER_SITE_OVERRIDE, but overriding the master sites individually).
> >
> >
> > The reason for this (at least for .ro) is:
> >
> > 1. the vast majority of the sites (apache, oo, *linux) have MAN mirrors
> > 2. many of them only partially mirror the master site
> > 3. or have a slightly different directory structure
> >
> > This sites could be added to bsd.sites.mk, but:
> > 4. because of 2. and 3. above it would be a great idea
^^^^^^^^ = wouldn't
> > 5 and the majority of mirrors admins wouldn't like abroad downloaders as
> > the international bandwidth costs are very big . (10 to 50 compared to
> > us for example)
> >
> > The benefits would be:
> > 6. unloading master sites and internet
> > 7. faster download speed for users (and on large distfiles, like OO,
> > kde, etc. this would make a big difference esp. for "home" users with
> > low speed internet access).
> >
> > This can be easily achieved by including a ``bsd.local_sites.mk'' in
> > bsd.ports.mk above line 2158 where bsd.sites.mk is included.
> > Something like:
> >
> > # Local (MAN) master sites
> > .if exists(${PORTSDIR}/Mk/bsd.local_sites.mk)
> > .include "${PORTSDIR}/Mk/bsd.local_sites.mk"
> > .endif
> >
> > The user will be responsible for creating and populating the file.
>
> My first thought was that this was overkill, but upon further
> reflection, I like it.
It took me about 3-4 hours to produce such a file last year. Of course
it wasn't very exhaustive, but it covered about 63% of the total size of
the distfiles for i386.
> I'm working toward doing most updates on systems
> at work via read-only nfs access to checked out copies of the ports
> tree, and this would let me eliminate about half of the /etc/make.conf
> configuration I use by having the ports collection contain the files
> that redirect to our local mirrors.
I have three cases:
1. Our local machines: the hole thing is very simple, as one of them is
cvsup-ing and fetching each 4th hour, and it's used as
MASTER_SITE_OVERRIDE for the others (being in a LAN this setup is just
easy and fast), on which the normal make install clean sequence also has
distclean to save space;
2. Our customers: some use our server - of course, when more of them
fetch in the same time the throughput drops drastically and maintaining
access rules is becoming to much time consuming, not to add that
maintaining a few versions for the distfiles is consuming space that
could be otherwise better use.
3. Our friends: being home users with lousy and / or expensive
international connections - they suffer the most.
> > Eventually an option for make.conf could be added, like:
> > LOCAL_FETCH_SITES= cc
> > where cc would be the country code.
> >
> > # Local (MAN) master sites
> > .if defined(LOCAL_FETCH_SITES)
> > .if exists(${PORTSDIR}/Mk/bsd.sites_${LOCAL_FETCH_SITES}.mk)
> > .include "${PORTSDIR}/Mk/bsd.sites_${LOCAL_FETCH_SITES}.mk"
> > .endif
> > .endif
> >
> > The _cc files could be maintained by local user groups or something (I
> > would volunteer for _ro).
>
> This seems like a reasonable idea to me.
Then perhaps _cc file plus local_sites, entries from local_sites being
on top (as they would be defined by the user - this would cover in your
case and some of my second case).
--
IOnut
Unregistered ;) FreeBSD user
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040310220849.78247c5b>