Date: Sat, 26 May 2001 23:24:49 -0700 From: Jordan Hubbard <jkh@osd.bsdi.com> To: jason@dstc.edu.au Cc: grisha@verio.net, jesper@skriver.dk, hubs@FreeBSD.ORG Subject: Re: final state of mirroring ? Message-ID: <20010526232449W.jkh@osd.bsdi.com> In-Reply-To: <Pine.OSF.4.20.0105271429100.7520-100000@azure.dstc.edu.au> References: <Pine.BSF.4.32.0105262334230.63070-100000@localhost> <Pine.OSF.4.20.0105271429100.7520-100000@azure.dstc.edu.au>
next in thread | previous in thread | raw e-mail | index | archive | help
From: jason andrade <jason@dstc.edu.au>
Subject: Re: final state of mirroring ?
Date: Sun, 27 May 2001 14:50:05 +1000 (EST)
> o ftp-master.freebsd.org is now the authoritative "master"
> site for tier 1 mirrors. where possible, tier 1 mirrors
> should switch to mirroring from ftp-master rather than
> ftp.freebsd.org
Correct. If they really want to continue to mirror from
ftp.freebsd.org then then can compete for the scarcer resources, but
there's no reason why a legitimate tier-1 mirror should have to.
> o a tier 1 mirror is defined as the main mirror for a given
> country (e.g ftp.uk.freebsd.org). it may also include
> other mirrors within that country in some circumstances
> (politics, bandwidth issues, whatever)
That and a willingness to maintain a _full_ mirror of ftp-master, yes.
> o a tier 2 mirror is defined as a non official mirror (not
> registered within the xx.freebsd.org namespace) or a mirror
> that updates from a tier 1 mirror. there is no "status"
> difference to the community that is served by a tier2 mirror
> except that they are *not* required to be a full mirror and
> may in fact carry a much more restricted subset of freebsd
> (e.g no alpha, or only the latest 2 releases)
Correct.
> o ftp.freebsd.org and ftp2.freebsd.org are now updating from
> ftp-master.freebsd.org (side pedantic node, can we also
> cname ftp-master to ftp0 and ftp to ftp1).
We'll probably have at least 5 machines updating from ftp-master in
the long run. I's also possible that "ftp.freebsd.org" will someday
stop pointing to a single site and instead become a round-robin DNS or
Inktomi/F5 load-balanced, least-cost routed mirror. We reserve the
right to do that with "ftp.freebsd.org" at any time, though it's also
something which would only happen if we could guarantee that
"ftp.freebsd.org" would repeatedly resolve to a site containing the
exact same bits each time.
> o tier1 mirrors should carry a *complete* mirror of ftp-master?
Yes.
> o ftp-master no longer appears to cleanup versions of files in
> the ports/distfiles area. this is to some extent annoying
That's manpower constrained right now. If we can get people to dive
in and start pruning/organizing this like it used to be, it will be.
Satoshi used to do it but it appears that he's being progressively
sidelined by Real Work(tm) nowadays.
> o the cleanup of the whole site has stalled a bit - parts have
> been cleaned up but there's still historical stuff lying around.
This is an ongoing process as time permits and will probably always
hold true for some value of "historical."
> o the questions of snapshots and archival status doesn't appear to
> have been resolved.
Snapshots will continue to be made at stable.freebsd.org and
current.freebsd.org for those respective branches. I won't be copying
them to ftp-master.freebsd.org due to the increased cost to the tier-1
mirrors and keeping just one snapshot around for a *day* (or until
it's outdated by the next successful snapshot) would be silly - it
would be gone before some of the slower mirrors could get it. I also
don't want to get into the argument of keeping n snapshots around
since for a reasonable value of n, you might just as well go to
${branch}.freebsd.org and have access to the last 30 or so.
As to archival stuff, nobody seems all that interested in hosting
those bits, at least nobody we could put into some sort of meaningful
and long-lived archival site taxonomy. One is currently just as well
served by using ftpsearch and finding one of the many "unofficial
archives" out there.
> o what are the total services offered by / can be offered by the freebsd
> project to end users ?
ftp and rsync seem to be about the most likely services that we can
offer and still have some form of security and sustainable "costs" in
terms of running each service. Not that even just these two are not
without cost. A running rsync invocation takes up a lot of memory,
as a small snapshot of ftp-master shows:
88649 ftp 2 0 13492K 13144K select 11:55 0.00% 0.00% rsync
90604 ftp 2 0 19016K 18468K select 8:48 0.00% 0.00% rsync
95678 ftp 2 0 19032K 18488K select 4:31 0.00% 0.00% rsync
95775 ftp 2 0 19032K 18488K select 3:08 0.00% 0.00% rsync
And that's about a third of the total rsync processes running.
I edited the list for brevity's sake.
http transfers are also pretty expensive, from what I hear, and the
security people get the wobblies at the idea of running apache on the
site as well. Who knows though - could happen.
- Jordan
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hubs" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010526232449W.jkh>