Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 21 Dec 2000 16:19:02 +0200
From:      Maxim Sobolev <sobomax@FreeBSD.org>
To:        Andrew Reilly <areilly@bigpond.net.au>, freebsd-ports@FreeBSD.org
Subject:   Re: Integration of ports and 3rd party anoncvs repositories?
Message-ID:  <3A421156.147EA402@FreeBSD.org>
References:  <20001221103408.A76507@gurney.reilly.home> <3A420FEC.F9A719D9@FreeBSD.org>

next in thread | previous in thread | raw e-mail | index | archive | help
Maxim Sobolev wrote:

> Andrew Reilly wrote:
>
> > There are some large and fairly rapidly evolving code bases out
> > there at the moment.  They aren't part of the base FreeBSD
> > distribution, but are frequently installed via the ports
> > collection:  XFree86, Wine, mozilla, kde and gnome, probably
> > openoffice soon.  All of these are available through incremental
> > means: anoncvs, CVSup, or inter-tarball diffs.
> >
> > Please correct me if I'm wrong here, but the current Ports
> > facility is based on the notion of operating from distribution
> > tarballs that wind up in /usr/ports/distfiles, one way or
> > another.  Some of these tarballs are now really big, which (for
> > those of us who pay for our bandwidth by the megabyte) is a
> > disincentive for staying current.
> >
> > I've managed to track Wine for a while by building my own
> > tarballs incrementally, with the deltas.  I'm just about to have
> > a go at grabbing XFree86-4.0.2 by CVSup.
> >
> > Has anyone been thinking of tweaking the ports "extract" target
> > to copy from a local copy of the original repository, rather
> > than going straight for a tarball file?
> >
> > How could we standardise access to source repositories from
> > different vendors, so that the ports makefiles could determine
> > if they were present automagically?
> >
> > Would it be best to go for full local CVS repositories, and have
> > the "extract" target do a cvs co, or could we get by with local
> > "checked-out" trees?  (I haven't really used CVS myself yet: I
> > follow FreeBSD-stable with CVSup in "check out" mode.)
>
> If I understand you correctly, you are proposing to keep extracted from
> distfiles stuff in some sort of anoncvs repository, which will simplify
> incremental updates of 3rd party software for the users with limited bandwidth
> available, so for example when upgrading from the XFree4.0.1 to XFree4.0.2 the
> user will be saved from the necessity to d/l the whole 40-some megabytes
> archive, and will need to only update his existing XFree86 sources using `cvs
> update' command. While this idea in general sounds good, but I think that the
> cost involved for keeping code of each of our 4200+ ports in cvs is just too
> high (remember, this repo will be growing with insane speed with each update of
> underlying software).
>
> The another possibility that I'm thinking about now is to set up a service
> which at a specific request will generate xdelta(1) patch for any two arbitrary
> versions of distfiles (for example bzip'ed version of xdelta-generated diff
> between XFree4.0.1 and XFree4.0.2 distfiles is only 3MB long). Then the user
> will be able to download only the difference between two versions in question
> and apply diff to his local existing previous versions of software. Popular
> diffs could be cached at the "server", as the creating xdelta diffs for large
> files is io/cpu consuming task. The only problem with this schema is that the
> xdelta(1) can't reconstruct gzipped tarball in such a way that the md5 sum for
> original archive and md5 sum for reconstructed will match (however checksums of
> underlying tar files will be equial), so we need some form of a workaround here
> (for example use two md5 entries for each file in distinfo - gzip'ed and
> gunzip'ed).
>
> As I said it's only the idea right now, but I'm close to starting implementing
> something like that because internet costs are insanely high here ;).

Forgot to mention that xdelta(1) is a tool for generating diffs between binary
files, it's available from ports/misc/xdelta.

-Maxim



To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-ports" in the body of the message




Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A421156.147EA402>