Date: Thu, 21 Dec 2000 16:13:01 +0200 From: Maxim Sobolev <sobomax@FreeBSD.org> To: Andrew Reilly <areilly@bigpond.net.au> Cc: freebsd-ports@FreeBSD.org Subject: Re: Integration of ports and 3rd party anoncvs repositories? Message-ID: <3A420FEC.F9A719D9@FreeBSD.org> References: <20001221103408.A76507@gurney.reilly.home>
next in thread | previous in thread | raw e-mail | index | archive | help
Andrew Reilly wrote: > There are some large and fairly rapidly evolving code bases out > there at the moment. They aren't part of the base FreeBSD > distribution, but are frequently installed via the ports > collection: XFree86, Wine, mozilla, kde and gnome, probably > openoffice soon. All of these are available through incremental > means: anoncvs, CVSup, or inter-tarball diffs. > > Please correct me if I'm wrong here, but the current Ports > facility is based on the notion of operating from distribution > tarballs that wind up in /usr/ports/distfiles, one way or > another. Some of these tarballs are now really big, which (for > those of us who pay for our bandwidth by the megabyte) is a > disincentive for staying current. > > I've managed to track Wine for a while by building my own > tarballs incrementally, with the deltas. I'm just about to have > a go at grabbing XFree86-4.0.2 by CVSup. > > Has anyone been thinking of tweaking the ports "extract" target > to copy from a local copy of the original repository, rather > than going straight for a tarball file? > > How could we standardise access to source repositories from > different vendors, so that the ports makefiles could determine > if they were present automagically? > > Would it be best to go for full local CVS repositories, and have > the "extract" target do a cvs co, or could we get by with local > "checked-out" trees? (I haven't really used CVS myself yet: I > follow FreeBSD-stable with CVSup in "check out" mode.) If I understand you correctly, you are proposing to keep extracted from distfiles stuff in some sort of anoncvs repository, which will simplify incremental updates of 3rd party software for the users with limited bandwidth available, so for example when upgrading from the XFree4.0.1 to XFree4.0.2 the user will be saved from the necessity to d/l the whole 40-some megabytes archive, and will need to only update his existing XFree86 sources using `cvs update' command. While this idea in general sounds good, but I think that the cost involved for keeping code of each of our 4200+ ports in cvs is just too high (remember, this repo will be growing with insane speed with each update of underlying software). The another possibility that I'm thinking about now is to set up a service which at a specific request will generate xdelta(1) patch for any two arbitrary versions of distfiles (for example bzip'ed version of xdelta-generated diff between XFree4.0.1 and XFree4.0.2 distfiles is only 3MB long). Then the user will be able to download only the difference between two versions in question and apply diff to his local existing previous versions of software. Popular diffs could be cached at the "server", as the creating xdelta diffs for large files is io/cpu consuming task. The only problem with this schema is that the xdelta(1) can't reconstruct gzipped tarball in such a way that the md5 sum for original archive and md5 sum for reconstructed will match (however checksums of underlying tar files will be equial), so we need some form of a workaround here (for example use two md5 entries for each file in distinfo - gzip'ed and gunzip'ed). As I said it's only the idea right now, but I'm close to starting implementing something like that because internet costs are insanely high here ;). -Maxim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A420FEC.F9A719D9>