Date: Thu, 21 Dec 2000 16:19:02 +0200 From: Maxim Sobolev <sobomax@FreeBSD.org> To: Andrew Reilly <areilly@bigpond.net.au>, freebsd-ports@FreeBSD.org Subject: Re: Integration of ports and 3rd party anoncvs repositories? Message-ID: <3A421156.147EA402@FreeBSD.org> References: <20001221103408.A76507@gurney.reilly.home> <3A420FEC.F9A719D9@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Maxim Sobolev wrote: > Andrew Reilly wrote: > > > There are some large and fairly rapidly evolving code bases out > > there at the moment. They aren't part of the base FreeBSD > > distribution, but are frequently installed via the ports > > collection: XFree86, Wine, mozilla, kde and gnome, probably > > openoffice soon. All of these are available through incremental > > means: anoncvs, CVSup, or inter-tarball diffs. > > > > Please correct me if I'm wrong here, but the current Ports > > facility is based on the notion of operating from distribution > > tarballs that wind up in /usr/ports/distfiles, one way or > > another. Some of these tarballs are now really big, which (for > > those of us who pay for our bandwidth by the megabyte) is a > > disincentive for staying current. > > > > I've managed to track Wine for a while by building my own > > tarballs incrementally, with the deltas. I'm just about to have > > a go at grabbing XFree86-4.0.2 by CVSup. > > > > Has anyone been thinking of tweaking the ports "extract" target > > to copy from a local copy of the original repository, rather > > than going straight for a tarball file? > > > > How could we standardise access to source repositories from > > different vendors, so that the ports makefiles could determine > > if they were present automagically? > > > > Would it be best to go for full local CVS repositories, and have > > the "extract" target do a cvs co, or could we get by with local > > "checked-out" trees? (I haven't really used CVS myself yet: I > > follow FreeBSD-stable with CVSup in "check out" mode.) > > If I understand you correctly, you are proposing to keep extracted from > distfiles stuff in some sort of anoncvs repository, which will simplify > incremental updates of 3rd party software for the users with limited bandwidth > available, so for example when upgrading from the XFree4.0.1 to XFree4.0.2 the > user will be saved from the necessity to d/l the whole 40-some megabytes > archive, and will need to only update his existing XFree86 sources using `cvs > update' command. While this idea in general sounds good, but I think that the > cost involved for keeping code of each of our 4200+ ports in cvs is just too > high (remember, this repo will be growing with insane speed with each update of > underlying software). > > The another possibility that I'm thinking about now is to set up a service > which at a specific request will generate xdelta(1) patch for any two arbitrary > versions of distfiles (for example bzip'ed version of xdelta-generated diff > between XFree4.0.1 and XFree4.0.2 distfiles is only 3MB long). Then the user > will be able to download only the difference between two versions in question > and apply diff to his local existing previous versions of software. Popular > diffs could be cached at the "server", as the creating xdelta diffs for large > files is io/cpu consuming task. The only problem with this schema is that the > xdelta(1) can't reconstruct gzipped tarball in such a way that the md5 sum for > original archive and md5 sum for reconstructed will match (however checksums of > underlying tar files will be equial), so we need some form of a workaround here > (for example use two md5 entries for each file in distinfo - gzip'ed and > gunzip'ed). > > As I said it's only the idea right now, but I'm close to starting implementing > something like that because internet costs are insanely high here ;). Forgot to mention that xdelta(1) is a tool for generating diffs between binary files, it's available from ports/misc/xdelta. -Maxim To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-ports" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3A421156.147EA402>