Date: Sun, 19 Oct 2003 17:53:24 +0200 From: Marc Espie <espie@nerim.net> To: tech-pkg@netbsd.org, ports@openbsd.org, freebsd-ports@freebsd.org Subject: Unification of distfiles Message-ID: <20031019155324.GA28256@tetto.gentiane.org>
next in thread | raw e-mail | index | archive | help
I think trying to unify distfiles somehow is a great idea. If I remember right, both Free and Net have mechanisms similar to DIST_SUBDIR, so it just means that we ought to try to have similar naming there. I'm all in favor for greater communication in that area, and possibly people advertizing distfiles mirrors that would hold stuff for all three BSDs. Now, for stuff that exists over in OpenBSD land, that might be of interest to other people. One other issue which has been fairly annoying as far as OpenBSD was concerned, is distfiles changing checksums without changing names. We've had a fairly thorough mechanism for dealing with this in place for a long while, look at the code handling REFETCH in our bsd.port.mk, plus the /usr/ports/infrastructure/fetch/link-checksums script. Basically, every mirror should run this once in a while. This establishes alternate access methods for files, based on md5/sha1/rmd160 checksums. The price to pay is that the mirror ought to be located on a system with unix filesystem semantics (and preferably a single partition) to avoid duplication of files. We also have a `mirror-maker' target that knows how to create a master Makefile in /usr/ports/distfiles, that is 10000+ lines long, and can be used to do automated operations on all distfiles (fetching them, checking checksums, verifying mirrors). The fact that it is a Makefile makes it very easy to run operations in parallel and diminish network bandwidth bottlenecks for grabbing a file. I've been very happy with this redesign of our infrastructure (note that the Makefile is portable enough that Bob Beck actually runs it under Solaris on a sunsite for a distfiles mirror), and I believe it could be useful to other people.... Feel free to ignore this, if you feel offended by the idea that OpenBSD may have done something right.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20031019155324.GA28256>