Date: Fri, 20 Oct 2006 23:40:48 +0200 From: Ulrich Spoerlein <uspoerlein@gmail.com> To: Doug Barton <dougb@FreeBSD.org> Cc: Benjamin Lutz <mail@maxlor.com>, freebsd-ports@FreeBSD.org Subject: Re: Parallel Builds Message-ID: <20061020214048.GA11663@roadrunner.q.local> In-Reply-To: <45386B99.8080501@FreeBSD.org> References: <200610191124.39379.mail@maxlor.com> <45386B99.8080501@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Doug Barton wrote:
> > o Leave the ports framework as it is, and implement support for
> > parallel building in add-on tool, eg., portupgrade. The tool would
> > support automatic parallelism ("portupgrade -a" would automatically
> > build ports in parallel where possible), or having several
> > user-created instances running at the same time. I'll call this
> > "tool-based macro-parallelism".
>
> I specifically designed portmaster to be able to do this. Separate instances do not share
> anything in common, other than the ports tree and package database. Assuming that you were
> updating two (or more) ports that you were positive did not share any dependencies in common,
> you could run as many portmaster instances as you wanted to, and not have any problems.
>
> Where this gets really sticky is in (for example) the -a case. If you run 'portmaster -l' on
> a typical system, you'll see that there are some root ports (no dependencies, not depended
> on), some leaf ports (not depended on), but most of the installed ports are in the "middle,"
> where they are depended on by something else, and most of those have dependencies of their
> own. It would be a SMOP to have portmaster track the ports that are "safe" to upgrade in
> parallel. Then of course you have to keep track of your child processes, create a queue for
> what ports to run next, etc. etc. It's all doable of course, but I think the return on
> investment for this project would be very small. Patches are always welcome however. :)
>
> > Disadvantages:
> > - Moderately difficult to implement.
>
> *cough*
I have a small script in production, which grabs all dependant ports to
compile and does a serial package build, where all required packages are
built/installed prior to the next package being built.
With some minor additions, I *think* I could make those parallel package
builds work without too much fuss.
Package upgrade a something different though, but the whole ports
framework is flawed in that area anyway ('make package' should come
*before* 'make install').
Don't hold your breath though, ENOTIME as always ...
Ulrich Spoerlein
--
A: Yes.
>Q: Are you sure?
> >A: Because it reverses the logical flow of conversation.
> >>Q: Why is top posting frowned upon?
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20061020214048.GA11663>