From owner-freebsd-ports@FreeBSD.ORG Sat Oct 21 07:12:21 2006 Return-Path: X-Original-To: freebsd-ports@freebsd.org Delivered-To: freebsd-ports@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 967BC16A403 for ; Sat, 21 Oct 2006 07:12:21 +0000 (UTC) (envelope-from uspoerlein@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.168]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2C14843D5F for ; Sat, 21 Oct 2006 07:12:19 +0000 (GMT) (envelope-from uspoerlein@gmail.com) Received: by ug-out-1314.google.com with SMTP id m2so830955uge for ; Sat, 21 Oct 2006 00:12:19 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:date:from:to:cc:subject:message-id:mail-followup-to:references:mime-version:content-type:content-disposition:in-reply-to; b=PHcJqMjNAvOiEkfKdzu9ODllZzzi+0Bcj0PGhYZ9KAqrUGNlYw5RT+JXhHMKeQcCOalY7/eSzGQs3RqLAHJPobUBBSJDgqj6WCmdMfA+elfpCbFinjGqCO7LWGfh0c7NU95xYfBF80u2fOADh4F6tlqOQXJh7zWPgBOro5qDy8c= Received: by 10.66.222.9 with SMTP id u9mr3401844ugg; Sat, 21 Oct 2006 00:12:19 -0700 (PDT) Received: from roadrunner.q.local ( [85.180.161.199]) by mx.google.com with ESMTP id q40sm2313356ugc.2006.10.21.00.12.19; Sat, 21 Oct 2006 00:12:19 -0700 (PDT) Received: from roadrunner.q.local (localhost [127.0.0.1]) by roadrunner.q.local (8.13.8/8.13.8) with ESMTP id k9KLenNf018979; Fri, 20 Oct 2006 23:40:49 +0200 (CEST) (envelope-from uspoerlein@gmail.com) Received: (from q@localhost) by roadrunner.q.local (8.13.8/8.13.8/Submit) id k9KLemlN018978; Fri, 20 Oct 2006 23:40:49 +0200 (CEST) (envelope-from uspoerlein@gmail.com) Date: Fri, 20 Oct 2006 23:40:48 +0200 From: Ulrich Spoerlein To: Doug Barton Message-ID: <20061020214048.GA11663@roadrunner.q.local> Mail-Followup-To: Doug Barton , Benjamin Lutz , freebsd-ports@freebsd.org References: <200610191124.39379.mail@maxlor.com> <45386B99.8080501@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45386B99.8080501@FreeBSD.org> Cc: Benjamin Lutz , freebsd-ports@FreeBSD.org Subject: Re: Parallel Builds X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 21 Oct 2006 07:12:21 -0000 Doug Barton wrote: > > o Leave the ports framework as it is, and implement support for > > parallel building in add-on tool, eg., portupgrade. The tool would > > support automatic parallelism ("portupgrade -a" would automatically > > build ports in parallel where possible), or having several > > user-created instances running at the same time. I'll call this > > "tool-based macro-parallelism". > > I specifically designed portmaster to be able to do this. Separate instances do not share > anything in common, other than the ports tree and package database. Assuming that you were > updating two (or more) ports that you were positive did not share any dependencies in common, > you could run as many portmaster instances as you wanted to, and not have any problems. > > Where this gets really sticky is in (for example) the -a case. If you run 'portmaster -l' on > a typical system, you'll see that there are some root ports (no dependencies, not depended > on), some leaf ports (not depended on), but most of the installed ports are in the "middle," > where they are depended on by something else, and most of those have dependencies of their > own. It would be a SMOP to have portmaster track the ports that are "safe" to upgrade in > parallel. Then of course you have to keep track of your child processes, create a queue for > what ports to run next, etc. etc. It's all doable of course, but I think the return on > investment for this project would be very small. Patches are always welcome however. :) > > > Disadvantages: > > - Moderately difficult to implement. > > *cough* I have a small script in production, which grabs all dependant ports to compile and does a serial package build, where all required packages are built/installed prior to the next package being built. With some minor additions, I *think* I could make those parallel package builds work without too much fuss. Package upgrade a something different though, but the whole ports framework is flawed in that area anyway ('make package' should come *before* 'make install'). Don't hold your breath though, ENOTIME as always ... Ulrich Spoerlein -- A: Yes. >Q: Are you sure? > >A: Because it reverses the logical flow of conversation. > >>Q: Why is top posting frowned upon?