From owner-freebsd-ports@FreeBSD.ORG Thu Nov 22 20:09:27 2007 Return-Path: Delivered-To: freebsd-ports@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 89CEA16A418 for ; Thu, 22 Nov 2007 20:09:27 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: from mail2.fluidhosting.com (mx21.fluidhosting.com [204.14.89.4]) by mx1.freebsd.org (Postfix) with SMTP id 587CC13C4D1 for ; Thu, 22 Nov 2007 20:09:27 +0000 (UTC) (envelope-from dougb@FreeBSD.org) Received: (qmail 2181 invoked by uid 399); 22 Nov 2007 20:09:26 -0000 Received: from localhost (HELO lap.dougb.net) (dougb@dougbarton.us@127.0.0.1) by localhost with ESMTP; 22 Nov 2007 20:09:26 -0000 X-Originating-IP: 127.0.0.1 Message-ID: <4745E1F4.6030101@FreeBSD.org> Date: Thu, 22 Nov 2007 12:09:24 -0800 From: Doug Barton Organization: http://www.FreeBSD.org/ User-Agent: Thunderbird 2.0.0.9 (X11/20071119) MIME-Version: 1.0 To: "Jason C. Wells" References: <474262D5.3010603@highperformance.net> <47459C80.8000206@highperformance.net> <4745C795.5020101@FreeBSD.org> <4745D4E7.6090907@highperformance.net> In-Reply-To: <4745D4E7.6090907@highperformance.net> X-Enigmail-Version: 0.95.5 OpenPGP: id=D5B2F0FB Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: freebsd-ports@FreeBSD.org Subject: Re: Package Building in the Large X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 22 Nov 2007 20:09:27 -0000 Jason C. Wells wrote: > Most of the message below is just rounding out the discussion. There is > one significant question on recursion though. Sounds good, I'll snip the bits that don't require comment. > Doug Barton wrote: > >>> I also ended up with shared a library version problem >>> in at least one port (grip) in spite of having started my build with a >>> completely vacant /usr/local. >> That sounds like a problem with the ports. I can't think of any way >> that portmaster would cause that. > > It might be due to a non-clean /usr/local on the target install host. It > was a foo.so.3 is required but foo.so.2 is installed sort of error. I > haven't dealt with the installation step in a meaningful way yet. Don't > worry about responding to this bit. I think you're on the right track there. >>> I used 'portmaster -GgDt -p /usr/ports/ports $pmarg' for my run >>> where $pmarg was the list of 31 ports. >> That's definitely not going to work, for several reasons. The -G >> switch should only be used _after_ you've already run portmaster >> without it for a given port build. > > I had manually run 'make config-recursive' previously. I'll do as > advised at any rate. The config-recursive target does most of what portmaster does, but not all. Most significantly, it's not adaptive to changing an OPTION that results in a new dependency graph. What probably happened (and this is just a WAG) is that because portmaster didn't "see" some of the dependencies further down the tree they were installed by the ports infrastructure, so portmaster made no packages for them. In version 1.24 I added the -DNO_DEPENDS flag to 'make install' so that this problem can't happen. >> I'm also unsure >> what /usr/ports/ports is supposed to be, unless that's a local path >> issue. > > It's a local hierarchy issue. I pushed the ports tree up one level and > therein lie 6.2, -current, and the soon to be 7.0 trees. Ok, you might want to take a look at the PORTSDIR variable in ports/Mk/bsd.port.mk, and set that in your environment accordingly. >> Assuming that your list of ports is relative to the /usr/ports >> directory (e.g., 'archivers/unzip astro/planets', etc.), what I would >> do is: >> for dir in $pmarg; do >> portmaster -BgD -p /usr/ports/$dir >> done > > Aha! I had an error (don't recall exactly) without using -p where the > complaint had something to do with not finding something in /usr/ports. Yeah, that makes sense. If you set PORTSDIR in your environment you should be able to just specify the list of ports (astro/planets, etc.) without the -p option. > But what about recursion? I held the notion the portmaster did > magically delicious recursion Heh, I like that phrase. > and so I thought that providing the list > of ports to a single invocation of portmaster would allow portmaster to > sort the build order correctly. Well, yes and no. If you give the whole long list of ports to build it will save the information about which ports are already up to date between builds, but that feature has only been tested for "a few" ports at a time. In theory what you're proposing should work, but I haven't tested it. In fact, if you do that I'd be interested in your results. :) There is one other issue that I think would help you understand the process; the concept of root, trunk, branch, and leaf ports. If you do 'portmaster -l' on a system that has the stuff you want installed on it already, you'll get a good idea what I'm talking about. To start from scratch on a clean system you want to extract the list of root (no dependencies, not depended on) and leaf (have dependencies, not depended on) ports from a working system (add any additional ports you want of course), and then use that list to build on your clean system. This will allow portmaster to handle the recursion through the dependencies (trunk and branch ports). > The code snippet you show would cause > portmaster to _forget_ recursion information during each iteration of > the loop. My concern was that some dependencies would be built more > than once. Well good news there, if the installed version of the port is up to date it won't be rebuilt. The version check is pretty light weight, and only occurs once during the config phase. After that it's cached. Like I said above, specifying the whole list of ports on the command line _should_ work, and would avoid even having to do the check more than once, but doing the roots and leaves one by one shouldn't add very much of a performance penalty either. hope this helps, Doug -- This .signature sanitized for your protection