From owner-freebsd-ports@FreeBSD.ORG Wed Mar 16 10:51:50 2011 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 59543106564A for ; Wed, 16 Mar 2011 10:51:50 +0000 (UTC) (envelope-from matthias.andree@gmx.de) Received: from mailout-de.gmx.net (mailout-de.gmx.net [213.165.64.22]) by mx1.freebsd.org (Postfix) with SMTP id BFB068FC0C for ; Wed, 16 Mar 2011 10:51:49 +0000 (UTC) Received: (qmail invoked by alias); 16 Mar 2011 10:51:48 -0000 Received: from dtmd-4db21a48.pool.mediaWays.net (EHLO [192.168.178.26]) [77.178.26.72] by mail.gmx.net (mp038) with SMTP; 16 Mar 2011 11:51:48 +0100 X-Authenticated: #428038 X-Provags-ID: V01U2FsdGVkX198GAwi81V8QPQBIgZyOkjlkYDqjIyXoi6spbvMDB nGU0hcNmR4X3xV Message-ID: <4D809643.3020902@gmx.de> Date: Wed, 16 Mar 2011 11:51:47 +0100 From: Matthias Andree User-Agent: Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.15) Gecko/20110303 Mnenhy/0.8.3 Thunderbird/3.1.9 MIME-Version: 1.0 To: freebsd-ports@freebsd.org References: <4D7FBC0E.5020302@reiteration.net> <1468BFDD-5E3C-4756-830B-266D0942AED0@mac.com> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Subject: Re: can make -j be used for ports? X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 16 Mar 2011 10:51:50 -0000 Am 15.03.2011 23:44, schrieb Chuck Swiger: > On Mar 15, 2011, at 3:35 PM, Eitan Adler wrote: >> [ ... ] >>> Yes. Ports which support parallel builds will have MAKE_JOBS_SAFE=yes set in the port Makefile. It defaults to running -j with MAKE_JOBS_NUMBER=`${SYSCTL} -n kern.smp.cpus`, but you can change that to some other # if you like. >> >> No, this is incorrect. The MAKE_JOBS_NUMBER and MAKE_JOBS_SAFE is used >> internally when building a single port. > > What is incorrect? > >> When the OP is asking if he can manually specify -j on the command line which would end up >> building multiple ports in parallel. This can not be done (primarily >> because there is no locking done on ports) > > > It certainly wasn't clear to me that this is what the OP meant. If you: > > cd /usr/ports/www/apache22 > make -j 3 make FORCE_MAKE_JOBS=yes and the -j argument will default to the number of CPU cores as provided by the corresponding sysctl. If the port breaks with FORCE_MAKE_JOBS=yes please file a PR with a patch fixing it, or if that is too much of an effort, a patch to add a line "MAKE_JOBS_UNSAFE= yes" (on a single line with a TAB between MAKE_JOBS_UNSAFE= and yes). > ...what do you expect to happen, and how many ports would you expect to be built at once? > > (Building one port in parallel is supported, where the port itself is safe to do so; building many at the same time is not. Supporting the former provides more speed gain for many situations as compared to the latter; which doesn't help at all if you are just installing or updating one port.) I beg to differ on the speed assertion. I own a somewhat fast computer (Phenom II X4 i. e. 4 x 2.5 GHz) and I find that a lot of serialization takes place, particularly behind the configure phases and the mtree/registration stuff. In that respect, building non-dependent ports in parallel would be rather useful to actually exploit SMP computers; however it would require a dependency analysis so that only those ports build in parallel that do not depend on one another, and I also fear that sometimes mtree during installation might get in the way. I seem to recall that Doug stated something to the extent he wouldn't be doing it in portmaster, and I don't see any other tool that is similarly mature so it would be worthwhile even trying that. Best -- Matthias Andree