From owner-freebsd-questions@FreeBSD.ORG Tue Mar 6 01:40:55 2007 Return-Path: X-Original-To: freebsd-questions@freebsd.org Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 55CBB16A400 for ; Tue, 6 Mar 2007 01:40:55 +0000 (UTC) (envelope-from fbsd06@mlists.homeunix.com) Received: from mxout-03.mxes.net (mxout-03.mxes.net [216.86.168.178]) by mx1.freebsd.org (Postfix) with ESMTP id 2ED5013C428 for ; Tue, 6 Mar 2007 01:40:55 +0000 (UTC) (envelope-from fbsd06@mlists.homeunix.com) Received: from gumby.homeunix.com (unknown [87.81.140.128]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by smtp.mxes.net (Postfix) with ESMTP id CD3AF51937; Mon, 5 Mar 2007 20:40:53 -0500 (EST) Date: Tue, 6 Mar 2007 01:40:51 +0000 From: RW To: freebsd-questions@freebsd.org Message-ID: <20070306014051.0261645d@gumby.homeunix.com> In-Reply-To: <006c01c75f78$08097420$181c5c60$@Sparrevohn@btinternet.com> References: <20070228183733.4f6ddfe7@gumby.homeunix.com> <20070305212122.672136df@gumby.homeunix.com> <006c01c75f78$08097420$181c5c60$@Sparrevohn@btinternet.com> X-Mailer: Claws Mail 2.8.0 (GTK+ 2.10.9; i386-portbld-freebsd6.2) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Cc: Thomas.Sparrevohn@btinternet.com Subject: Re: compiling ports with more than one job X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Mar 2007 01:40:55 -0000 On Mon, 5 Mar 2007 22:45:53 -0000 "Thomas Sparrevohn" wrote: > There really two answers possible here - > > 1) Let's call it one depth e.g. make -j - Which works with some not > all ports - Nice when it works and I guess ports/Mk could hold a flag > 2) Let's call it width - e.g. the ability to compile packages at the > same time given that all dependencies has been resolved > 3) combination of 1 and 2 > > In practical terms option 2 is much more attractive as it is possible > to determine that just from the INDEX file and the installed ports. You would probably have to do it like portmanger does, the INDEX file isn't accurate. > However due to the way compilation options are treated e.g. I am not > sure that it is completely safe - I will require some locking > during the make (de/re)install phase but possible to handle - It > would still cut portupgrade significantly It couldn't be done easily in the ports system itself, a new tool would be needed. However I think it can be done without any locking at all. The way I would do it is have a single master process that works through the packages in dependency order. For each package it does a "make configure"and then spawns off the build in a separate process. Once N builds are in progress, the master process sits there waiting for processes to complete, serializes the installs and kicks off new build processes. The beauty of this is that anything in the least bit critical is done in the single master process, but most of the work is done in the build processes, so the master wont be a bottleneck. There's a bit more to it, for example there would have to be special handling for packages that appear to have missing dependencies, consequently a different logic would be needed for new installs. > With 15-16000+ ports I think that 1 and 3 are unpractical - however I don't think it is, it could be handled the way amd64 ports are/were handled, by asking people to try them and report back.