From owner-freebsd-current@FreeBSD.ORG Fri Nov 12 14:52:42 2004 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5D7F816A4CE; Fri, 12 Nov 2004 14:52:42 +0000 (GMT) Received: from critter.freebsd.dk (critter.freebsd.dk [212.242.86.163]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6363743D5A; Fri, 12 Nov 2004 14:52:41 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.1/8.13.1) with ESMTP id iACEqbQU006751; Fri, 12 Nov 2004 15:52:38 +0100 (CET) (envelope-from phk@critter.freebsd.dk) To: Ruslan Ermilov From: "Poul-Henning Kamp" In-Reply-To: Your message of "Fri, 12 Nov 2004 15:05:48 +0200." <20041112130548.GA86162@ip.net.ua> Date: Fri, 12 Nov 2004 15:52:37 +0100 Message-ID: <6750.1100271157@critter.freebsd.dk> Sender: phk@critter.freebsd.dk cc: Harti Brandt cc: current@freebsd.org Subject: Re: [TEST] make -j patch [take 2] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Nov 2004 14:52:42 -0000 In message <20041112130548.GA86162@ip.net.ua>, Ruslan Ermilov writes: >> At the very least, do not commit your patch until you have managed >> to come up with at least one instance of real world data where it >> is a good idea. >> >I gave you one example already, and Harti gave you another. > >"make universe" builds 6 worlds today. By starting it with >"make -j12", you build 6 worlds in parallel, and another 6 >jobs are made available for building these worlds. When one >world finishes, 7 jobs will be available for building 5 other >worlds, etc. This is fine as long as you're interested in >the final result of "make universe". You have not actually _tried_ running make universe with my patch, have you ? I thought so :-( make -j 12 universe will make 12 jobs available. The main make will grab six of these to start the buildworlds, and those six buildworlds compete for the remaining 6 tokens. The result is that those six buildworlds progress _in parallel_ each of them using as many tokens as they can lay their hands on at any one time. If you want them to have more slots, say "-j 18" instead and they will have 12 slots to compete for. This is, as far as I can tell, just the thing you want to happen, only it doesn't suffer from pointless compartementalization of resources such as in your proposal. The crucial point is that if at one particular instance the i386.buildworld needs only one token, it will only claim one token, and the other jobs can get more tokens and start more things. Once the i386 needs to run more things in parallel, it will compete for tokens again and run as much in parallel as it can get tokens for. In your scenario, the tokens not used by i386 cannot be used by the other buildworlds and they are wasted. >If you're by chance familiar with how you can ask Oracle to >optimize your queries, you can either optimize for ALL ROWS or >for FIRST ROWS, and it's up to you to decide on this. The >"rows" in "make universe" are results of doing buildworld >and buildkernel, and you have control over this. This is not a database. This is not even a cartesian problem space. >Do you disagree with anything of the above? Yes, I disagree with pretty much all of it. It is clear that you have not actually tried running with the code which is in the tree. >Can you agree to the below patch? It doesn't change anything >except when JFLAG is set explicitly to request another level >of parallelism for building individual worlds and kernels? The patch is pointless, but harmless once people learn not to set JFLAG. But I will not agree to this patch _until_ you show me _actual_ _real _life_ data that shows it performs better than just running without a JFLAG at all. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence.