Date: Fri, 12 Nov 2004 20:57:21 +0200 From: Ruslan Ermilov <ru@freebsd.org> To: Poul-Henning Kamp <phk@phk.freebsd.dk> Cc: current@freebsd.org Subject: Re: [TEST] make -j patch [take 2] Message-ID: <20041112185721.GA88473@ip.net.ua> In-Reply-To: <6750.1100271157@critter.freebsd.dk> References: <20041112130548.GA86162@ip.net.ua> <6750.1100271157@critter.freebsd.dk>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --] On Fri, Nov 12, 2004 at 03:52:37PM +0100, Poul-Henning Kamp wrote: > In message <20041112130548.GA86162@ip.net.ua>, Ruslan Ermilov writes: > > >> At the very least, do not commit your patch until you have managed > >> to come up with at least one instance of real world data where it > >> is a good idea. > >> > >I gave you one example already, and Harti gave you another. > > > >"make universe" builds 6 worlds today. By starting it with > >"make -j12", you build 6 worlds in parallel, and another 6 > >jobs are made available for building these worlds. When one > >world finishes, 7 jobs will be available for building 5 other > >worlds, etc. This is fine as long as you're interested in > >the final result of "make universe". > > You have not actually _tried_ running make universe with > my patch, have you ? > > I thought so :-( > > make -j 12 universe will make 12 jobs available. The main > make will grab six of these to start the buildworlds, and > those six buildworlds compete for the remaining 6 tokens. > > The result is that those six buildworlds progress _in parallel_ > each of them using as many tokens as they can lay their hands on > at any one time. > This is exactly what I said above. I well understand that six buildworlds will compete for six remaining slots, then five buildworld will compete for seven slots, and so on. > If you want them to have more slots, say "-j 18" instead > and they will have 12 slots to compete for. > > This is, as far as I can tell, just the thing you want to happen, > only it doesn't suffer from pointless compartementalization > of resources such as in your proposal. > No, this is not what I want. :-) > The crucial point is that if at one particular instance the > i386.buildworld needs only one token, it will only claim > one token, and the other jobs can get more tokens and start > more things. Once the i386 needs to run more things in > parallel, it will compete for tokens again and run as much > in parallel as it can get tokens for. > I well understand this. > In your scenario, the tokens not used by i386 cannot be > used by the other buildworlds and they are wasted. > Not wasted, they just don't share the same pool of jobs. ;) > >If you're by chance familiar with how you can ask Oracle to > >optimize your queries, you can either optimize for ALL ROWS or > >for FIRST ROWS, and it's up to you to decide on this. The > >"rows" in "make universe" are results of doing buildworld > >and buildkernel, and you have control over this. > > This is not a database. > > This is not even a cartesian problem space. > I'm interested in completing i386 buildworld as fast as possible. In "make -j12 universe", the i386 world will be built in approximately (TOTAL) seconds (let's assume for the moment that we don't build kernels). If I use "make universe JFLAG=-j12", i386 world will complete in (TOTAL / 6) seconds. If I use "make -j2 universe JFLAG=-j6", i386 and pc98 worlds will complete in (TOTAL / 6 * 2) seconds. Can you see a difference? I'm interested in "first rows". And I'm not saying I'm always interested in them, just want to be able to test my patches as quickly as possible. ;) > >Do you disagree with anything of the above? > > Yes, I disagree with pretty much all of it. It is clear that > you have not actually tried running with the code which is in > the tree. > Why? It's clear that I understand how it works, and you fail to see _my_ point. ;) > >Can you agree to the below patch? It doesn't change anything > >except when JFLAG is set explicitly to request another level > >of parallelism for building individual worlds and kernels? > > The patch is pointless, but harmless once people learn not to set > JFLAG. > > But I will not agree to this patch _until_ you show me _actual_ _real > _life_ data that shows it performs better than just running without > a JFLAG at all. > You disagree that the i386 part of "make universe" will complete faster in my scenario, while still providing an overall limit of 12 jobs? I understand the drawback of my approach -- that the total time for "make universe" will be longer in my case. I now feel like everything that should have been said was said, and this thread should die. ;) Cheers, -- Ruslan Ermilov ru@FreeBSD.org FreeBSD committer [-- Attachment #2 --] -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.6 (FreeBSD) iD8DBQFBlQeRqRfpzJluFF4RAqQIAJ9u4pr0F/B+lXwUC6O4jKWhgakneACfc0aJ eoZ5B4M08eGbh/xC8Ig9j2Y= =8NEY -----END PGP SIGNATURE-----help
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041112185721.GA88473>