Date: Mon, 28 May 2007 18:45:44 -0400 From: Mike Meyer <mwm@mired.org> To: Hartmut Brandt <hartmut.brandt@dlr.de> Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Looking for speed increases in "make index" and pkg_version for ports Message-ID: <18011.23448.513335.199513@bhuda.mired.org> In-Reply-To: <465B4A84.6050407@dlr.de> References: <4659EF80.70100@math.missouri.edu> <465AB421.10802@dlr.de> <18011.6019.436391.128372@bhuda.mired.org> <465B4A84.6050407@dlr.de>
next in thread | previous in thread | raw e-mail | index | archive | help
In <465B4A84.6050407@dlr.de>, Hartmut Brandt <hartmut.brandt@dlr.de> typed: > Mike Meyer wrote: > > In <465AB421.10802@dlr.de>, Hartmut Brandt <hartmut.brandt@dlr.de> typed: > >> 1. make and its sub-makes for a) reading the file; b) parsing the file > >> (note that .if and .for processing is done while parsing); c) processing > >> targets. > > > > Make and submakes have been gone over already. See <URL: > > http://miller.emu.id.au/pmiller/books/rmch/ >. > > > > I'm not sure it can be applied to the ports tree, though. I haven't > > looked into it, but recalled this paper when you mentioned measuring > > makes and sub-makes. > Unfortunately you deleted the sentence before, so I rephrase it: before > looking into optimizations find out where the time is actually spend - > how many seconds of the hours the process takes, are actually spent in > make and sub-makes. If the entire process takes 2 hours of which the > makes take 20 seconds then by enhancing performance of make by 50% you > win 10 seconds. This is probably not worth a single line of additional code. > > The paper you point to talks about something entirely different. It think we're talking about two different things. You're talking about the efficiency of make, whereas he's talking about the efficiency of make. Um, wait. You're talking about what I'll call the *internal* efficiency of make, defined as how fast it does the things it does. He's talking about what I'll call the *external* efficiency of make, which is how well it does at doing the minimum amount of work it needs to do. I hope you can see where the confusion comes from. In particular, he talks about how recursive makefiles screw up evaluating complex variables, causing them to be executed multiple times. So if you're running a makefile to pull some variables value, as opposed to do real commands, and your entire process takes 2 hours and the Makefile takes 20 seconds, but it evaluates all the variables twice, then by fixing your makefile you win at least 59 minutes and 50 seconds. I think cutting the run time by 50% is worth some work. Benchmarking can help you decide which things it pays to work on if all you're worried about is the internal efficiency. However, the goal is to make the process faster, so we need to worry about the external efficiency as well. The problem here is that the worse it is, the less it looks like you stand to gain by looking at your makefile when you look at the benchmarks. Given that the ports system has both highly complex variables and is very recursive, I believe that it warrants investigation if you're going to work on making make in the ports faster. <mike -- Mike Meyer <mwm@mired.org> http://www.mired.org/consulting.html Independent Network/Unix/Perforce consultant, email for more information.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?18011.23448.513335.199513>