Date: Tue, 29 May 2007 20:02:56 +1000 From: Peter Jeremy <peterjeremy@optushome.com.au> To: Stephen Montgomery-Smith <stephen@math.missouri.edu> Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Looking for speed increases in "make index" and pkg_version for ports Message-ID: <20070529100255.GA70055@turion.vk2pj.dyndns.org> In-Reply-To: <4659EF80.70100@math.missouri.edu> References: <4659EF80.70100@math.missouri.edu>
next in thread | previous in thread | raw e-mail | index | archive | help
--+HP7ph2BbKc20aGI Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 2007-May-27 15:52:16 -0500, Stephen Montgomery-Smith <stephen@math.misso= uri.edu> wrote: > I have been thinking a lot about looking for speed increases for "make=20 > index" and pkg_version and things like that. So for example, in=20 > pkg_version, it calls "make -V PKGNAME" for every installed package. Now= =20 > "make -V PKGNAME" should be a speedy operation, but the make has to load = in=20 > and analyze bsd.port.mk, a quite complicated file with about 200,000=20 > characters in it, when all it is needing to do is to figure out the value= of=20 > the variable PKGNAME. This would be trivial if some packages didn't change names depending on options and what was installed. I agree that parsing a 210KB file 17,000 times is not going to be fast. Especially since some ports include bsd.ports.mk multiple times... > I suggest rewriting "make" so that variables are only evaluated on a "nee= d=20 > to know" basis. This sounds like a good idea but I suspect it's not going to be feasible. The biggest problem I see is that the make language allows variables to be expanded either when they are assigned or when they are referenced. If a variable expansion is delayed from the assignment to the first use then the expansion must be performed using the state of make as it was when the variable was assigned. The cost of keeping this state probably exceeds the cost of actually evaluating the variable. > So, for example, if all we need to know is PKGNAME, there=20 > is no need to evaluate, for example, _RUN_LIB_DEPENDS, unless the writer = of=20 > that particular port has done something like having PORTNAME depend on th= e=20 > value of _RUN_LIB_DEPENDS. Rather than trying to develop a tool that can quickly expand PKGNAME irrespective of what convoluted code the author has used, how about a partial solution? For most ports, PKGNAME depends solely on 3 or 4 variables that are statically defined in the port Makefile. The obvious solution would seem to be to develop a script that can handle the easy cases itself and punt the difficult cases back to make. The definition of 'easy' can be adjusted over time to increase performance. This approach would seem to have a relatively low bar to entry whilst offering good effort/performance tradeoff at the low end. The various depends lists would seem amenable to the same approach - though the entry level tool will have far lower coverage due to the extensive use of USE_GNOME=3D... and similar 'macro'-style constructs. --=20 Peter Jeremy --+HP7ph2BbKc20aGI Content-Type: application/pgp-signature Content-Disposition: inline -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.7 (FreeBSD) iD8DBQFGW/pP/opHv/APuIcRArw9AJ9WsRYjwIrgxdypFLGTK4G7JnQKPQCeNJvl QvVG9B881APC8Ap2MDhjbyA= =M39M -----END PGP SIGNATURE----- --+HP7ph2BbKc20aGI--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20070529100255.GA70055>