From owner-freebsd-ports@FreeBSD.ORG Mon May 28 09:03:03 2007 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E0FC316A4FF; Mon, 28 May 2007 09:03:03 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from smtp.infracaninophile.co.uk (happy-idiot-talk.infracaninophile.co.uk [81.187.76.162]) by mx1.freebsd.org (Postfix) with ESMTP id 339B213C484; Mon, 28 May 2007 09:03:02 +0000 (UTC) (envelope-from m.seaman@infracaninophile.co.uk) Received: from happy-idiot-talk.infracaninophile.co.uk (localhost.infracaninophile.co.uk [IPv6:::1]) by smtp.infracaninophile.co.uk (8.14.1/8.14.1) with ESMTP id l4S8bKn6006910; Mon, 28 May 2007 09:37:22 +0100 (BST) (envelope-from m.seaman@infracaninophile.co.uk) Authentication-Results: smtp.infracaninophile.co.uk from=m.seaman@infracaninophile.co.uk; sender-id=permerror; spf=permerror X-SenderID: Sendmail Sender-ID Filter v0.2.14 smtp.infracaninophile.co.uk l4S8bKn6006910 Message-ID: <465A94C0.2080301@infracaninophile.co.uk> Date: Mon, 28 May 2007 09:37:20 +0100 From: Matthew Seaman Organization: Infracaninophile User-Agent: Thunderbird 2.0.0.0 (X11/20070521) MIME-Version: 1.0 To: Stephen Montgomery-Smith References: <4659EF80.70100@math.missouri.edu> In-Reply-To: <4659EF80.70100@math.missouri.edu> X-Enigmail-Version: 0.95.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (smtp.infracaninophile.co.uk [IPv6:::1]); Mon, 28 May 2007 09:37:32 +0100 (BST) X-Virus-Scanned: ClamAV 0.90.2/3311/Mon May 28 01:21:25 2007 on happy-idiot-talk.infracaninophile.co.uk X-Virus-Status: Clean X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00, DKIM_POLICY_TESTING, DK_POLICY_SIGNSOME,NO_RELAYS autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on happy-idiot-talk.infracaninophile.co.uk Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Looking for speed increases in "make index" and pkg_version for ports X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2007 09:03:04 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Stephen Montgomery-Smith wrote: > I have been thinking a lot about looking for speed increases for "make > index" and pkg_version and things like that. So for example, in > pkg_version, it calls "make -V PKGNAME" for every installed package. Now > "make -V PKGNAME" should be a speedy operation, but the make has to load > in and analyze bsd.port.mk, a quite complicated file with about 200,000 > characters in it, when all it is needing to do is to figure out the > value of the variable PKGNAME. pkg_version is one thing -- but to build the INDEX you need to extract at least the values of the following variables: PKGNAME .CURDIR PREFIX COMMENT DESCR MAINTAINER CATEGORIES EXTRACT_DEPENDS PATCH_DEPENDS FETCH_DEPENDS BUILD_DEPENDS RUN_DEPENDS LIB_DEPENDS Plus you need to grep in the referenced pkg-descr file for any WWW links. I also extract the values of: MASTER_PORT .MAKEFILE_LIST SUBDIR for my FreeBSD::Portindex stuff. Trouble is, by the time you've extracted all that lot, you have pretty much done the same level of variable processing as you would were you actually going to build the port. One thing that would speed up this process would be a make option to just do parsing of the Makefile and variable expansion, without calling stat(2) on all the various sources and dependencies involved. For instance: happy-idiot-talk:...ports/databases/mysql-connector-java:% truss make -V PKGNAME |& grep stat | wc -l 49 It is quite instructive to see what files make(1) touches while doing that. At least half of them are irrelevant if all make(1) is going to do is print out the values of some variables. Multiply that by 17,000 and it adds up to a big waste of effort. Cheers, Matthew - -- Dr Matthew J Seaman MA, D.Phil. 7 Priory Courtyard Flat 3 PGP: http://www.infracaninophile.co.uk/pgpkey Ramsgate Kent, CT11 9PW -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.3 (FreeBSD) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFGWpTA8Mjk52CukIwRCI0BAJ0bX5hTAJkMCO6Pl+cA4THv3mKulwCgg+39 kCyAGOTYYz9vEzzM9NRe3no= =MqFV -----END PGP SIGNATURE-----