From owner-freebsd-ports@FreeBSD.ORG Thu Nov 2 10:54:07 2006 Return-Path: X-Original-To: freebsd-ports@freebsd.org Delivered-To: freebsd-ports@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6AC4516A407 for ; Thu, 2 Nov 2006 10:54:07 +0000 (UTC) (envelope-from michel@lpthe.jussieu.fr) Received: from shiva.jussieu.fr (shiva.jussieu.fr [134.157.0.129]) by mx1.FreeBSD.org (Postfix) with ESMTP id B563E43D53 for ; Thu, 2 Nov 2006 10:54:06 +0000 (GMT) (envelope-from michel@lpthe.jussieu.fr) Received: from parthe.lpthe.jussieu.fr (parthe.lpthe.jussieu.fr [134.157.10.1]) by shiva.jussieu.fr (8.13.7/jtpda-5.4) with ESMTP id kA2As4Mk048195 for ; Thu, 2 Nov 2006 11:54:04 +0100 (CET) X-Ids: 166 Received: from niobe.lpthe.jussieu.fr (niobe.lpthe.jussieu.fr [134.157.10.41]) by parthe.lpthe.jussieu.fr (Postfix) with ESMTP id 25575A00AB for ; Thu, 2 Nov 2006 11:54:04 +0100 (CET) Received: by niobe.lpthe.jussieu.fr (Postfix, from userid 2005) id A39D01D; Thu, 2 Nov 2006 11:54:23 +0100 (CET) Date: Thu, 2 Nov 2006 11:54:23 +0100 From: Michel Talon To: freebsd-ports@freebsd.org Message-ID: <20061102105423.GA63547@lpthe.jussieu.fr> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4.2.1i X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0.2 (shiva.jussieu.fr [134.157.0.166]); Thu, 02 Nov 2006 11:54:04 +0100 (CET) X-Virus-Scanned: ClamAV 0.88.5/2146/Thu Nov 2 07:58:29 2006 on shiva.jussieu.fr X-Virus-Status: Clean X-Miltered: at shiva.jussieu.fr with ID 4549CE4C.002 by Joe's j-chkmail (http://j-chkmail.ensmp.fr)! Subject: Building the INDEX X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 02 Nov 2006 10:54:07 -0000 Hello, Some time ago i have reported about a python script able to build the INDEX. Vasil Dimov pointed out that it had a performance problem, compared to make index INDEX_JOBS=3 This is of course because the make index command, however ugly it is programmed, achieves a lot of parallelism, which is difficult to achieve with python. Anyways, numbers floated around about the build time necessary to compute the INDEX, and the problem this is causing to the ports team. People mentioned times of the order of 30' for the present day ports tree with 16000 ports. Today someone lent me a Core 2 Duo machine on which i have loaded FreeBSD-64. Nice result is that both my python program (that i have multithreaded - but there is room for improvement) and make index INDEX_JOBS=3 complete building the INDEX in 8 minutes. Of course this program requires python, but make index requires perl, which is not better. In view of this, one may imagine that a properly threaded C program, which would allow better parallelism, could build the INDEX in perhaps 6 minutes. Frankly, these Core 2 Duo machines are absolutely fantastic. Basically my python program runs more than 3 times faster than on a P4, 3 Ghz, hyperthreaded. -- Michel TALON