From owner-freebsd-ports@FreeBSD.ORG Mon May 28 15:55:49 2007 Return-Path: X-Original-To: ports@freebsd.org Delivered-To: freebsd-ports@FreeBSD.ORG Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 34B6116A46E; Mon, 28 May 2007 15:55:49 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from smtp-1.dlr.de (smtp-1.dlr.de [195.37.61.185]) by mx1.freebsd.org (Postfix) with ESMTP id C4AF313C483; Mon, 28 May 2007 15:55:48 +0000 (UTC) (envelope-from Hartmut.Brandt@dlr.de) Received: from [129.247.12.6] ([129.247.12.6]) by smtp-1.dlr.de with Microsoft SMTPSVC(6.0.3790.1830); Mon, 28 May 2007 17:55:47 +0200 Message-ID: <465AFB7B.40105@dlr.de> Date: Mon, 28 May 2007 17:55:39 +0200 From: Hartmut Brandt User-Agent: Thunderbird 1.5.0.10 (Windows/20070221) MIME-Version: 1.0 To: Stephen Montgomery-Smith References: <4659EF80.70100@math.missouri.edu> <465AB421.10802@dlr.de> <465AF2CC.9020506@math.missouri.edu> In-Reply-To: <465AF2CC.9020506@math.missouri.edu> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-OriginalArrivalTime: 28 May 2007 15:55:47.0265 (UTC) FILETIME=[A80B7310:01C7A140] Cc: ports@freebsd.org, hackers@freebsd.org Subject: Re: Looking for speed increases in "make index" and pkg_version for ports X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 May 2007 15:55:49 -0000 Stephen Montgomery-Smith wrote: > Hartmut Brandt wrote: > >> Having done a great deal of rewriting of make some two years ago I can >> tell you that even a small change to make is a tough job testing-wise: >> run all the combinations of !-j and -j on all architectures and run >> the change through the port-building cluster. That's a warning to start >> with. >> >> Second I would start with careful profiling to find out where the >> problem actually is. You might be surprised. As an example: several >> times the idea came up to use a hash structure instead of linear lists >> for make variables. I got a patch for this and - it makes absolutely no >> difference performance-wise (well, there was some indication that >> performance gets worse, but that was around or below noise level). With >> careful I mean to find out who takes the time: > > Yes, I must admit that I thought that a hash structure for the variables > would greatly increase the speed of make. I rewrote it using Berkeley > databases, and like you said - absolutely no difference!! I even tried > btrees. > My guess at that time was that because there are actually many variable tables (one per target and the global one) and only a small number of variables in most of the tables the initialisation overhead outweights what you win through the hashing. As for the profiling - I did some profiling on buildworld then. From the several hours a buildworld took only one or two minutes were used by all the makes. At this point I stopped optimizing make :-) (I don't remember the exact numbers - that was two or three years ago). harti