From owner-freebsd-ports@FreeBSD.ORG Sun May 13 08:58:47 2007 Return-Path: X-Original-To: freebsd-ports@freebsd.org Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id C581D16A402 for ; Sun, 13 May 2007 08:58:47 +0000 (UTC) (envelope-from LoN_Kamikaze@gmx.de) Received: from mail.gmx.net (mail.gmx.net [213.165.64.20]) by mx1.freebsd.org (Postfix) with SMTP id 3A5B513C447 for ; Sun, 13 May 2007 08:58:47 +0000 (UTC) (envelope-from LoN_Kamikaze@gmx.de) Received: (qmail invoked by alias); 13 May 2007 08:58:46 -0000 Received: from nat-wh-1.rz.uni-karlsruhe.de (EHLO mobileKamikaze.norad) [129.13.72.169] by mail.gmx.net (mp036) with SMTP; 13 May 2007 10:58:46 +0200 X-Authenticated: #5465401 X-Provags-ID: V01U2FsdGVkX19sM7FqXfBsg+jdO4/JllP06+4r7Q8wPXHBvaffBY ClKARLR3qNW4se Message-ID: <4646D32E.8020507@gmx.de> Date: Sun, 13 May 2007 10:58:22 +0200 From: "[LoN]Kamikaze" User-Agent: Thunderbird 2.0.0.0 (X11/20070506) MIME-Version: 1.0 To: Kris Kennaway References: <464597C6.3030406@gmx.de> <20070512174011.GA22526@xor.obsecurity.org> <4645FF71.60100@gmx.de> <20070512175824.GA23103@xor.obsecurity.org> <20070512133054.B5588@math.missouri.edu> <4646193E.5040503@gmx.de> <464665D5.1090509@math.missouri.edu> <4646AC48.1010107@gmx.de> <4646CFE3.1020105@gmx.de> <20070513084931.GA35696@xor.obsecurity.org> In-Reply-To: <20070513084931.GA35696@xor.obsecurity.org> X-Enigmail-Version: 0.95.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Y-GMX-Trusted: 0 Cc: Stephen Montgomery-Smith , freebsd-ports@freebsd.org Subject: Re: Time to abandon recursive pulling of dependencies? X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 13 May 2007 08:58:47 -0000 Kris Kennaway wrote: > On Sun, May 13, 2007 at 10:44:19AM +0200, [LoN]Kamikaze wrote: >> [LoN]Kamikaze wrote: >>> Stephen Montgomery-Smith wrote: >>>> OK chaps, this is what I came up with. So for example, if I do "make >>>> install" on /usr/ports/x11/xorg (having made all the dependencies), on >>>> my computer it turns the pkg_create from taking about 4 minutes to the >>>> blink of an eye. Now people need to figure out how to speed up the >>>> "make package-depends" in bsd.ports.mk, but that is beyond my abilities. >>> I will give that a try, I've been experimenting with make a lot lately. Thank >>> you for your patch, the second attempt seems to work fine on my system. >> So far I've got a 20% performance increase, not really satisfying. The trouble >> is that for every dependency the whole bsd.port.mk and whatever else is pulled >> in by the dependency has to be parsed again. So the only way to make a >> significant improvement is to have a short bsd.port.mk. > > Again, what evidence do you have that *that* is really the problem? > Please try to be more scientific in your approach, it will help all of > us to figure this out. > > Kris Well, it takes about 1 minute and 15 seconds for x11/xorg on my system, which has 273 dependencies, with my optimizations it still takes 60 seconds. Parsing bsd.port.mk takes an average of 0.2 seconds on my system. 0.2 * 273 = 54.6 seconds. Which looks like the major part of computation time to me. The only way to avoid this I see, is to abandon the recursive pulling in. Packages wouldn't know all their dependencies any more, but the dependencies would still know what they depend on and so forth. I will just change my system and see weather it breaks. I suspect that some things like pkg_add -r would have to be slightly adjusted to support such a change.