Date: Sun, 13 May 2007 08:43:57 -0500 From: Stephen Montgomery-Smith <stephen@math.missouri.edu> To: Kris Kennaway <kris@obsecurity.org> Cc: "\[LoN\]Kamikaze" <LoN_Kamikaze@gmx.de>, freebsd-ports@freebsd.org Subject: Re: Time to abandon recursive pulling of dependencies? Message-ID: <4647161D.2040100@math.missouri.edu> In-Reply-To: <20070513084931.GA35696@xor.obsecurity.org> References: <464597C6.3030406@gmx.de> <20070512174011.GA22526@xor.obsecurity.org> <4645FF71.60100@gmx.de> <20070512175824.GA23103@xor.obsecurity.org> <20070512133054.B5588@math.missouri.edu> <4646193E.5040503@gmx.de> <464665D5.1090509@math.missouri.edu> <4646AC48.1010107@gmx.de> <4646CFE3.1020105@gmx.de> <20070513084931.GA35696@xor.obsecurity.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Kris Kennaway wrote:
> On Sun, May 13, 2007 at 10:44:19AM +0200, [LoN]Kamikaze wrote:
>> [LoN]Kamikaze wrote:
>>> Stephen Montgomery-Smith wrote:
>>>> OK chaps, this is what I came up with. So for example, if I do "make
>>>> install" on /usr/ports/x11/xorg (having made all the dependencies), on
>>>> my computer it turns the pkg_create from taking about 4 minutes to the
>>>> blink of an eye. Now people need to figure out how to speed up the
>>>> "make package-depends" in bsd.ports.mk, but that is beyond my abilities.
>>> I will give that a try, I've been experimenting with make a lot lately. Thank
>>> you for your patch, the second attempt seems to work fine on my system.
>> So far I've got a 20% performance increase, not really satisfying. The trouble
>> is that for every dependency the whole bsd.port.mk and whatever else is pulled
>> in by the dependency has to be parsed again. So the only way to make a
>> significant improvement is to have a short bsd.port.mk.
>
> Again, what evidence do you have that *that* is really the problem?
> Please try to be more scientific in your approach, it will help all of
> us to figure this out.
>
> Kris
I looked inside the bsd.ports.mk, and put in a few ${ECHO_MSG}
statements here and there. I also put a few printf statements in
pkg_create. I found out that there are two parts that take a long time:
1. Pulling in the dependencies. This is in effect doing a "make
package-depends". You can do this for yourself, and see that it takes a
long time. I honestly don't see how to make this faster, as presumably
it involves calling "make" on all the dependency ports, and I would
guess that each new invocation of make adds considerable overhead just
loading the program.
2. Sorting the dependencies in pkg_create. My fix now cuts this out
completely. Thus there is no need to change the structure of
/var/db/pkg to use databases or anything like this.
Stephen
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4647161D.2040100>