From owner-freebsd-ports@FreeBSD.ORG  Wed Jul 23 22:15:18 2008
Return-Path: <owner-freebsd-ports@FreeBSD.ORG>
Delivered-To: freebsd-ports@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 6E8771065688;
	Wed, 23 Jul 2008 22:15:18 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Received: from weak.local (freefall.freebsd.org [IPv6:2001:4f8:fff6::28])
	by mx1.freebsd.org (Postfix) with ESMTP id 13ED28FC1A;
	Wed, 23 Jul 2008 22:15:10 +0000 (UTC)
	(envelope-from kris@FreeBSD.org)
Message-ID: <4887AD6F.4070704@FreeBSD.org>
Date: Thu, 24 Jul 2008 00:15:11 +0200
From: Kris Kennaway <kris@FreeBSD.org>
User-Agent: Thunderbird 2.0.0.14 (Macintosh/20080421)
MIME-Version: 1.0
To: "V.Chukharev" <chukharev@mail.ru>
References: <op.ueqfl1xy0g54sc@localhost>
In-Reply-To: <op.ueqfl1xy0g54sc@localhost>
Content-Type: text/plain; charset=ISO-8859-15; format=flowed
Content-Transfer-Encoding: 7bit
Cc: dinoex@freebsd.org, "freebsd-ports@freebsd.org" <freebsd-ports@freebsd.org>
Subject: Re: Mk/bsd.openssl.mk optimization
X-BeenThere: freebsd-ports@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Porting software to FreeBSD <freebsd-ports.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-ports>
List-Post: <mailto:freebsd-ports@freebsd.org>
List-Help: <mailto:freebsd-ports-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-ports>,
	<mailto:freebsd-ports-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 23 Jul 2008 22:15:18 -0000

V.Chukharev wrote:
> Dear maintainer,
> 
> I have noticed that most time of 'make index' on my system takes a grep for libssl.
> The following is done while 'make index' was running.
> 
> $ ps axww | grep libssl
> 23119  p1  IN+    0:00,00 sh -c grep -l -r "^lib/libssl.so." "/var/db/pkg" |  while read contents; do  sslprefix=`grep "^@cwd " "${contents}" | /usr/bin/head -n 1`;  if test "${sslprefix}" = "@cwd /usr/local" ; then  echo "${contents}"; break; fi; done
> 23124  p1  DN+    0:00,84 grep -l -r ^lib/libssl.so. /var/db/pkg
> 23125  p1  IN+    0:00,00 sh -c grep -l -r "^lib/libssl.so." "/var/db/pkg" |  while read contents; do  sslprefix=`grep "^@cwd " "${contents}" | /usr/bin/head -n 1`;  if test "${sslprefix}" = "@cwd /usr/local" ; then  echo "${contents}"; break; fi; done
> 24555  p1  IN+    0:00,00 sh -c grep -l -r "^lib/libssl.so." "/var/db/pkg" |  while read contents; do  sslprefix=`grep "^@cwd " "${contents}" | /usr/bin/head -n 1`;  if test "${sslprefix}" = "@cwd /usr/local" ; then  echo "${contents}"; break; fi; done
> 24556  p1  DN+    0:00,41 grep -l -r ^lib/libssl.so. /var/db/pkg
> 24557  p1  IN+    0:00,00 sh -c grep -l -r "^lib/libssl.so." "/var/db/pkg" |  while read contents; do  sslprefix=`grep "^@cwd " "${contents}" | /usr/bin/head -n 1`;  if test "${sslprefix}" = "@cwd /usr/local" ; then  echo "${contents}"; break; fi; done
> 
> So I looked through some Mk stuff, and tryed to optimize it. Since I do not
> know any magic of bsd.*.mk, I did just a simple test. And with the below shown
> patch the time is only 1500 s, while with the original version the time is more
> than 15000 s. I guess the time depens very much on number of installed ports,
> I have about 1300.

The right way is either/or:

a) to do the != assignment in bsd.port.subdir.mk and pass it in to the 
child make environment.  Actually the other != assignments also look 
like they should be fixed too, since they'll also be eating up your CPU 
time.

b) Try to reduce the number of external commands being invoked by doing 
it in shell or make instead (at least partially).

See the recent commits I did to the Mk files for more specifics.

Kris