Date: Mon, 14 May 2007 09:39:13 -0500 From: Stephen Montgomery-Smith <stephen@math.missouri.edu> To: "[LoN]Kamikaze" <LoN_Kamikaze@gmx.de> Cc: Pav Lucistnik <pav@freebsd.org>, freebsd-ports@freebsd.org, Kris Kennaway <kris@obsecurity.org> Subject: Re: Time to abandon recursive pulling of dependencies? Message-ID: <46487491.6020806@math.missouri.edu> In-Reply-To: <4647F182.6000402@gmx.de> References: <464597C6.3030406@gmx.de> <20070512174011.GA22526@xor.obsecurity.org> <4645FF71.60100@gmx.de> <20070512175824.GA23103@xor.obsecurity.org> <20070512133054.B5588@math.missouri.edu> <4646193E.5040503@gmx.de> <464665D5.1090509@math.missouri.edu> <464676CC.2000203@math.missouri.edu> <46469BF5.9030603@math.missouri.edu> <1179058696.13237.3.camel@ikaros.oook.cz> <20070513101332.L82202@math.missouri.edu> <46479215.1080401@math.missouri.edu> <4647C4FB.9050607@math.missouri.edu> <4647F182.6000402@gmx.de>
index | next in thread | previous in thread | raw e-mail
[-- Attachment #1 --]
[LoN]Kamikaze wrote:
> Stephen Montgomery-Smith wrote:
>> Basically I think we are stuck on making "make package-depends" go any
>> faster.
>>
>> However I do think that the modifications I made to pkg_create go a very
>> significant way to solving the problem of registration taking so very long.
>>
>> Stephen
>>
>
> You are right about invoking make being the problem. However, I was able to
> increase the performance of package-depends by 20 to 30 percent, by avoiding
> recursive make calls and moving the recursion into the stack of the shell
> invocation.
>
> With my changes there's less than 10% overhead over the sum of make calls. I.e.
> for 56 seconds of make calls (x11/xorg on my system) there are only 4 more
> seconds of processing, instead of 15 to 20 seconds.
>
> The original make package-depends does some things I don't understand. Once
> I've figured them out I will post my little patch here.
>
>
Someone pointed out that what I was proposing in +DEPENDENCIES is
already to be found in +CONTENTS. So here is a proof of concept patch
to /usr/ports/Mk/bsd.port.mk (proof of concept because no error
checking, and things like that).
For me it makes registration about 3 times faster.
[-- Attachment #2 --]
--- bsd.port.mk-orig Mon May 14 09:28:42 2007
+++ bsd.port.mk Mon May 14 09:33:39 2007
@@ -2387,7 +2387,7 @@
DISABLE_CONFLICTS= YES
.endif
.if !defined(PKG_ARGS)
-PKG_ARGS= -v -c -${COMMENT:Q} -d ${DESCR} -f ${TMPPLIST} -p ${PREFIX} -P "`cd ${.CURDIR} && ${MAKE} package-depends | ${GREP} -v -E ${PKG_IGNORE_DEPENDS} | ${SORT} -u`" ${EXTRA_PKG_ARGS} $${_LATE_PKG_ARGS}
+PKG_ARGS= -v -c -${COMMENT:Q} -d ${DESCR} -f ${TMPPLIST} -p ${PREFIX} -P "`cd ${.CURDIR} && ${MAKE} actual-package-depends | ${GREP} -v -E ${PKG_IGNORE_DEPENDS} | ${SORT} -u`" ${EXTRA_PKG_ARGS} $${_LATE_PKG_ARGS}
.if !defined(NO_MTREE)
PKG_ARGS+= -m ${MTREE_FILE}
.endif
@@ -5216,10 +5216,26 @@
fi; \
done
+ACTUAL-PACKAGE-DEPENDS?= \
+ pkgs=`(for dir in ${_LIB_RUN_DEPENDS:C,[^:]*:([^:]*):?.*,\1,}; do \
+ dir=$$(${REALPATH} $$dir); \
+ if [ -d $$dir ]; then \
+ pkgname=$$(cd $$dir; ${MAKE} -V PKGNAME); \
+ ${ECHO_CMD} $$pkgname; \
+ ${GREP} @pkgdep /var/db/pkg/$$pkgname/+CONTENTS | ${SED} "s/@pkgdep //"; \
+ fi; \
+ done) | sort -u`; \
+ for pkgname in $$pkgs; do \
+ ${ECHO_CMD} $$pkgname`${GREP} "@comment ORIGIN:" /var/db/pkg/$$pkgname/+CONTENTS | ${SED} "s/@comment ORIGIN//"`; \
+ done
+
# Print out package names.
package-depends:
@${PACKAGE-DEPENDS-LIST} | ${AWK} '{print $$1":"$$3}'
+
+actual-package-depends:
+ @${ACTUAL-PACKAGE-DEPENDS}
# Build packages for port and dependencies
home |
help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?46487491.6020806>