Date: Wed, 9 Nov 2016 12:14:47 +0000 (UTC) From: Mathieu Arnold <mat@FreeBSD.org> To: ports-committers@freebsd.org, svn-ports-all@freebsd.org, svn-ports-head@freebsd.org Subject: svn commit: r425793 - head/Mk Message-ID: <201611091214.uA9CEl2d057412@repo.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: mat Date: Wed Nov 9 12:14:47 2016 New Revision: 425793 URL: https://svnweb.freebsd.org/changeset/ports/425793 Log: Optimize COPYTREE_* macros. The main problem slowing down those macros was that it was ending up calling chmod(1) once for each file and directory. It was doing this because it was running find(1) in the source tree and needed to chmod in the target tree, so it had to append the full path to the chmod(1) calls, which made it impossible to use -exec +. Rewrite the -exec calls to call sh and cd into it before running chmod(1) and do so on as many files as we can using the -exec + construct. While there, optimize a bit more after figuring out that it is possible to only use one find(1) to run two -exec for different options. Also switch to using the more powerfull modern regular expressions instead of the usual basic regular expressions by using find -E. This will allow the use of | which as no basic regular expressions equivalent (as in, for example, "*.(html|txt)") as well as the + and ? keywords, and back references. About the speed gain, I did a few tests, using time's builtin from csh (table with all the data available in the review), comparing the home baked versions of COPYTREE_SHARE found in ports, it boils down to: | version | www/sit (~2k) | www/mathjax (~30k) | +------------+------------------+--------------------+ | home baked | 0.032u 0.104s | 0.946u 7.446s | | current | 0.281u 0.649s | 5.446u 18.626s | | new | 0.024u 0.081s | 0.770u 7.012s | PR: 213614 Submitted by: mat Exp-run by: antoine Sponsored by: Absolight Differential Revision: https://reviews.freebsd.org/D8283 Modified: head/Mk/bsd.port.mk (contents, props changed) Modified: head/Mk/bsd.port.mk ============================================================================== --- head/Mk/bsd.port.mk Wed Nov 9 11:19:20 2016 (r425792) +++ head/Mk/bsd.port.mk Wed Nov 9 12:14:47 2016 (r425793) @@ -2057,14 +2057,15 @@ MAKE_ENV+= ${INSTALL_MACROS} SCRIPTS_ENV+= ${INSTALL_MACROS} # Macro for copying entire directory tree with correct permissions -COPYTREE_BIN= ${SH} -c '(${FIND} -d $$0 $$2 | ${CPIO} -dumpl $$1 >/dev/null \ - 2>&1) && \ - ${FIND} -d $$0 $$2 -type d -exec chmod 755 $$1/{} \; && \ - ${FIND} -d $$0 $$2 -type f -exec chmod ${BINMODE} $$1/{} \;' -- -COPYTREE_SHARE= ${SH} -c '(${FIND} -d $$0 $$2 | ${CPIO} -dumpl $$1 >/dev/null \ - 2>&1) && \ - ${FIND} -d $$0 $$2 -type d -exec chmod 755 $$1/{} \; && \ - ${FIND} -d $$0 $$2 -type f -exec chmod ${SHAREMODE} $$1/{} \;' -- +# In the -exec shell commands, we add add a . as the first argument, it would +# end up being $0 aka the script name, which is not part of $@, so we force it +# to be able to use $@ directly. +COPYTREE_BIN= ${SH} -c '(${FIND} -Ed $$0 $$2 | ${CPIO} -dumpl $$1 >/dev/null 2>&1) && \ + ${FIND} -Ed $$0 $$2 \( -type d -exec ${SH} -c '\''cd '\''$$1'\'' && chmod 755 "$$@"'\'' -- . {} + \ + -o -type f -exec ${SH} -c '\''cd '\''$$1'\'' && chmod ${BINMODE} "$$@"'\'' -- . {} + \)' -- +COPYTREE_SHARE= ${SH} -c '(${FIND} -Ed $$0 $$2 | ${CPIO} -dumpl $$1 >/dev/null 2>&1) && \ + ${FIND} -Ed $$0 $$2 \( -type d -exec ${SH} -c '\''cd '\''$$1'\'' && chmod 755 "$$@"'\'' -- . {} + \ + -o -type f -exec ${SH} -c '\''cd '\''$$1'\'' && chmod ${SHAREMODE} "$$@"'\'' -- . {} + \)' -- # The user can override the NO_PACKAGE by specifying this from # the make command line
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201611091214.uA9CEl2d057412>