Date: Sun, 18 Oct 2009 20:43:49 -0700 From: Kenyon Ralph <kenyon@kenyonralph.com> To: freebsd-ports@freebsd.org Subject: portmaster: grep: Argument list too long Message-ID: <20091019034349.GM24097@kenyonralph.com>
next in thread | raw e-mail | index | archive | help
--b0op/nKJ9CeIhp9z
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
I'm trying to replace perl 5.8 with perl 5.10, so I did this (as in the
20090328 /usr/ports/UPDATING entry):
% sudo portmaster -v -t -D -o lang/perl5.10 lang/perl5.8
% sudo portmaster -v -D -R -m 'FORCE_PKG_REGISTER=yes' -r perl-threaded-5.10.1
(FORCE_PKG_REGISTER seems to be needed to keep libtool22 from erroring
and stopping the portmaster run.)
The second command goes OK for a while, but eventually I start getting
grep errors like those below, and ports like libtool22, libiconv, and
gettext are repeatedly reinstalled (with some other ports occasionally
reinstalled). Also, a couple times, portmaster died due to these 'grep
argument list too long' errors (hence my usage of -R).
...
===> Registering installation for libtool-2.2.6a_1
===> Cleaning for libtool-2.2.6a_1
/usr/local/sbin/portmaster: egrep: Argument list too long
===>>> Installation of devel/libtool22 (libtool-2.2.6a_1) succeeded
===>>> Returning to dependency check for converters/libiconv
===>>> Dependency check complete for converters/libiconv
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> devel/gmake >> devel/gettext >> converters/libiconv
===> Cleaning for libiconv-1.13.1
...
===>>> Starting check for runtime dependencies
===>>> Gathering dependency list for devel/gettext from ports
===>>> Starting dependency check
===>>> Checking dependency: converters/libiconv
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update converters/libiconv
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> devel/gmake >> devel/gettext >> converters/libiconv
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/converters/libiconv
===>>> Starting check for build dependencies
===>>> Gathering dependency list for converters/libiconv from ports
===>>> Starting dependency check
===>>> Checking dependency: devel/libtool22
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update devel/libtool22
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> devel/gmake >> devel/gettext >> converters/libiconv >> devel/libtool22
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/devel/libtool22
===>>> Starting check for build dependencies
===>>> Gathering dependency list for devel/libtool22 from ports
===>>> No dependencies for devel/libtool22
===> Cleaning for libtool-2.2.6a_1
...
===> Registering installation for gettext-0.17_1
===> Cleaning for gettext-0.17_1
/usr/local/sbin/portmaster: egrep: Argument list too long
===>>> Installation of devel/gettext (gettext-0.17_1) succeeded
===>>> Returning to dependency check for devel/gmake
===>>> Dependency check complete for devel/gmake
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> devel/gmake
===> Installing for gmake-3.81_3
...
===> Cleaning for pkg-config-0.23_1
/usr/local/sbin/portmaster: egrep: Argument list too long
===>>> No installed ports depend on pkg-config-0.23_1
===>>> Emptying +REQUIRED_BY file. Try portmaster -s
===>>> Installation of devel/pkg-config (pkg-config-0.23_1) succeeded
===>>> Returning to dependency check for dns/libidn
===>>> Checking dependency: lang/perl5.10
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update lang/perl5.10
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/lang/perl5.10
===>>> Starting check for build dependencies
===>>> Gathering dependency list for lang/perl5.10 from ports
===>>> Starting dependency check
===>>> Checking dependency: databases/gdbm
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update databases/gdbm
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10 >> databases/gdbm
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/databases/gdbm
===>>> Starting check for build dependencies
===>>> Gathering dependency list for databases/gdbm from ports
===>>> Starting dependency check
===>>> Checking dependency: devel/gmake
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update devel/gmake
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10 >> databases/gdbm >> devel/gmake
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/devel/gmake
===>>> Starting check for build dependencies
===>>> Gathering dependency list for devel/gmake from ports
===>>> Starting dependency check
===>>> Checking dependency: devel/gettext
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update devel/gettext
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10 >> databases/gdbm >> devel/gmake >> devel/gettext
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/devel/gettext
===>>> Starting check for build dependencies
===>>> Gathering dependency list for devel/gettext from ports
===>>> Starting dependency check
===>>> Checking dependency: converters/libiconv
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update converters/libiconv
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10 >> databases/gdbm >> devel/gmake >> devel/gettext >> converters/libiconv
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/converters/libiconv
===>>> Starting check for build dependencies
===>>> Gathering dependency list for converters/libiconv from ports
===>>> Starting dependency check
===>>> Checking dependency: devel/libtool22
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Launching child to update devel/libtool22
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10 >> databases/gdbm >> devel/gmake >> devel/gettext >> converters/libiconv >> devel/libtool22
/usr/local/sbin/portmaster: grep: Argument list too long
===>>> Port directory: /usr/ports/devel/libtool22
===>>> Starting check for build dependencies
===>>> Gathering dependency list for devel/libtool22 from ports
===>>> No dependencies for devel/libtool22
===> Cleaning for libtool-2.2.6a_1
===> Extracting for libtool-2.2.6a_1
....
I don't think I have an excessive number of ports installed, but I do
have all of x11/gnome2 and x11/kde4:
% pkg_info | wc -l
845
% portmaster --version
===>>> FreeBSD version 2.12
% uname -a
FreeBSD gauss.kenyonralph.com 8.0-RC1 FreeBSD 8.0-RC1 #0 r197973: Sun Oct 11 18:48:41 PDT 2009 root@gauss.kenyonralph.com:/usr/obj/usr/src/sys/GAUSS amd64
I'm not sure exactly where the grep argument list too long errors are
coming from, but there's not that many calls to grep in portmaster:
% grep -n -E '.?grep' =portmaster
75: while ps -axo pid,ppid,command | grep -v egrep | egrep -q "(make -DBATCH checksum|/fetch |\[sh\])"; do
129: grep -q '%%%%%%%%%%%%' $DI_FILES || kill_bad_children
197: [ -n "$grep_deps" ] && pm_unlink $grep_deps
387: dir=`grep -l "@comment ORIGIN:${1}$" $pdb/*/+CONTENTS`
397: o=`grep '@comment ORIGIN:' $pdb/$1/+CONTENTS 2>/dev/null`
406: # Global: grep_deps
411: # egrep hates + in file names
418: # Always rely on the grep'ed dependencies instead of +REQUIRED_BY
419: grep_deps=`pm_mktemp grep-deps-${iport}`
420: egrep -l "DEPORIGIN:($origin|$ro_opd)$" $pdb/*/+CONTENTS |
421: cut -f 5 -d '/' | sort -u > $grep_deps
423: if [ ! -s "$grep_deps" ]; then
541: for l in `grep "^$sf|" $pd/MOVED`; do
619: grep '^MD5' $distinfo | while read disc1 f disc2; do
647: if grep -ql '^@pkgdep ' $pkg/+CONTENTS 2>/dev/null; then
653: if grep -ql '^@pkgdep ' $pkg/+CONTENTS 2>/dev/null; then
674: if ! grep -ql $f $DI_FILES; then
724: if [ -s "$grep_deps" ]; then
726: sort $pkg/+REQUIRED_BY | cmp -s $grep_deps - ||
735: pm_install_s $grep_deps $pkg/+REQUIRED_BY
738: [ -n "$grep_deps" ] && {
739: pm_unlink $grep_deps && unset grep_deps; }
871: if egrep -ql '^(FORBIDDEN|DEPRECATED|BROKEN|IGNORE)' Makefile; then
1001: local pkgrep
1007: pkgrep=`pm_make -f/usr/share/mk/bsd.port.mk -V PKGREPOSITORY`
1008: if [ ! "$packages" = "${pkgrep%/All}" ]; then
1062: # The grep is needed to allow for comments, etc.
1063: for file in `grep ^DISTFILE $dist_list`; do
1076: for file in `grep ^DISTFILE $dist_list`; do
1126: grep -ql ${ps}$file $DI_FILES && continue
1356: deplist=`grep -l DEPORIGIN:$origin$ $pdb/*/+CONTENTS`
1394: deplist=`grep -l DEPORIGIN:$origin$ $pdb/*/+CONTENTS`
1511: nf=`ps -ax | grep [f]etch | wc -l`
1614: grep -ql ^CONFLICTS Makefile &&
1692: for req_by in `grep -l DEPORIGIN:$portdir$ $pdb/*/+CONTENTS | \
1981: if grep -ql "DEPORIGIN:$ro_opd$" $pdb/*/+CONTENTS; then
2127: TESTINT=`grep -l ^IS_INTERACTIVE Makefile`
2264: grep -v ^$LOCALBASE_COMPAT > $ldconfig_out
2387: size=`grep "^SIZE (${ds}${file})" $distinfo`
2388: sha256=`grep "^SHA256 (${ds}${file})" $distinfo`
2389: md5=`grep "^MD5 (${ds}${file})" $distinfo`
2412:if [ -s "$grep_deps" ]; then
2421: grep -ql "DEPORIGIN:$ro_opd$" $dp_cont &&
2426: if grep -ql "DEPORIGIN:$portdir$" $dp_cont; then
2429: done < $grep_deps
2433: sort $pdb/$new_port/+REQUIRED_BY | cmp -s $grep_deps - ||
2442: pm_install_s $grep_deps $pdb/$new_port/+REQUIRED_BY
2445: pm_unlink $grep_deps && unset grep_deps do_update
I'm thinking some of the grep calls should be replaced with find -exec
or find | xargs types of calls? Or am I doing something crazy and wrong
here? Maybe it would just be better to delete all ports, and start from
scratch with perl5.10 installed at the start?
Also, this may be a separate issue, but looks like portmaster is going
into a circle (this can also be seen in the snippets above):
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10
...
perl-threaded-5.10.1 >> kde4-4.3.1 >> kdeutils-4.3.1_1 >> security/gnupg >> ftp/curl >> dns/libidn >> lang/perl5.10 >> databases/gdbm >> devel/gmake >> devel/gettext >> converters/libiconv
(Meanwhile, an important user of perl, vim, still hasn't been rebuilt.)
Thanks,
Kenyon Ralph
--b0op/nKJ9CeIhp9z
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: Digital signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
iEYEARECAAYFAkrb4HUACgkQmFtUtJKnbnVIcgCgyrE3Hb7nBjy7wbz4XJe4+Zs+
JVAAoNTjcGVS/NAYP8AJZLIvluaDkEH/
=8eQH
-----END PGP SIGNATURE-----
--b0op/nKJ9CeIhp9z--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091019034349.GM24097>