From owner-freebsd-ports@FreeBSD.ORG Mon May 25 20:46:43 2009 Return-Path: Delivered-To: freebsd-ports@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C19C3106566B; Mon, 25 May 2009 20:46:43 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: from ey-out-2122.google.com (ey-out-2122.google.com [74.125.78.25]) by mx1.freebsd.org (Postfix) with ESMTP id B3C4B8FC0A; Mon, 25 May 2009 20:46:42 +0000 (UTC) (envelope-from naylor.b.david@gmail.com) Received: by ey-out-2122.google.com with SMTP id 9so692942eyd.7 for ; Mon, 25 May 2009 13:46:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:from:organization:to:subject :date:user-agent:cc:references:in-reply-to:mime-version:content-type :content-transfer-encoding:message-id; bh=zQqmO808ZLY+CpjMmmC5HmNqoEFgx6B0gDOf+Qmz+Fw=; b=Gppc3m50OZid1FLZxT1XJ6ZRkvCQpyhBkOOIL/2wq+kkcwOxq2sk+S5EmqmWpaJkTA xjwc5QkNke4015XAvoVjFyZaeWfHw3eZb6LmHBPWFgrDMytihRxOeVlRD/M8RMH1ZXuU jrAOlx+3s6eTfol4G80QQ49qvwrDYuJr/rlCc= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=from:organization:to:subject:date:user-agent:cc:references :in-reply-to:mime-version:content-type:content-transfer-encoding :message-id; b=sseXAOEd0DGBZeRhXAQzCnz8nUMnAtYtKUjvpJ4DFCJHq0NwjjMYcDj8j4jH5xFMlv 8ivawZPso6mx9wPmxCQ+lm+xQLzd2cZLqrX5rxfuOAzQfRfL1l+4k1GbBpTrASVOgxbf SB+qGvMO8JUe0Y2cOtfNJBblCCuhjeCew68Rg= Received: by 10.216.20.74 with SMTP id o52mr2221486weo.147.1243284400965; Mon, 25 May 2009 13:46:40 -0700 (PDT) Received: from ?0.0.0.0? ([196.34.241.123]) by mx.google.com with ESMTPS id 10sm2412127eyd.52.2009.05.25.13.46.38 (version=TLSv1/SSLv3 cipher=RC4-MD5); Mon, 25 May 2009 13:46:40 -0700 (PDT) From: David Naylor Organization: Private To: "Ion-Mihai Tetcu" Date: Mon, 25 May 2009 22:47:29 +0200 User-Agent: KMail/1.9.10 References: <20090523.182426.193741786.chat95@mac.com> <200905251003.15408.naylor.b.david@gmail.com> <20090525210125.1239c21e@it.buh.tecnik93.com> In-Reply-To: <20090525210125.1239c21e@it.buh.tecnik93.com> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart2294081.TG4sGDh83Q"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Message-Id: <200905252247.32725.naylor.b.david@gmail.com> Cc: pgollucci@p6m7g8.com, pav@freebsd.org, freebsd-ports@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) X-BeenThere: freebsd-ports@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting software to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2009 20:46:44 -0000 --nextPart2294081.TG4sGDh83Q Content-Type: multipart/mixed; boundary="Boundary-01=_hPwGKUb/hCYmdXs" Content-Transfer-Encoding: 7bit Content-Disposition: inline --Boundary-01=_hPwGKUb/hCYmdXs Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline On Monday 25 May 2009 20:01:25 Ion-Mihai Tetcu wrote: > On Mon, 25 May 2009 10:03:12 +0200 > > David Naylor wrote: > > On Sunday 24 May 2009 21:37:45 Ion-Mihai Tetcu wrote: > > > On Sun, 24 May 2009 10:26:23 +0200 > > > > > > David Naylor wrote: > > > > On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote: > > > > > Hi I tested it yesterday, > > > > > > > > > > 1. > > > > > I need > > > > > > > > > > > MAKE_JOBS_SAFE=3D yes > > > > > > > > > > in the Makefile. > > > > > > > > Yes, you would need that. I believe that will be default. > > > > > > > > > 2. with above patch, ooo2 doesn't launch parallele jobs. > > > > > > > > I spotted that problem after submitting the patch, if you > > > > explicitly set MAKE_JOBS_NUMBER to something it will work. > > > > > > > > The problem is that ooo2 does (in effect): > > > > .if (${MAKE_JOBS_NUMBER} > 1) > > > > # Stuff > > > > .else > > > > # Other stuff > > > > .endif > > > > and that doesn't work as expected with MAKE_JOBS_NUMBER=3D`sysctl > > > > kern.smp.cpus` as the command is not resolved. > > > > > > w/o patch > > > editors/openoffice.org-3 openoffice.org-3.1.0 4:53:27 > > > > > > with patch: > > > + MAKE_JOBS_SAFE=3D yes > > > + MAKE_JOBS_NUMBER=3D 4 > > > + MAXPROCESSES?=3D ${MAKE_JOBS_NUMBER} > > > + MAXMODULES?=3D ${MAKE_JOBS_NUMBER} > > > > > > editors/openoffice.org-3 openoffice.org-3.1.0 48:51 > > > > > > The build is done in > > > /dev/md0 on /usr/local/tinderbox/7-STABLE-FPT-NPD (ufs, > > > asynchronous, local, noatime) > > > > Wow, that is quite a speedup. Is it even possible (4 * 60 + 53)/4 =3D > > 73, and you get 48 (that is 152% scaling efficiency). This would > > mean a serious performance problem with the ooo3 build script and > > MAX* =3D1. > > > > I'll make a patch tonight (+10 hours) that will fix ooo2 in the > > default case. You can test ooo2 with patch and MAKE_JOBS_NUMBER > > preset (not using default value) and MAKE_JOBS_SAFE=3Dyes. > > BTW, what about using the same vars for parallel building in all OOo > port? Done, the following patch uses MAKE_JOBS_NUMBER for all the variables in OO= o. =20 It also tries to be efficient when resolving the MAKE_JOBS_NUMBER to a valu= e=20 (only done when a port sets USE_MAKE_JOBS, as in the OOo2-RC and OOo2 case)= =2E =20 This should fix OOo2* builds and support such use cases for other ports... --Boundary-01=_hPwGKUb/hCYmdXs Content-Type: text/x-diff; charset="iso 8859-15"; name="openoffice2.patch" Content-Transfer-Encoding: 7bit Content-Disposition: attachment; filename="openoffice2.patch" diff -ru /usr/ports/Mk/bsd.port.mk ports/Mk/bsd.port.mk --- /usr/ports/Mk/bsd.port.mk 2009-05-23 13:20:58.000000000 +0200 +++ ports/Mk/bsd.port.mk 2009-05-25 22:08:58.000000000 +0200 @@ -1361,6 +1361,19 @@ .include "${PORTSDIR}/Mk/bsd.linux-apps.mk" .endif +.if defined(USE_MAKE_JOBS) +.if defined(MAKE_JOBS_UNSAFE) +.error "USE_MAKE_JOBS requested yet port is marked as MAKE_JOBS_UNSAFE" +.endif +.if defined(DISABLE_MAKE_JOBS) +MAKE_JOBS_NUMBER= 1 +.elif defined(FORCE_MAKE_JOBS) || defined(MAKE_JOBS_SAFE) +.if !defined(MAKE_JOBS_NUMBER) +MAKE_JOBS_NUMBER!= ${SYSCTL} -n kern.smp.cpus +.endif +.endif +.endif + .if defined(X_WINDOW_SYSTEM) && ${X_WINDOW_SYSTEM:L} != "xorg" IGNORE= cannot be installed: bad X_WINDOW_SYSTEM setting; valid value is 'xorg' .endif @@ -2182,15 +2195,13 @@ # Multiple make jobs support .if defined(DISABLE_MAKE_JOBS) || defined(MAKE_JOBS_UNSAFE) +MAKE_JOBS_NUMBER= 1 _MAKE_JOBS= # .else .if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS) -.if defined(MAKE_JOBS_NUMBER) +MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus` _MAKE_JOBS= -j${MAKE_JOBS_NUMBER} -.else -_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus` -.endif -.if defined(FORCE_MAKE_JOBS) +.if defined(FORCE_MAKE_JOBS) && !defined(MAKE_JOBS_SAFE) BUILD_FAIL_MESSAGE+= "You have chosen to use multiple make jobs (parallelization) for all ports. This port was not tested for this setting. Please remove FORCE_MAKE_JOBS and retry the build before reporting the failure to the maintainer." .endif .endif diff -ru /usr/ports/editors/openoffice.org-2/Makefile ports/editors/openoffice.org-2/Makefile --- /usr/ports/editors/openoffice.org-2/Makefile 2009-01-25 10:45:45.000000000 +0200 +++ ports/editors/openoffice.org-2/Makefile 2009-05-25 22:10:21.000000000 +0200 @@ -48,9 +48,11 @@ USE_XORG= x11 ice xaw xau xext xrender xrandr \ xi xt xcursor xdamage xcomposite xfixes USE_GMAKE= yes +USE_MAKE_JOBS= yes USE_PERL5= yes USE_BZIP2= yes WITHOUT_CPU_CFLAGS= true +MAKE_JOBS_SAFE= yes .include @@ -132,7 +134,6 @@ CONFIGURE_WRKSRC= ${WRKSRC}/config_office TCSH?= /bin/tcsh PKGMESSAGE= ${WRKDIR}/pkg-message -NUMOFPROCESSES?= 1 CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \ --with-gnu-patch=${LOCALBASE}/bin/gpatch \ @@ -192,8 +193,8 @@ do-build: @cd ${WRKSRC} ; ./bootstrap # PR:84786 #i53289# -.if (${NUMOFPROCESSES}>1) - @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all" +.if (${MAKE_JOBS_NUMBER}>1) + @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all" .else @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; dmake" .endif diff -ru /usr/ports/editors/openoffice.org-2-RC/Makefile ports/editors/openoffice.org-2-RC/Makefile --- /usr/ports/editors/openoffice.org-2-RC/Makefile 2009-01-25 10:45:45.000000000 +0200 +++ ports/editors/openoffice.org-2-RC/Makefile 2009-05-25 22:05:54.000000000 +0200 @@ -49,9 +49,11 @@ USE_XORG= x11 ice xaw xau xext xrender xrandr \ xi xt xcursor xdamage xcomposite xfixes USE_GMAKE= yes +USE_MAKE_JOBS= yes USE_PERL5= yes USE_BZIP2= yes WITHOUT_CPU_CFLAGS= true +MAKE_JOBS_SAFE= yes .include @@ -134,7 +136,6 @@ CONFIGURE_WRKSRC= ${WRKSRC}/config_office TCSH?= /bin/tcsh PKGMESSAGE= ${WRKDIR}/pkg-message -NUMOFPROCESSES?= 1 CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \ --with-gnu-patch=${LOCALBASE}/bin/gpatch \ @@ -194,8 +195,8 @@ do-build: @cd ${WRKSRC} ; ./bootstrap # PR:84786 #i53289# -.if (${NUMOFPROCESSES}>1) - @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all" +.if (${MAKE_JOBS_NUMBER}>1) + @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all" .else @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; dmake" .endif diff -ru /usr/ports/editors/openoffice.org-2-devel/Makefile ports/editors/openoffice.org-2-devel/Makefile --- /usr/ports/editors/openoffice.org-2-devel/Makefile 2009-01-25 10:45:45.000000000 +0200 +++ ports/editors/openoffice.org-2-devel/Makefile 2009-05-25 22:11:53.000000000 +0200 @@ -52,6 +52,7 @@ USE_PERL5= yes USE_BZIP2= yes WITHOUT_CPU_CFLAGS= true +MAKE_JOBS_SAFE= yes .include @@ -137,8 +138,6 @@ CONFIGURE_WRKSRC= ${WRKSRC}/config_office TCSH?= /bin/tcsh PKGMESSAGE= ${WRKDIR}/pkg-message -MAXPROCESSES?= 1 -MAXMODULES?= 1 CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \ --with-gnu-patch=${LOCALBASE}/bin/gpatch \ @@ -200,7 +199,7 @@ do-build: @cd ${WRKSRC} ; ./bootstrap - @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}" + @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}" .if ${LOCALIZED_LANG} == "alllangs" @${MAKE} languagepack diff -ru /usr/ports/editors/openoffice.org-3/Makefile ports/editors/openoffice.org-3/Makefile --- /usr/ports/editors/openoffice.org-3/Makefile 2009-05-21 20:14:02.000000000 +0200 +++ ports/editors/openoffice.org-3/Makefile 2009-05-25 22:18:21.000000000 +0200 @@ -50,6 +50,7 @@ USE_PERL5= yes USE_BZIP2= yes WITHOUT_CPU_CFLAGS= true +MAKE_JOBS_SAFE= yes .include @@ -134,8 +135,6 @@ WRKSRC?= ${WRKDIR}/${OOOTAG} TCSH?= /bin/tcsh PKGMESSAGE= ${WRKDIR}/pkg-message -MAXPROCESSES?= 1 -MAXMODULES?= 1 CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \ --with-gnu-patch=${LOCALBASE}/bin/gpatch \ @@ -201,7 +201,7 @@ do-build: @cd ${WRKSRC} ; ./bootstrap - @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}" + @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}" .if ${LOCALIZED_LANG} == "alllangs" @${MAKE} languagepack diff -ru /usr/ports/editors/openoffice.org-3-RC/Makefile ports/editors/openoffice.org-3-RC/Makefile --- /usr/ports/editors/openoffice.org-3-RC/Makefile 2009-05-21 20:14:06.000000000 +0200 +++ ports/editors/openoffice.org-3-RC/Makefile 2009-05-25 22:13:08.000000000 +0200 @@ -50,6 +50,7 @@ USE_PERL5= yes USE_BZIP2= yes WITHOUT_CPU_CFLAGS= true +MAKE_JOBS_SAFE= yes .include @@ -135,8 +136,6 @@ WRKSRC?= ${WRKDIR}/${OOOTAG} TCSH?= /bin/tcsh PKGMESSAGE= ${WRKDIR}/pkg-message -MAXPROCESSES?= 1 -MAXMODULES?= 1 USE_AUTOTOOLS= autoconf:262 CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \ @@ -203,7 +203,7 @@ do-build: @cd ${WRKSRC} ; ./bootstrap - @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}" + @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}" .if ${LOCALIZED_LANG} == "alllangs" @${MAKE} languagepack diff -ru /usr/ports/editors/openoffice.org-3-devel/Makefile ports/editors/openoffice.org-3-devel/Makefile --- /usr/ports/editors/openoffice.org-3-devel/Makefile 2009-05-25 09:23:28.000000000 +0200 +++ ports/editors/openoffice.org-3-devel/Makefile 2009-05-25 22:16:59.000000000 +0200 @@ -48,6 +48,7 @@ USE_PERL5= yes USE_BZIP2= yes WITHOUT_CPU_CFLAGS= true +MAKE_JOBS_SAFE= yes .include @@ -133,8 +134,6 @@ WRKSRC?= ${WRKDIR}/${OOOTAG} TCSH?= /bin/tcsh PKGMESSAGE= ${WRKDIR}/pkg-message -MAXPROCESSES?= 1 -MAXMODULES?= 1 CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \ --with-gnu-patch=${LOCALBASE}/bin/gpatch \ @@ -200,7 +200,7 @@ do-build: @cd ${WRKSRC} ; ./bootstrap - @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}" + @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}" .if ${LOCALIZED_LANG} == "alllangs" @${MAKE} languagepack --Boundary-01=_hPwGKUb/hCYmdXs-- --nextPart2294081.TG4sGDh83Q Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEABECAAYFAkobA+QACgkQUaaFgP9pFrK1igCcDIYHAasV4C2AeZCXkohvvzKU FEsAnjjtaYkSZPPl55PjiBFLiJPiyXOh =fJjB -----END PGP SIGNATURE----- --nextPart2294081.TG4sGDh83Q--