Date: Mon, 25 May 2009 22:47:29 +0200 From: David Naylor <naylor.b.david@gmail.com> To: "Ion-Mihai Tetcu" <itetcu@freebsd.org> Cc: pgollucci@p6m7g8.com, pav@freebsd.org, freebsd-ports@freebsd.org Subject: Re: MAKE_JOBS_UNSAFE (some more ports) Message-ID: <200905252247.32725.naylor.b.david@gmail.com> In-Reply-To: <20090525210125.1239c21e@it.buh.tecnik93.com> References: <20090523.182426.193741786.chat95@mac.com> <200905251003.15408.naylor.b.david@gmail.com> <20090525210125.1239c21e@it.buh.tecnik93.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--nextPart2294081.TG4sGDh83Q
Content-Type: multipart/mixed;
boundary="Boundary-01=_hPwGKUb/hCYmdXs"
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
--Boundary-01=_hPwGKUb/hCYmdXs
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline
On Monday 25 May 2009 20:01:25 Ion-Mihai Tetcu wrote:
> On Mon, 25 May 2009 10:03:12 +0200
>
> David Naylor <naylor.b.david@gmail.com> wrote:
> > On Sunday 24 May 2009 21:37:45 Ion-Mihai Tetcu wrote:
> > > On Sun, 24 May 2009 10:26:23 +0200
> > >
> > > David Naylor <naylor.b.david@gmail.com> wrote:
> > > > On Sunday 24 May 2009 00:16:37 Maho NAKATA wrote:
> > > > > Hi I tested it yesterday,
> > > > >
> > > > > 1.
> > > > > I need
> > > > >
> > > > > > MAKE_JOBS_SAFE=3D yes
> > > > >
> > > > > in the Makefile.
> > > >
> > > > Yes, you would need that. I believe that will be default.
> > > >
> > > > > 2. with above patch, ooo2 doesn't launch parallele jobs.
> > > >
> > > > I spotted that problem after submitting the patch, if you
> > > > explicitly set MAKE_JOBS_NUMBER to something it will work.
> > > >
> > > > The problem is that ooo2 does (in effect):
> > > > .if (${MAKE_JOBS_NUMBER} > 1)
> > > > # Stuff
> > > > .else
> > > > # Other stuff
> > > > .endif
> > > > and that doesn't work as expected with MAKE_JOBS_NUMBER=3D`sysctl
> > > > kern.smp.cpus` as the command is not resolved.
> > >
> > > w/o patch
> > > editors/openoffice.org-3 openoffice.org-3.1.0 4:53:27
> > >
> > > with patch:
> > > + MAKE_JOBS_SAFE=3D yes
> > > + MAKE_JOBS_NUMBER=3D 4
> > > + MAXPROCESSES?=3D ${MAKE_JOBS_NUMBER}
> > > + MAXMODULES?=3D ${MAKE_JOBS_NUMBER}
> > >
> > > editors/openoffice.org-3 openoffice.org-3.1.0 48:51
> > >
> > > The build is done in
> > > /dev/md0 on /usr/local/tinderbox/7-STABLE-FPT-NPD (ufs,
> > > asynchronous, local, noatime)
> >
> > Wow, that is quite a speedup. Is it even possible (4 * 60 + 53)/4 =3D
> > 73, and you get 48 (that is 152% scaling efficiency). This would
> > mean a serious performance problem with the ooo3 build script and
> > MAX* =3D1.
> >
> > I'll make a patch tonight (+10 hours) that will fix ooo2 in the
> > default case. You can test ooo2 with patch and MAKE_JOBS_NUMBER
> > preset (not using default value) and MAKE_JOBS_SAFE=3Dyes.
>
> BTW, what about using the same vars for parallel building in all OOo
> port?
Done, the following patch uses MAKE_JOBS_NUMBER for all the variables in OO=
o. =20
It also tries to be efficient when resolving the MAKE_JOBS_NUMBER to a valu=
e=20
(only done when a port sets USE_MAKE_JOBS, as in the OOo2-RC and OOo2 case)=
=2E =20
This should fix OOo2* builds and support such use cases for other ports...
--Boundary-01=_hPwGKUb/hCYmdXs
Content-Type: text/x-diff; charset="iso 8859-15"; name="openoffice2.patch"
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename="openoffice2.patch"
diff -ru /usr/ports/Mk/bsd.port.mk ports/Mk/bsd.port.mk
--- /usr/ports/Mk/bsd.port.mk 2009-05-23 13:20:58.000000000 +0200
+++ ports/Mk/bsd.port.mk 2009-05-25 22:08:58.000000000 +0200
@@ -1361,6 +1361,19 @@
.include "${PORTSDIR}/Mk/bsd.linux-apps.mk"
.endif
+.if defined(USE_MAKE_JOBS)
+.if defined(MAKE_JOBS_UNSAFE)
+.error "USE_MAKE_JOBS requested yet port is marked as MAKE_JOBS_UNSAFE"
+.endif
+.if defined(DISABLE_MAKE_JOBS)
+MAKE_JOBS_NUMBER= 1
+.elif defined(FORCE_MAKE_JOBS) || defined(MAKE_JOBS_SAFE)
+.if !defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER!= ${SYSCTL} -n kern.smp.cpus
+.endif
+.endif
+.endif
+
.if defined(X_WINDOW_SYSTEM) && ${X_WINDOW_SYSTEM:L} != "xorg"
IGNORE= cannot be installed: bad X_WINDOW_SYSTEM setting; valid value is 'xorg'
.endif
@@ -2182,15 +2195,13 @@
# Multiple make jobs support
.if defined(DISABLE_MAKE_JOBS) || defined(MAKE_JOBS_UNSAFE)
+MAKE_JOBS_NUMBER= 1
_MAKE_JOBS= #
.else
.if defined(MAKE_JOBS_SAFE) || defined(FORCE_MAKE_JOBS)
-.if defined(MAKE_JOBS_NUMBER)
+MAKE_JOBS_NUMBER?= `${SYSCTL} -n kern.smp.cpus`
_MAKE_JOBS= -j${MAKE_JOBS_NUMBER}
-.else
-_MAKE_JOBS= -j`${SYSCTL} -n kern.smp.cpus`
-.endif
-.if defined(FORCE_MAKE_JOBS)
+.if defined(FORCE_MAKE_JOBS) && !defined(MAKE_JOBS_SAFE)
BUILD_FAIL_MESSAGE+= "You have chosen to use multiple make jobs (parallelization) for all ports. This port was not tested for this setting. Please remove FORCE_MAKE_JOBS and retry the build before reporting the failure to the maintainer."
.endif
.endif
diff -ru /usr/ports/editors/openoffice.org-2/Makefile ports/editors/openoffice.org-2/Makefile
--- /usr/ports/editors/openoffice.org-2/Makefile 2009-01-25 10:45:45.000000000 +0200
+++ ports/editors/openoffice.org-2/Makefile 2009-05-25 22:10:21.000000000 +0200
@@ -48,9 +48,11 @@
USE_XORG= x11 ice xaw xau xext xrender xrandr \
xi xt xcursor xdamage xcomposite xfixes
USE_GMAKE= yes
+USE_MAKE_JOBS= yes
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include <bsd.port.pre.mk>
@@ -132,7 +134,6 @@
CONFIGURE_WRKSRC= ${WRKSRC}/config_office
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-NUMOFPROCESSES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -192,8 +193,8 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
# PR:84786 #i53289#
-.if (${NUMOFPROCESSES}>1)
- @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all"
+.if (${MAKE_JOBS_NUMBER}>1)
+ @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all"
.else
@cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; dmake"
.endif
diff -ru /usr/ports/editors/openoffice.org-2-RC/Makefile ports/editors/openoffice.org-2-RC/Makefile
--- /usr/ports/editors/openoffice.org-2-RC/Makefile 2009-01-25 10:45:45.000000000 +0200
+++ ports/editors/openoffice.org-2-RC/Makefile 2009-05-25 22:05:54.000000000 +0200
@@ -49,9 +49,11 @@
USE_XORG= x11 ice xaw xau xext xrender xrandr \
xi xt xcursor xdamage xcomposite xfixes
USE_GMAKE= yes
+USE_MAKE_JOBS= yes
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include <bsd.port.pre.mk>
@@ -134,7 +136,6 @@
CONFIGURE_WRKSRC= ${WRKSRC}/config_office
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-NUMOFPROCESSES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -194,8 +195,8 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
# PR:84786 #i53289#
-.if (${NUMOFPROCESSES}>1)
- @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${NUMOFPROCESSES} --all"
+.if (${MAKE_JOBS_NUMBER}>1)
+ @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl -P${MAKE_JOBS_NUMBER} --all"
.else
@cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; dmake"
.endif
diff -ru /usr/ports/editors/openoffice.org-2-devel/Makefile ports/editors/openoffice.org-2-devel/Makefile
--- /usr/ports/editors/openoffice.org-2-devel/Makefile 2009-01-25 10:45:45.000000000 +0200
+++ ports/editors/openoffice.org-2-devel/Makefile 2009-05-25 22:11:53.000000000 +0200
@@ -52,6 +52,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include <bsd.port.pre.mk>
@@ -137,8 +138,6 @@
CONFIGURE_WRKSRC= ${WRKSRC}/config_office
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-MAXPROCESSES?= 1
-MAXMODULES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -200,7 +199,7 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
- @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}"
+ @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; setenv TMP ${WRKSRC} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}"
.if ${LOCALIZED_LANG} == "alllangs"
@${MAKE} languagepack
diff -ru /usr/ports/editors/openoffice.org-3/Makefile ports/editors/openoffice.org-3/Makefile
--- /usr/ports/editors/openoffice.org-3/Makefile 2009-05-21 20:14:02.000000000 +0200
+++ ports/editors/openoffice.org-3/Makefile 2009-05-25 22:18:21.000000000 +0200
@@ -50,6 +50,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include <bsd.port.pre.mk>
@@ -134,8 +135,6 @@
WRKSRC?= ${WRKDIR}/${OOOTAG}
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-MAXPROCESSES?= 1
-MAXMODULES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -201,7 +201,7 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
- @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}"
+ @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}"
.if ${LOCALIZED_LANG} == "alllangs"
@${MAKE} languagepack
diff -ru /usr/ports/editors/openoffice.org-3-RC/Makefile ports/editors/openoffice.org-3-RC/Makefile
--- /usr/ports/editors/openoffice.org-3-RC/Makefile 2009-05-21 20:14:06.000000000 +0200
+++ ports/editors/openoffice.org-3-RC/Makefile 2009-05-25 22:13:08.000000000 +0200
@@ -50,6 +50,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include <bsd.port.pre.mk>
@@ -135,8 +136,6 @@
WRKSRC?= ${WRKDIR}/${OOOTAG}
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-MAXPROCESSES?= 1
-MAXMODULES?= 1
USE_AUTOTOOLS= autoconf:262
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
@@ -203,7 +203,7 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
- @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}"
+ @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}"
.if ${LOCALIZED_LANG} == "alllangs"
@${MAKE} languagepack
diff -ru /usr/ports/editors/openoffice.org-3-devel/Makefile ports/editors/openoffice.org-3-devel/Makefile
--- /usr/ports/editors/openoffice.org-3-devel/Makefile 2009-05-25 09:23:28.000000000 +0200
+++ ports/editors/openoffice.org-3-devel/Makefile 2009-05-25 22:16:59.000000000 +0200
@@ -48,6 +48,7 @@
USE_PERL5= yes
USE_BZIP2= yes
WITHOUT_CPU_CFLAGS= true
+MAKE_JOBS_SAFE= yes
.include <bsd.port.pre.mk>
@@ -133,8 +134,6 @@
WRKSRC?= ${WRKDIR}/${OOOTAG}
TCSH?= /bin/tcsh
PKGMESSAGE= ${WRKDIR}/pkg-message
-MAXPROCESSES?= 1
-MAXMODULES?= 1
CONFIGURE_ARGS+= --with-gnu-cp=${LOCALBASE}/bin/gcp \
--with-gnu-patch=${LOCALBASE}/bin/gpatch \
@@ -200,7 +200,7 @@
do-build:
@cd ${WRKSRC} ; ./bootstrap
- @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAXMODULES} --all --html --dontgraboutput -- -P${MAXPROCESSES}"
+ @cd ${WRKSRC} ; ${SETENV} "LANG=C" "LC_ALL=C" "TMP=${WRKSRC}" ${TCSH} -c "source ${FREEBSD_ENV_SET} ; cd instsetoo_native ; build.pl --checkmodules ; build.pl -P${MAKE_JOBS_NUMBER} --all --html --dontgraboutput -- -P${MAKE_JOBS_NUMBER}"
.if ${LOCALIZED_LANG} == "alllangs"
@${MAKE} languagepack
--Boundary-01=_hPwGKUb/hCYmdXs--
--nextPart2294081.TG4sGDh83Q
Content-Type: application/pgp-signature; name=signature.asc
Content-Description: This is a digitally signed message part.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (FreeBSD)
iEYEABECAAYFAkobA+QACgkQUaaFgP9pFrK1igCcDIYHAasV4C2AeZCXkohvvzKU
FEsAnjjtaYkSZPPl55PjiBFLiJPiyXOh
=fJjB
-----END PGP SIGNATURE-----
--nextPart2294081.TG4sGDh83Q--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200905252247.32725.naylor.b.david>