Date: Tue, 17 Aug 2004 11:10:30 +0300 From: Ruslan Ermilov <ru@FreeBSD.org> To: Poul-Henning Kamp <phk@FreeBSD.org>, "David O'Brien" <obrien@FreeBSD.org> Cc: current@FreeBSD.org Subject: Fully parallelized "make universe" Message-ID: <20040817081030.GC17578@ip.net.ua>
next in thread | raw e-mail | index | archive | help
--tEFtbjk+mNEviIIX
Content-Type: multipart/mixed; boundary="uh9ZiVrAOUUm9fzH"
Content-Disposition: inline
--uh9ZiVrAOUUm9fzH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
Poul-Henning Kamp wrote:
> David O'Brien writes:
> >Actually I'd really love for a way to do the outter loop in parallel.
> >Due to -j messing up error output too much, I'd like to have the work for
> >each platform going in parallel instead. E.g.:
> >
> > make universe for i386 only &
> > make universe for powerpc only &
> > make universe for sparc64 only &
> > make universe for amd64 only &
> > make universe for alpha only &
> > make universe for ia64 only &
>=20
> True, but I can't get that working no matter how much I've tried.
>=20
OK, here it comes. The attached patch unrolls the main universe
loop, making it possible for "universe" subtargets corresponding
to individual architectures to be built in parallel.
EXAMPLES
make -j4 universe
will build four architectures in parallel, each will be
built using four parallel make's.
make -j8 universe JFLAG=3D-j4
will build eight architectures in parallel, each will be
built using four parallel make's.
make -j8 universe JFLAG=3D-B
will build eight architectures in parallel, each will be
built using one make.
Please test.
Points considered: since "buildkernels" is implemented using the
make's .for loop, kernels for a given architecture are built
sequentially (each can still be built in parallel). I didn't see
much point in parallelizing this task, but this is easily doable.
(One redundant JFLAG when calling buildkernels was removed by
the patch.)
Cheers,
--=20
Ruslan Ermilov
ru@FreeBSD.org
FreeBSD committer
--uh9ZiVrAOUUm9fzH
Content-Type: text/plain; charset=us-ascii
Content-Disposition: attachment; filename=p
Content-Transfer-Encoding: quoted-printable
Index: Makefile
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
RCS file: /home/ncvs/src/Makefile,v
retrieving revision 1.306
diff -u -r1.306 Makefile
--- Makefile 9 Aug 2004 11:38:41 -0000 1.306
+++ Makefile 17 Aug 2004 07:54:43 -0000
@@ -144,10 +144,9 @@
# Set a reasonable default
.MAIN: all
=20
-.if defined(HISTORICAL_MAKE_WORLD) || defined(DESTDIR)
-.if make(world)
STARTTIME!=3D LC_ALL=3DC date
-.endif
+
+.if defined(HISTORICAL_MAKE_WORLD) || defined(DESTDIR)
#
# world
#
@@ -242,16 +241,20 @@
# universe
#
# Attempt to rebuild *everything* for all supported architectures,
-# with reasonable chance of success, regardless of how old your
+# with a reasonable chance of success, regardless of how old your
# existing system is.
#
-i386_mach=3D pc98
-universe:
+universe: universe_prologue
+universe_prologue:
@echo "--------------------------------------------------------------"
@echo ">>> make universe started on ${STARTTIME}"
@echo "--------------------------------------------------------------"
-.for arch in i386 sparc64 alpha ia64 amd64
-.for mach in ${arch} ${${arch}_mach}
+.for target in i386 i386:pc98 sparc64 alpha ia64 amd64
+.for arch in ${target:C/:.*$//}
+.for mach in ${target:C/^.*://}
+universe: universe_${mach}
+.ORDER: universe_prologue universe_${mach} universe_epilogue
+universe_${mach}:
@echo ">> ${mach} started on `LC_ALL=3DC date`"
-cd ${.CURDIR} && ${MAKE} ${JFLAG} buildworld \
TARGET_ARCH=3D${arch} TARGET=3D${mach} \
@@ -262,10 +265,13 @@
-cd ${.CURDIR}/sys/${mach}/conf && ${MAKE} LINT \
> ${.CURDIR}/_.${mach}.makeLINT 2>&1
.endif
- cd ${.CURDIR} && ${MAKE} ${JFLAG} buildkernels TARGET_ARCH=3D${arch} TARG=
ET=3D${mach}
+ cd ${.CURDIR} && ${MAKE} buildkernels TARGET_ARCH=3D${arch} TARGET=3D${ma=
ch}
@echo ">> ${mach} completed on `LC_ALL=3DC date`"
.endfor
.endfor
+.endfor
+universe: universe_epilogue
+universe_epilogue:
@echo "--------------------------------------------------------------"
@echo ">>> make universe completed on `LC_ALL=3DC date`"
@echo " (started ${STARTTIME})"
@@ -274,8 +280,6 @@
.if make(buildkernels)
KERNCONFS!=3D cd ${.CURDIR}/sys/${TARGET}/conf && \
find [A-Z]*[A-Z] -type f -maxdepth 0 ! -name NOTES
-.endif
-
buildkernels:
.for kernel in ${KERNCONFS}
-cd ${.CURDIR} && ${MAKE} ${JFLAG} buildkernel \
@@ -283,3 +287,4 @@
__MAKE_CONF=3D/dev/null \
> _.${TARGET}.${kernel} 2>&1
.endfor
+.endif
--uh9ZiVrAOUUm9fzH--
--tEFtbjk+mNEviIIX
Content-Type: application/pgp-signature
Content-Disposition: inline
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.5 (FreeBSD)
iD8DBQFBIb12qRfpzJluFF4RArzaAKCRFRH9fOlMwHuJ9dtUbyTvhwPmhACeMS0J
vupi+D+XG1x+3ZtlFTDpefw=
=utmc
-----END PGP SIGNATURE-----
--tEFtbjk+mNEviIIX--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20040817081030.GC17578>