Date: Thu, 27 Mar 2014 11:24:52 -0600 From: Ian Lepore <ian@FreeBSD.org> To: Warner Losh <imp@bsdimp.com> Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Dimitry Andric <dim@FreeBSD.org> Subject: Re: svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin Message-ID: <1395941092.81853.115.camel@revolution.hippie.lan> In-Reply-To: <AA90F6B0-3A7A-473D-82C2-CFDFD263E9AC@gmail.com> References: <201403262230.s2QMUdH6021943@svn.freebsd.org> <AA90F6B0-3A7A-473D-82C2-CFDFD263E9AC@gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
It's also a bit odd that the way of opting in makes it appear as if you're opting out. -- Ian On Thu, 2014-03-27 at 11:05 -0600, Warner Losh wrote: > Why not have this ‘opt out’ rather than ‘opt in’ like it is now? Are there any known bad dependencies this introduces? > > Warner > > On Mar 26, 2014, at 4:30 PM, Dimitry Andric <dim@freebsd.org> wrote: > > > Author: dim > > Date: Wed Mar 26 22:30:38 2014 > > New Revision: 263778 > > URL: http://svnweb.freebsd.org/changeset/base/263778 > > > > Log: > > Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process > > all the SUBDIR entries in parallel, instead of serially. Apply this > > option to a selected number of Makefiles, which can greatly speed up the > > build on multi-core machines, when using make -j. > > > > This can be extended to more Makefiles later on, whenever they are > > verified to work correctly with parallel building. > > > > I tested this on a 24-core machine, with make -j48 buildworld (N = 6): > > > > before stddev after stddev > > ======= ====== ======= ====== > > real time 1741.1 16.5 959.8 2.7 > > user time 12468.7 16.4 14393.0 16.8 > > sys time 1825.0 54.8 2110.6 22.8 > > > > (user+sys)/real 8.2 17.1 > > > > E.g. the build was approximately 45% faster in real time. On machines > > with less cores, or with lower -j settings, the speedup will not be as > > impressive. But at least you can now almost max out a machine with > > buildworld! > > > > Submitted by: jilles > > MFC after: 2 weeks > > > > Modified: > > head/bin/Makefile > > head/lib/Makefile > > head/lib/clang/Makefile > > head/sbin/Makefile > > head/share/mk/bsd.subdir.mk > > head/usr.bin/Makefile > > head/usr.sbin/Makefile > > > > Modified: head/bin/Makefile > > ============================================================================== > > --- head/bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -60,4 +60,6 @@ SUBDIR+= tests > > > > SUBDIR:= ${SUBDIR:O} > > > > +SUBDIR_PARALLEL= > > + > > .include <bsd.subdir.mk> > > > > Modified: head/lib/Makefile > > ============================================================================== > > --- head/lib/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/lib/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -276,4 +276,8 @@ afterinstall: > > ${INSTALL_SYMLINK} ../include ${DESTDIR}/usr/lib/include > > .endif > > > > +.if !make(install) > > +SUBDIR_PARALLEL= > > +.endif > > + > > .include <bsd.subdir.mk> > > > > Modified: head/lib/clang/Makefile > > ============================================================================== > > --- head/lib/clang/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/lib/clang/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -147,4 +147,6 @@ SUBDIR+=liblldb \ > > > > SUBDIR+= include > > > > +SUBDIR_PARALLEL= > > + > > .include <bsd.subdir.mk> > > > > Modified: head/sbin/Makefile > > ============================================================================== > > --- head/sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -126,4 +126,6 @@ SUBDIR+= tests > > > > SUBDIR:= ${SUBDIR:O} > > > > +SUBDIR_PARALLEL= > > + > > .include <bsd.subdir.mk> > > > > Modified: head/share/mk/bsd.subdir.mk > > ============================================================================== > > --- head/share/mk/bsd.subdir.mk Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/share/mk/bsd.subdir.mk Wed Mar 26 22:30:38 2014 (r263778) > > @@ -71,7 +71,26 @@ ${SUBDIR}: .PHONY .MAKE > > .for __target in all all-man checkdpadd clean cleandepend cleandir \ > > cleanilinks depend distribute lint maninstall manlint obj objlink \ > > realinstall regress tags ${SUBDIR_TARGETS} > > +.ifdef SUBDIR_PARALLEL > > +.for __dir in ${SUBDIR} > > +${__target}: ${__target}_subdir_${__dir} > > +${__target}_subdir_${__dir}: .MAKE > > + @${_+_}set -e; \ > > + if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ > > + ${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \ > > + edir=${__dir}.${MACHINE_ARCH}; \ > > + cd ${.CURDIR}/$${edir}; \ > > + else \ > > + ${ECHODIR} "===> ${DIRPRFX}${__dir} (${__target:realinstall=install})"; \ > > + edir=${__dir}; \ > > + cd ${.CURDIR}/$${edir}; \ > > + fi; \ > > + ${MAKE} ${__target:realinstall=install} \ > > + DIRPRFX=${DIRPRFX}$$edir/ > > +.endfor > > +.else > > ${__target}: _SUBDIR > > +.endif > > .endfor > > > > .for __target in files includes > > > > Modified: head/usr.bin/Makefile > > ============================================================================== > > --- head/usr.bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/usr.bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -379,4 +379,6 @@ SUBDIR+= svn > > > > SUBDIR:= ${SUBDIR:O} > > > > +SUBDIR_PARALLEL= > > + > > .include <bsd.subdir.mk> > > > > Modified: head/usr.sbin/Makefile > > ============================================================================== > > --- head/usr.sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/usr.sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -320,4 +320,6 @@ SUBDIR+= wpa > > > > SUBDIR:= ${SUBDIR:O} > > > > +SUBDIR_PARALLEL= > > + > > .include <bsd.subdir.mk> > > > >
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1395941092.81853.115.camel>