Date: Wed, 26 Mar 2014 22:30:39 +0000 (UTC) From: Dimitry Andric <dim@FreeBSD.org> To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin Message-ID: <201403262230.s2QMUdH6021943@svn.freebsd.org>
next in thread | raw e-mail | index | archive | help
Author: dim Date: Wed Mar 26 22:30:38 2014 New Revision: 263778 URL: http://svnweb.freebsd.org/changeset/base/263778 Log: Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process all the SUBDIR entries in parallel, instead of serially. Apply this option to a selected number of Makefiles, which can greatly speed up the build on multi-core machines, when using make -j. This can be extended to more Makefiles later on, whenever they are verified to work correctly with parallel building. I tested this on a 24-core machine, with make -j48 buildworld (N = 6): before stddev after stddev ======= ====== ======= ====== real time 1741.1 16.5 959.8 2.7 user time 12468.7 16.4 14393.0 16.8 sys time 1825.0 54.8 2110.6 22.8 (user+sys)/real 8.2 17.1 E.g. the build was approximately 45% faster in real time. On machines with less cores, or with lower -j settings, the speedup will not be as impressive. But at least you can now almost max out a machine with buildworld! Submitted by: jilles MFC after: 2 weeks Modified: head/bin/Makefile head/lib/Makefile head/lib/clang/Makefile head/sbin/Makefile head/share/mk/bsd.subdir.mk head/usr.bin/Makefile head/usr.sbin/Makefile Modified: head/bin/Makefile ============================================================================== --- head/bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -60,4 +60,6 @@ SUBDIR+= tests SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: head/lib/Makefile ============================================================================== --- head/lib/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/lib/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -276,4 +276,8 @@ afterinstall: ${INSTALL_SYMLINK} ../include ${DESTDIR}/usr/lib/include .endif +.if !make(install) +SUBDIR_PARALLEL= +.endif + .include <bsd.subdir.mk> Modified: head/lib/clang/Makefile ============================================================================== --- head/lib/clang/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/lib/clang/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -147,4 +147,6 @@ SUBDIR+=liblldb \ SUBDIR+= include +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: head/sbin/Makefile ============================================================================== --- head/sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -126,4 +126,6 @@ SUBDIR+= tests SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: head/share/mk/bsd.subdir.mk ============================================================================== --- head/share/mk/bsd.subdir.mk Wed Mar 26 20:43:40 2014 (r263777) +++ head/share/mk/bsd.subdir.mk Wed Mar 26 22:30:38 2014 (r263778) @@ -71,7 +71,26 @@ ${SUBDIR}: .PHONY .MAKE .for __target in all all-man checkdpadd clean cleandepend cleandir \ cleanilinks depend distribute lint maninstall manlint obj objlink \ realinstall regress tags ${SUBDIR_TARGETS} +.ifdef SUBDIR_PARALLEL +.for __dir in ${SUBDIR} +${__target}: ${__target}_subdir_${__dir} +${__target}_subdir_${__dir}: .MAKE + @${_+_}set -e; \ + if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ + ${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \ + edir=${__dir}.${MACHINE_ARCH}; \ + cd ${.CURDIR}/$${edir}; \ + else \ + ${ECHODIR} "===> ${DIRPRFX}${__dir} (${__target:realinstall=install})"; \ + edir=${__dir}; \ + cd ${.CURDIR}/$${edir}; \ + fi; \ + ${MAKE} ${__target:realinstall=install} \ + DIRPRFX=${DIRPRFX}$$edir/ +.endfor +.else ${__target}: _SUBDIR +.endif .endfor .for __target in files includes Modified: head/usr.bin/Makefile ============================================================================== --- head/usr.bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/usr.bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -379,4 +379,6 @@ SUBDIR+= svn SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include <bsd.subdir.mk> Modified: head/usr.sbin/Makefile ============================================================================== --- head/usr.sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/usr.sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -320,4 +320,6 @@ SUBDIR+= wpa SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include <bsd.subdir.mk>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201403262230.s2QMUdH6021943>