From owner-svn-src-head@FreeBSD.ORG Wed Mar 26 22:30:40 2014 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id E17A7861; Wed, 26 Mar 2014 22:30:40 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id CD4A86BC; Wed, 26 Mar 2014 22:30:40 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.8/8.14.8) with ESMTP id s2QMUeZ9021951; Wed, 26 Mar 2014 22:30:40 GMT (envelope-from dim@svn.freebsd.org) Received: (from dim@localhost) by svn.freebsd.org (8.14.8/8.14.8/Submit) id s2QMUdH6021943; Wed, 26 Mar 2014 22:30:39 GMT (envelope-from dim@svn.freebsd.org) Message-Id: <201403262230.s2QMUdH6021943@svn.freebsd.org> From: Dimitry Andric Date: Wed, 26 Mar 2014 22:30:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Mar 2014 22:30:41 -0000 Author: dim Date: Wed Mar 26 22:30:38 2014 New Revision: 263778 URL: http://svnweb.freebsd.org/changeset/base/263778 Log: Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to process all the SUBDIR entries in parallel, instead of serially. Apply this option to a selected number of Makefiles, which can greatly speed up the build on multi-core machines, when using make -j. This can be extended to more Makefiles later on, whenever they are verified to work correctly with parallel building. I tested this on a 24-core machine, with make -j48 buildworld (N = 6): before stddev after stddev ======= ====== ======= ====== real time 1741.1 16.5 959.8 2.7 user time 12468.7 16.4 14393.0 16.8 sys time 1825.0 54.8 2110.6 22.8 (user+sys)/real 8.2 17.1 E.g. the build was approximately 45% faster in real time. On machines with less cores, or with lower -j settings, the speedup will not be as impressive. But at least you can now almost max out a machine with buildworld! Submitted by: jilles MFC after: 2 weeks Modified: head/bin/Makefile head/lib/Makefile head/lib/clang/Makefile head/sbin/Makefile head/share/mk/bsd.subdir.mk head/usr.bin/Makefile head/usr.sbin/Makefile Modified: head/bin/Makefile ============================================================================== --- head/bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -60,4 +60,6 @@ SUBDIR+= tests SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include Modified: head/lib/Makefile ============================================================================== --- head/lib/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/lib/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -276,4 +276,8 @@ afterinstall: ${INSTALL_SYMLINK} ../include ${DESTDIR}/usr/lib/include .endif +.if !make(install) +SUBDIR_PARALLEL= +.endif + .include Modified: head/lib/clang/Makefile ============================================================================== --- head/lib/clang/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/lib/clang/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -147,4 +147,6 @@ SUBDIR+=liblldb \ SUBDIR+= include +SUBDIR_PARALLEL= + .include Modified: head/sbin/Makefile ============================================================================== --- head/sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -126,4 +126,6 @@ SUBDIR+= tests SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include Modified: head/share/mk/bsd.subdir.mk ============================================================================== --- head/share/mk/bsd.subdir.mk Wed Mar 26 20:43:40 2014 (r263777) +++ head/share/mk/bsd.subdir.mk Wed Mar 26 22:30:38 2014 (r263778) @@ -71,7 +71,26 @@ ${SUBDIR}: .PHONY .MAKE .for __target in all all-man checkdpadd clean cleandepend cleandir \ cleanilinks depend distribute lint maninstall manlint obj objlink \ realinstall regress tags ${SUBDIR_TARGETS} +.ifdef SUBDIR_PARALLEL +.for __dir in ${SUBDIR} +${__target}: ${__target}_subdir_${__dir} +${__target}_subdir_${__dir}: .MAKE + @${_+_}set -e; \ + if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ + ${ECHODIR} "===> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=install})"; \ + edir=${__dir}.${MACHINE_ARCH}; \ + cd ${.CURDIR}/$${edir}; \ + else \ + ${ECHODIR} "===> ${DIRPRFX}${__dir} (${__target:realinstall=install})"; \ + edir=${__dir}; \ + cd ${.CURDIR}/$${edir}; \ + fi; \ + ${MAKE} ${__target:realinstall=install} \ + DIRPRFX=${DIRPRFX}$$edir/ +.endfor +.else ${__target}: _SUBDIR +.endif .endfor .for __target in files includes Modified: head/usr.bin/Makefile ============================================================================== --- head/usr.bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/usr.bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -379,4 +379,6 @@ SUBDIR+= svn SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include Modified: head/usr.sbin/Makefile ============================================================================== --- head/usr.sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) +++ head/usr.sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) @@ -320,4 +320,6 @@ SUBDIR+= wpa SUBDIR:= ${SUBDIR:O} +SUBDIR_PARALLEL= + .include