From owner-svn-src-head@FreeBSD.ORG Thu Mar 27 17:25:02 2014 Return-Path: Delivered-To: svn-src-head@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 4749E500; Thu, 27 Mar 2014 17:25:02 +0000 (UTC) Received: from mho-01-ewr.mailhop.org (mho-03-ewr.mailhop.org [204.13.248.66]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 04F33C4C; Thu, 27 Mar 2014 17:25:01 +0000 (UTC) Received: from c-24-8-230-52.hsd1.co.comcast.net ([24.8.230.52] helo=damnhippie.dyndns.org) by mho-01-ewr.mailhop.org with esmtpsa (TLSv1:AES256-SHA:256) (Exim 4.72) (envelope-from ) id 1WTE2x-0005v0-3V; Thu, 27 Mar 2014 17:24:55 +0000 Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id s2RHOq1U078904; Thu, 27 Mar 2014 11:24:52 -0600 (MDT) (envelope-from ian@FreeBSD.org) X-Mail-Handler: Dyn Standard SMTP by Dyn X-Originating-IP: 24.8.230.52 X-Report-Abuse-To: abuse@dyndns.com (see http://www.dyndns.com/services/sendlabs/outbound_abuse.html for abuse reporting information) X-MHO-User: U2FsdGVkX194mRqHL0ne6w3r7WYTMvtO Subject: Re: svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin From: Ian Lepore To: Warner Losh In-Reply-To: References: <201403262230.s2QMUdH6021943@svn.freebsd.org> Content-Type: text/plain; charset="windows-1251" Date: Thu, 27 Mar 2014 11:24:52 -0600 Message-ID: <1395941092.81853.115.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: quoted-printable X-MIME-Autoconverted: from 8bit to quoted-printable by damnhippie.dyndns.org id s2RHOq1U078904 Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Dimitry Andric X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 17:25:02 -0000 It's also a bit odd that the way of opting in makes it appear as if you're opting out. -- Ian On Thu, 2014-03-27 at 11:05 -0600, Warner Losh wrote: > Why not have this =91opt out=92 rather than =91opt in=92 like it is now= ? Are there any known bad dependencies this introduces? >=20 > Warner >=20 > On Mar 26, 2014, at 4:30 PM, Dimitry Andric wrote: >=20 > > Author: dim > > Date: Wed Mar 26 22:30:38 2014 > > New Revision: 263778 > > URL: http://svnweb.freebsd.org/changeset/base/263778 > >=20 > > Log: > > Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to proc= ess > > all the SUBDIR entries in parallel, instead of serially. Apply this > > option to a selected number of Makefiles, which can greatly speed up= the > > build on multi-core machines, when using make -j. > >=20 > > This can be extended to more Makefiles later on, whenever they are > > verified to work correctly with parallel building. > >=20 > > I tested this on a 24-core machine, with make -j48 buildworld (N =3D= 6): > >=20 > > before stddev after stddev > > =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D =3D= =3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D > > real time 1741.1 16.5 959.8 2.7 > > user time 12468.7 16.4 14393.0 16.8 > > sys time 1825.0 54.8 2110.6 22.8 > >=20 > > (user+sys)/real 8.2 17.1 > >=20 > > E.g. the build was approximately 45% faster in real time. On machin= es > > with less cores, or with lower -j settings, the speedup will not be = as > > impressive. But at least you can now almost max out a machine with > > buildworld! > >=20 > > Submitted by: jilles > > MFC after: 2 weeks > >=20 > > Modified: > > head/bin/Makefile > > head/lib/Makefile > > head/lib/clang/Makefile > > head/sbin/Makefile > > head/share/mk/bsd.subdir.mk > > head/usr.bin/Makefile > > head/usr.sbin/Makefile > >=20 > > Modified: head/bin/Makefile > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -60,4 +60,6 @@ SUBDIR+=3D tests > >=20 > > SUBDIR:=3D ${SUBDIR:O} > >=20 > > +SUBDIR_PARALLEL=3D > > + > > .include > >=20 > > Modified: head/lib/Makefile > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/lib/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/lib/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -276,4 +276,8 @@ afterinstall: > > ${INSTALL_SYMLINK} ../include ${DESTDIR}/usr/lib/include > > .endif > >=20 > > +.if !make(install) > > +SUBDIR_PARALLEL=3D > > +.endif > > + > > .include > >=20 > > Modified: head/lib/clang/Makefile > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/lib/clang/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/lib/clang/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -147,4 +147,6 @@ SUBDIR+=3Dliblldb \ > >=20 > > SUBDIR+=3D include > >=20 > > +SUBDIR_PARALLEL=3D > > + > > .include > >=20 > > Modified: head/sbin/Makefile > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -126,4 +126,6 @@ SUBDIR+=3D tests > >=20 > > SUBDIR:=3D ${SUBDIR:O} > >=20 > > +SUBDIR_PARALLEL=3D > > + > > .include > >=20 > > Modified: head/share/mk/bsd.subdir.mk > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/share/mk/bsd.subdir.mk Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/share/mk/bsd.subdir.mk Wed Mar 26 22:30:38 2014 (r263778) > > @@ -71,7 +71,26 @@ ${SUBDIR}: .PHONY .MAKE > > .for __target in all all-man checkdpadd clean cleandepend cleandir \ > > cleanilinks depend distribute lint maninstall manlint obj objlink= \ > > realinstall regress tags ${SUBDIR_TARGETS} > > +.ifdef SUBDIR_PARALLEL > > +.for __dir in ${SUBDIR} > > +${__target}: ${__target}_subdir_${__dir} > > +${__target}_subdir_${__dir}: .MAKE > > + @${_+_}set -e; \ > > + if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ > > + ${ECHODIR} "=3D=3D=3D> ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__ta= rget:realinstall=3Dinstall})"; \ > > + edir=3D${__dir}.${MACHINE_ARCH}; \ > > + cd ${.CURDIR}/$${edir}; \ > > + else \ > > + ${ECHODIR} "=3D=3D=3D> ${DIRPRFX}${__dir} (${__target:realinstall= =3Dinstall})"; \ > > + edir=3D${__dir}; \ > > + cd ${.CURDIR}/$${edir}; \ > > + fi; \ > > + ${MAKE} ${__target:realinstall=3Dinstall} \ > > + DIRPRFX=3D${DIRPRFX}$$edir/ > > +.endfor > > +.else > > ${__target}: _SUBDIR > > +.endif > > .endfor > >=20 > > .for __target in files includes > >=20 > > Modified: head/usr.bin/Makefile > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/usr.bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/usr.bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -379,4 +379,6 @@ SUBDIR+=3D svn > >=20 > > SUBDIR:=3D ${SUBDIR:O} > >=20 > > +SUBDIR_PARALLEL=3D > > + > > .include > >=20 > > Modified: head/usr.sbin/Makefile > > =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D > > --- head/usr.sbin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > > +++ head/usr.sbin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > > @@ -320,4 +320,6 @@ SUBDIR+=3D wpa > >=20 > > SUBDIR:=3D ${SUBDIR:O} > >=20 > > +SUBDIR_PARALLEL=3D > > + > > .include > >=20 >=20 >=20