From owner-svn-src-all@FreeBSD.ORG Thu Mar 27 17:05:09 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id D6B4DAC8 for ; Thu, 27 Mar 2014 17:05:09 +0000 (UTC) Received: from mail-pd0-f169.google.com (mail-pd0-f169.google.com [209.85.192.169]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A86A2A67 for ; Thu, 27 Mar 2014 17:05:09 +0000 (UTC) Received: by mail-pd0-f169.google.com with SMTP id fp1so3637301pdb.28 for ; Thu, 27 Mar 2014 10:05:03 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=r4Ao21kA+0eBnLZF1BHneQwZB7UMcJwFgi70IBJPKi4=; b=SbIdfnRi6VAWCjdZxDbxW83zXlx6wmIUfE0SNVV9vUQdzQq6Z8oNwjOCb+PzNXfHG/ l2NDnk33XALn01XDtkJrCcLqmGL/f2pPI45Oz/BA9m72bVqMQYV91JV5odidta6AyMeS W2u2aHKJhw2wYtxXpmY2+XwAMWwG1Fu24o96bQIZUdCy3gaxb4hhxvuDzWIG2gvk2Bcr bu0vi06yShiDg7qXfCdORhh5+V8Km6t1jV/WaXBvMwOvZgtVy1Ud8MYCL0xyoHqEU/IA DMz4uEMerFUUVRNbzUvFGJr8AkxmiAZFNbanl61S1sFkiiQoTOg+Vke7efDCT3MwGGlg x9iA== X-Gm-Message-State: ALoCoQncHO5EVWAcQZLl+TuOin0qrL+HFBxNy6BS1shjWiuw/1l/mf08yCkntv61tMJLZn1nAkoR X-Received: by 10.67.1.202 with SMTP id bi10mr3076190pad.68.1395939903721; Thu, 27 Mar 2014 10:05:03 -0700 (PDT) Received: from [10.64.26.19] (dc1-prod.netflix.com. [69.53.236.251]) by mx.google.com with ESMTPSA id hw8sm11319542pbc.62.2014.03.27.10.05.02 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Thu, 27 Mar 2014 10:05:03 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=windows-1252 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin From: Warner Losh In-Reply-To: <201403262230.s2QMUdH6021943@svn.freebsd.org> Date: Thu, 27 Mar 2014 11:05:00 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201403262230.s2QMUdH6021943@svn.freebsd.org> To: Dimitry Andric X-Mailer: Apple Mail (2.1874) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Mar 2014 17:05:09 -0000 Why not have this =91opt out=92 rather than =91opt in=92 like it is now? = Are there any known bad dependencies this introduces? Warner On Mar 26, 2014, at 4:30 PM, Dimitry Andric wrote: > Author: dim > Date: Wed Mar 26 22:30:38 2014 > New Revision: 263778 > URL: http://svnweb.freebsd.org/changeset/base/263778 >=20 > Log: > Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to = process > all the SUBDIR entries in parallel, instead of serially. Apply this > option to a selected number of Makefiles, which can greatly speed up = the > build on multi-core machines, when using make -j. >=20 > This can be extended to more Makefiles later on, whenever they are > verified to work correctly with parallel building. >=20 > I tested this on a 24-core machine, with make -j48 buildworld (N =3D = 6): >=20 > before stddev after stddev > =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D = =3D=3D=3D=3D=3D=3D=3D =3D=3D=3D=3D=3D=3D > real time 1741.1 16.5 959.8 2.7 > user time 12468.7 16.4 14393.0 16.8 > sys time 1825.0 54.8 2110.6 22.8 >=20 > (user+sys)/real 8.2 17.1 >=20 > E.g. the build was approximately 45% faster in real time. On = machines > with less cores, or with lower -j settings, the speedup will not be = as > impressive. But at least you can now almost max out a machine with > buildworld! >=20 > Submitted by: jilles > MFC after: 2 weeks >=20 > Modified: > head/bin/Makefile > head/lib/Makefile > head/lib/clang/Makefile > head/sbin/Makefile > head/share/mk/bsd.subdir.mk > head/usr.bin/Makefile > head/usr.sbin/Makefile >=20 > Modified: head/bin/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/bin/Makefile Wed Mar 26 20:43:40 2014 (r263777) > +++ head/bin/Makefile Wed Mar 26 22:30:38 2014 (r263778) > @@ -60,4 +60,6 @@ SUBDIR+=3D tests >=20 > SUBDIR:=3D ${SUBDIR:O} >=20 > +SUBDIR_PARALLEL=3D > + > .include >=20 > Modified: head/lib/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/lib/Makefile Wed Mar 26 20:43:40 2014 (r263777) > +++ head/lib/Makefile Wed Mar 26 22:30:38 2014 (r263778) > @@ -276,4 +276,8 @@ afterinstall: > ${INSTALL_SYMLINK} ../include ${DESTDIR}/usr/lib/include > .endif >=20 > +.if !make(install) > +SUBDIR_PARALLEL=3D > +.endif > + > .include >=20 > Modified: head/lib/clang/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/lib/clang/Makefile Wed Mar 26 20:43:40 2014 = (r263777) > +++ head/lib/clang/Makefile Wed Mar 26 22:30:38 2014 = (r263778) > @@ -147,4 +147,6 @@ SUBDIR+=3Dliblldb \ >=20 > SUBDIR+=3D include >=20 > +SUBDIR_PARALLEL=3D > + > .include >=20 > Modified: head/sbin/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/sbin/Makefile Wed Mar 26 20:43:40 2014 = (r263777) > +++ head/sbin/Makefile Wed Mar 26 22:30:38 2014 = (r263778) > @@ -126,4 +126,6 @@ SUBDIR+=3D tests >=20 > SUBDIR:=3D ${SUBDIR:O} >=20 > +SUBDIR_PARALLEL=3D > + > .include >=20 > Modified: head/share/mk/bsd.subdir.mk > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/share/mk/bsd.subdir.mk Wed Mar 26 20:43:40 2014 = (r263777) > +++ head/share/mk/bsd.subdir.mk Wed Mar 26 22:30:38 2014 = (r263778) > @@ -71,7 +71,26 @@ ${SUBDIR}: .PHONY .MAKE > .for __target in all all-man checkdpadd clean cleandepend cleandir \ > cleanilinks depend distribute lint maninstall manlint obj objlink = \ > realinstall regress tags ${SUBDIR_TARGETS} > +.ifdef SUBDIR_PARALLEL > +.for __dir in ${SUBDIR} > +${__target}: ${__target}_subdir_${__dir} > +${__target}_subdir_${__dir}: .MAKE > + @${_+_}set -e; \ > + if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ > + ${ECHODIR} "=3D=3D=3D> = ${DIRPRFX}${__dir}.${MACHINE_ARCH} (${__target:realinstall=3Dinstall})"; = \ > + edir=3D${__dir}.${MACHINE_ARCH}; \ > + cd ${.CURDIR}/$${edir}; \ > + else \ > + ${ECHODIR} "=3D=3D=3D> ${DIRPRFX}${__dir} = (${__target:realinstall=3Dinstall})"; \ > + edir=3D${__dir}; \ > + cd ${.CURDIR}/$${edir}; \ > + fi; \ > + ${MAKE} ${__target:realinstall=3Dinstall} \ > + DIRPRFX=3D${DIRPRFX}$$edir/ > +.endfor > +.else > ${__target}: _SUBDIR > +.endif > .endfor >=20 > .for __target in files includes >=20 > Modified: head/usr.bin/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/usr.bin/Makefile Wed Mar 26 20:43:40 2014 = (r263777) > +++ head/usr.bin/Makefile Wed Mar 26 22:30:38 2014 = (r263778) > @@ -379,4 +379,6 @@ SUBDIR+=3D svn >=20 > SUBDIR:=3D ${SUBDIR:O} >=20 > +SUBDIR_PARALLEL=3D > + > .include >=20 > Modified: head/usr.sbin/Makefile > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/usr.sbin/Makefile Wed Mar 26 20:43:40 2014 = (r263777) > +++ head/usr.sbin/Makefile Wed Mar 26 22:30:38 2014 = (r263778) > @@ -320,4 +320,6 @@ SUBDIR+=3D wpa >=20 > SUBDIR:=3D ${SUBDIR:O} >=20 > +SUBDIR_PARALLEL=3D > + > .include >=20