From owner-svn-src-all@FreeBSD.ORG Mon Apr 21 13:35:00 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id BC0C9974 for ; Mon, 21 Apr 2014 13:35:00 +0000 (UTC) Received: from mail-ig0-f175.google.com (mail-ig0-f175.google.com [209.85.213.175]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7AA99180C for ; Mon, 21 Apr 2014 13:35:00 +0000 (UTC) Received: by mail-ig0-f175.google.com with SMTP id h3so1761515igd.8 for ; Mon, 21 Apr 2014 06:34:59 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:sender:content-type:mime-version:subject:from :in-reply-to:date:cc:content-transfer-encoding:message-id:references :to; bh=EiGU/f9tBn0MHvIjKitXjs5bn7hZsUkHbbJafZsOfTg=; b=LC5CtwwxhHVnX86mq/LhKDqgYkW/7IDk5pI4VL5XgiI7l80GDfRxGhvL0x0sBoiHa+ bd+kSZ9aDUE7b0F0HPskF5PFEJjuub+Xz44AjSR6KqOVhCKUJc19lQVQoUKvQlI7H88o EjpBXTlDwD+yIxUCJZamZobgzpS47KN/4UI4zyTyV8bYOgCiJjYBhdykCx1yE2tfxMXA mgrYelobfnk94yqHTBcP+ygYEae4oKOljVM/HQm5Benuxr63puWZdfuiOSJ4Hz/61I4m aF/sAGfnnGTug74J5dd9VQg8eLOkqdi4LwOJ064XKjpXWmEhX34IvS5wHP/PCkMF8P3u /MZw== X-Gm-Message-State: ALoCoQnPRkS1lP8dcFyiaKB37T+FdIPl5q1Gp8V3XRcjSC37vxf3xOd5kaJpIqD8DWR0N4SbJOyu X-Received: by 10.42.176.131 with SMTP id be3mr30503577icb.2.1398087299350; Mon, 21 Apr 2014 06:34:59 -0700 (PDT) Received: from [192.168.43.111] ([172.56.8.60]) by mx.google.com with ESMTPSA id m8sm21135701igx.9.2014.04.21.06.34.57 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Mon, 21 Apr 2014 06:34:58 -0700 (PDT) Sender: Warner Losh Content-Type: text/plain; charset=iso-8859-7 Mime-Version: 1.0 (Mac OS X Mail 7.2 \(1874\)) Subject: Re: svn commit: r263778 - in head: bin lib lib/clang sbin share/mk usr.bin usr.sbin From: Warner Losh In-Reply-To: <1398086131.1124.371.camel@revolution.hippie.lan> Date: Mon, 21 Apr 2014 07:35:03 -0600 Content-Transfer-Encoding: quoted-printable Message-Id: References: <201403262230.s2QMUdH6021943@svn.freebsd.org> <20140327181245.GA69977@stack.nl> <7A86F5E9-DBE9-4D3F-B166-C02F8386B722@FreeBSD.org> <1398086131.1124.371.camel@revolution.hippie.lan> To: Ian Lepore X-Mailer: Apple Mail (2.1874) Cc: svn-src-head@FreeBSD.org, svn-src-all@FreeBSD.org, src-committers@FreeBSD.org, Dimitry Andric , Jilles Tjoelker X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2014 13:35:00 -0000 (sorry for the top post) This looks good to my eye. I=A2d be tempted to toss in a comment about = the __wait=3D.WAIT construct is due to the primitive nature of bmake=A2s = parser so it runs afoul of the .for/.if construction rules and this is needed = to expand the for variable. It tripped me up when I looked at it, until I recalled = a comment from similar code in NetBSD. Warner On Apr 21, 2014, at 7:15 AM, Ian Lepore wrote: > On Thu, 2014-03-27 at 20:44 +0100, Dimitry Andric wrote: >> On 27 Mar 2014, at 19:12, Jilles Tjoelker wrote: >>> On Thu, Mar 27, 2014 at 11:05:00AM -0600, Warner Losh wrote: >>>> On Mar 26, 2014, at 4:30 PM, Dimitry Andric = wrote: >>>>> Author: dim >>>>> Date: Wed Mar 26 22:30:38 2014 >>>>> New Revision: 263778 >>>>> URL: http://svnweb.freebsd.org/changeset/base/263778 >>>=20 >>>>> Log: >>>>> Add a SUBDIR_PARALLEL option to bsd.subdir.mk, to allow make to = process >>>>> all the SUBDIR entries in parallel, instead of serially. Apply = this >>>>> option to a selected number of Makefiles, which can greatly speed = up the >>>>> build on multi-core machines, when using make -j. >>>=20 >>>>> This can be extended to more Makefiles later on, whenever they are >>>>> verified to work correctly with parallel building. >>>=20 >>>> Why not have this =A1opt out=A2 rather than =A1opt in=A2 like it is = now? Are >>>> there any known bad dependencies this introduces? >>>=20 >>> I'm paranoid about build systems ;) It is easy to add dependencies >>> across directories and as long as directories are built in sequence, >>> nothing goes wrong. >>>=20 >>> In fact, I had enabled SUBDIR_PARALLEL in sys/modules/Makefile as = well, >>> but this caused mysterious failures with some kernels such as mips >>> ADM5120. >>=20 >> There are a bunch of other parts that don't really like parallel = builds >> at the moment. For example, gnu/usr.bin/binutils needs its libraries >> (libbfd.a, etc) built first, before it can link the programs. = Similar >> for gnu/usr.bin/cc, which needs libiberty, libcpp, etc before being = able >> to build the rest of gcc. >>=20 >> Most of these cases can hopefully be solved by adding .WAIT targets = at >> strategic points in the SUBDIR lists, but this also needs a bit of = extra >> logic in bsd.subdir.mk. >>=20 >> -Dimitry >>=20 >=20 > It turns out I needed the .WAIT functionality to use SUBDIR_PARALLEL = for > $work, so I came up with the attached, does this look okay to commit? >=20 > -- Ian >=20 > diff -r 67802e319fc6 share/mk/bsd.subdir.mk > --- a/share/mk/bsd.subdir.mk Sun Apr 20 21:01:07 2014 -0600 > +++ b/share/mk/bsd.subdir.mk Mon Apr 21 06:59:37 2014 -0600 > @@ -4,10 +4,10 @@ > # The include file contains the default targets > # for building subdirectories. > # > -# For all of the directories listed in the variable SUBDIRS, the > +# For all of the directories listed in the variable SUBDIR, the > # specified directory will be visited and the target made. There is > # also a default target which allows the command "make subdir" where > -# subdir is any directory listed in the variable SUBDIRS. > +# subdir is any directory listed in the variable SUBDIR. > # > # > # +++ variables +++ > @@ -42,7 +42,7 @@ distribute: >=20 > _SUBDIR: .USE > .if defined(SUBDIR) && !empty(SUBDIR) && !defined(NO_SUBDIR) > - @${_+_}for entry in ${SUBDIR}; do \ > + @${_+_}for entry in ${SUBDIR:N.WAIT}; do \ > if test -d ${.CURDIR}/$${entry}.${MACHINE_ARCH}; then \ > ${ECHODIR} "=3D=3D=3D> = ${DIRPRFX}$${entry}.${MACHINE_ARCH} (${.TARGET:realinstall=3Dinstall})"; = \ > edir=3D$${entry}.${MACHINE_ARCH}; \ > @@ -57,7 +57,7 @@ distribute: > done > .endif >=20 > -${SUBDIR}: .PHONY > +${SUBDIR:N.WAIT}: .PHONY > ${_+_}@if test -d ${.TARGET}.${MACHINE_ARCH}; then \ > cd ${.CURDIR}/${.TARGET}.${MACHINE_ARCH}; \ > else \ > @@ -65,13 +65,18 @@ distribute: > fi; \ > ${MAKE} all >=20 > +__wait=3D.WAIT > .for __target in all all-man checkdpadd clean cleandepend cleandir \ > depend distribute lint maninstall manlint \ > obj objlink realinstall regress tags \ > ${SUBDIR_TARGETS} > .ifdef SUBDIR_PARALLEL > +__subdir_targets=3D > .for __dir in ${SUBDIR} > -${__target}: ${__target}_subdir_${__dir} > +.if ${__wait} =3D=3D ${__dir} > +__subdir_targets+=3D .WAIT > +.else > +__subdir_targets+=3D ${__target}_subdir_${__dir} > ${__target}_subdir_${__dir}: .MAKE > @${_+_}set -e; \ > if test -d ${.CURDIR}/${__dir}.${MACHINE_ARCH}; then \ > @@ -85,7 +90,9 @@ distribute: > fi; \ > ${MAKE} ${__target:realinstall=3Dinstall} \ > DIRPRFX=3D${DIRPRFX}$$edir/ > +.endif > .endfor > +${__target}: ${__subdir_targets} > .else > ${__target}: _SUBDIR > .endif