From owner-svn-src-all@freebsd.org Thu Oct 15 22:41:02 2015 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CE9BEA16EA6; Thu, 15 Oct 2015 22:41:02 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id AFC0518B9; Thu, 15 Oct 2015 22:41:02 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [IPv6:::1]) by freefall.freebsd.org (Postfix) with ESMTP id A473916C9; Thu, 15 Oct 2015 22:41:02 +0000 (UTC) (envelope-from bdrewery@FreeBSD.org) Received: from mail.xzibition.com (localhost [172.31.3.2]) by mail.xzibition.com (Postfix) with ESMTP id 6472F118FE; Thu, 15 Oct 2015 22:41:02 +0000 (UTC) X-Virus-Scanned: amavisd-new at mail.xzibition.com Received: from mail.xzibition.com ([172.31.3.2]) by mail.xzibition.com (mail.xzibition.com [172.31.3.2]) (amavisd-new, port 10026) with LMTP id 8O8-eUZSl4Yx; Thu, 15 Oct 2015 22:40:58 +0000 (UTC) Subject: Re: svn commit: r289384 - head/lib DKIM-Filter: OpenDKIM Filter v2.9.2 mail.xzibition.com 98FE0118F8 To: Ian Lepore , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org References: <201510152027.t9FKRFgT031177@repo.freebsd.org> <5620127C.9070409@FreeBSD.org> <1444942810.71631.3.camel@freebsd.org> <5620180D.5010206@FreeBSD.org> From: Bryan Drewery Openpgp: id=F9173CB2C3AAEA7A5C8A1F0935D771BB6E4697CF; url=http://www.shatow.net/bryan/bryan2.asc X-Enigmail-Draft-Status: N1110 Organization: FreeBSD Message-ID: <56202B7C.9020105@FreeBSD.org> Date: Thu, 15 Oct 2015 15:41:00 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.3.0 MIME-Version: 1.0 In-Reply-To: <5620180D.5010206@FreeBSD.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="iOfpjA51JV7QM6MSnOoxV8ruxcagnlUbh" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 15 Oct 2015 22:41:03 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --iOfpjA51JV7QM6MSnOoxV8ruxcagnlUbh Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: quoted-printable On 10/15/2015 2:18 PM, Bryan Drewery wrote: > On 10/15/2015 2:00 PM, Ian Lepore wrote: >> On Thu, 2015-10-15 at 13:54 -0700, Bryan Drewery wrote: >>> On 10/15/2015 1:27 PM, Bryan Drewery wrote: >>>> Author: bdrewery >>>> Date: Thu Oct 15 20:27:15 2015 >>>> New Revision: 289384 >>>> URL: https://svnweb.freebsd.org/changeset/base/289384 >>>> >>>> Log: >>>> Let the SUBDIR_DEPEND*c++ variables actually work rather than >>>> being parsed >>>> as a +=3D. These were safe due to a .WAIT very early on. >>>> =20 >>> >>> My analysis was wrong. The libc++ dependency on libcxxrt was actually= >>> broken here and would not have built if you went directly into lib >>> and >>> built in parallel. It was hidden by the fact that a proper dependency= >>> is >>> in Makefile.inc1 via an __L target. The power of redundancy! >>> >>> This too allows kerberos5/lib to work with SUBDIR_PARALLEL=3D since >>> it's >>> so greatly defined in Makefile.inc1. >>> >>> >> >> But don't both sets of dependencies have to be correct? As I vaguely >> recall, the ones in Makefile.inc1 are used during bootstrapping and >> tool building, and the ones in lib/Makefile are used when world >> -building. It may be that when MACHINE_ARCH =3D=3D TARGET_ARCH some o= f >> them are the same libraries and the Makefile.inc1 dependencies take >> care of everything. >=20 > It's complicated, my words may seem to contradict but it's just the > nature of the convoluted process that we bootstrap the build by. >=20 > Libraries, and anything else we list in the bootstrap phases, are > special. Their SUBDIR_depend are less needed in the context of > buildworld since they will already be built by the time the 'make > libraries' builds all of lib/ with a subdir build or the 'make all' is > ran on the tree, because of the __L dependencies. The special bootstrap= > phase stuff and special libraries need to have all of the proper __L > targets to order them as they are built directly, not via bsd.subdir.mk= > recursing. One of the last steps of 'make libraries' does do a subdir > build in lib/ since _generic_libs contains 'lib'. So yes the > SUBDIR_depend are relevant for buildworld, but getting it wrong can be > fine in cases where the library was built early with proper __L deps li= sted. >=20 > The SUBDIR_depend are needed for manual direct subdir builds outside of= > buildworld and the builds 'make libraries' and 'make all' phases which > use subdir builds to build things lacking a __L list. >=20 > I've noticed that when people add new libraries into the early phases > they usually follow-up with multiple commits trying to get it all right= > and update all of the proper places. >=20 > Note that we actually have a 3rd dependency list in the tree now which > is in the scattered Makefile.depend files. These are used for "meta > mode" which I do think is more on the right track as it lets a clean > build start from any directory. It doesn't use the SUBDIR_depend or th= e > __L lists at all. The problem with it is the auto-generated nature of > it can lead to problems with optional dependencies and adding new > directories. >=20 Here's an example of how weird buildworld is with libraries. This is during the 'make all' phase. 0:00.32 make all DIRPRFX=3Dusr.bin/svn/lib/libsvn_client/ 0:09.27 make all DIRPRFX=3Dlib/libc/ 0:00.49 make all DIRPRFX=3Dusr.sbin/wpa/wpa_supplicant/ 0:00.05 make all DIRPRFX=3Dusr.bin/svn/ 0:00.03 make all DIRPRFX=3Dusr.bin/svn/lib/ 0:00.12 make all DIRPRFX=3Dusr.sbin/wpa/ 0:14.68 make -f Makefile.inc1 DESTDIR=3D/usr/obj/root/svn/base/tmp par-a= ll 0:06.35 make all DIRPRFX=3Dlib/ 0:12.63 make all DIRPRFX=3Dusr.bin/ 0:13.22 make all DIRPRFX=3Dusr.sbin/ 0:07.16 make -j15 buildworld It's building lib/ a 2nd time while also building things depending on libraries, so really at this point the SUBDIR_depend in lib/ and elsewhere are not too important since they were already built by 'make libraries' which had the __L targets. We probably don't need to run 'make all' in the library directories a 2nd time, just 'install'. Also interestingly they are building in parallel without actually using the SUBDIR_PARALLEL mechanism as it was made to be parallel in a similar way in r30113. This is likely redundant now as well. I'll look into those more. --=20 Regards, Bryan Drewery --iOfpjA51JV7QM6MSnOoxV8ruxcagnlUbh Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iQEcBAEBAgAGBQJWICt8AAoJEDXXcbtuRpfPOIgIAI/W9MyVSmkCwu1UAzPpJx8Z NM4zULsU9jhdXBBz8hnIxbOtf/A+YvCmdTgGTGUkGLndVfEa8eIXhIItceJLlUom BJ5wtIKL+zK5WinJus2gYTyd3IwnfHH09TJK4dBd9Cs/AeIDfVO28x+tPqg5ntJh 1PhbQSTHfAWgvbSGoFFjyj4GXSG5qbmcCyuNm8l3jRKAXVuXeAeUXo1ABFsva32B GxL8UtlqtwiqnflKZgSsrcaQiwFr7xMj05Oc+UxBBz7QOw3nhnAYPljzboTU4efm EhFK3IX+btETpm2+MDMGCTSKNNcK3MKKcASCgJ92JRj3PQHwfOwme0KvNPA9dEg= =Psba -----END PGP SIGNATURE----- --iOfpjA51JV7QM6MSnOoxV8ruxcagnlUbh--