From owner-freebsd-arch@FreeBSD.ORG Tue Dec 16 05:00:58 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id CEC0DBD4; Tue, 16 Dec 2014 05:00:58 +0000 (UTC) Received: from mail-ig0-x22d.google.com (mail-ig0-x22d.google.com [IPv6:2607:f8b0:4001:c05::22d]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 96516183; Tue, 16 Dec 2014 05:00:58 +0000 (UTC) Received: by mail-ig0-f173.google.com with SMTP id r2so6264611igi.12; Mon, 15 Dec 2014 21:00:58 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:from:date:message-id:subject:to:cc:content-type; bh=HLwcJ1rqAW9U8mm2Ud4VPWMdqJOYD7blWWtbl+vpXJM=; b=S+yxhL485cAtW7nFWUwUSM/sH7iSCX1DqCwRFIfti8I1ttK6hC/VfPBa78rtMeOExo 83Zk6KFzujwDWpYbnno75LXbbZxM7183M1wrGCLSvseZPewGAugDsJS0ukStkcRBPzRQ DrNoIV7fgMqvg+Fa3VayawBjRMEMlzWw3f+Vi6I/V2VsrGf2R90F/H02g6/o1R5PFHpK usuAUymAWcfu4O+eRQThNteGMdPbnk8cUM3iHrSu/CqTNXzwyXmNvvwvIcgD0xVDPBVR KnwMbnAk4tUOS+vAKB6HaIagfVfO3icF90NVH2fDLy2hGKWV3/vyzhiM8GxjHKWboyWB TTEg== X-Received: by 10.42.129.140 with SMTP id q12mr29942582ics.68.1418706051734; Mon, 15 Dec 2014 21:00:51 -0800 (PST) MIME-Version: 1.0 Received: by 10.107.175.4 with HTTP; Mon, 15 Dec 2014 21:00:21 -0800 (PST) From: Jia-Shiun Li Date: Tue, 16 Dec 2014 13:00:21 +0800 Message-ID: Subject: _bootstrap-tools parallel build (was: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds To: Ian Lepore Content-Type: multipart/mixed; boundary=20cf3011e059414030050a4e3ec9 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 16 Dec 2014 05:00:58 -0000 --20cf3011e059414030050a4e3ec9 Content-Type: text/plain; charset=UTF-8 On Sat, May 31, 2014 at 10:04 PM, Ian Lepore wrote: > > The parallelism in the bootstrap stuff in Makefile.inc1 is done with a > different-but-similar mechanism, and it would be nice to fix that to > just use bsd.subdir.mk instead of almost-duplicating it inline. That's > on a longer-term to-do list for me. > Hi Ian & all, I did it manually for _bootstrap-tools anyway. Patches attached. It reduces 'make -j24 _botostrap-tools' time from ~55 sec. to ~25 sec. on a 12C IVB-EP. It scales more linearly in comparison to single-job ~250 sec on the same machine. Main blockers were tblgen & groff. On -j4 it reduces time from ~82 sec. to ~60 sec. too. It passed 'make -j24 universe' except the seemingly broken i386 kernels. Any comments? -Jia-Shiun --20cf3011e059414030050a4e3ec9 Content-Type: application/octet-stream; name="bootstrap.patch" Content-Disposition: attachment; filename="bootstrap.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i3qsoqf70 SW5kZXg6IE1ha2VmaWxlLmluYzEKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gTWFrZWZpbGUuaW5jMQkocmV2aXNp b24gMjc1Njk5KQorKysgTWFrZWZpbGUuaW5jMQkod29ya2luZyBjb3B5KQpAQCAtMTMxMywxMSAr MTMxMyw3IEBACiAJdXNyLmJpbi9jb21waWxlX2V0CiAuZW5kaWYKIAotIwlQbGVhc2UgZG9jdW1l bnQgKGFkZCBjb21tZW50KSB3aHkgc29tZXRoaW5nIGlzIGluICdib290c3RyYXAtdG9vbHMnLgot IwlUcnkgdG8gYm91bmQgdGhlIGJ1aWxkaW5nIG9mIHRoZSBib290c3RyYXAtdG9vbCB0byBqdXN0 IHRoZQotIwlGcmVlQlNEIHZlcnNpb25zIHRoYXQgbmVlZCB0aGUgdG9vbCBidWlsdCBhdCB0aGlz IHN0YWdlIG9mIHRoZSBidWlsZC4KLWJvb3RzdHJhcC10b29sczogLk1BS0UKLS5mb3IgX3Rvb2wg aW4gXAorQlRfRElSUz0gXAogICAgICR7X2NsYW5nX3RibGdlbn0gXAogICAgICR7X2tlcmJlcm9z NV9ib290c3RyYXBfdG9vbHN9IFwKICAgICAke19kdHJhY2VfdG9vbHN9IFwKQEAgLTEzNDAsMTUg KzEzMzYsNDAgQEAKICAgICB1c3Iuc2Jpbi9jb25maWcgXAogICAgICR7X2NydW5jaH0gXAogICAg ICR7X25tdHJlZX0gXAotICAgICR7X3Z0Zm9udGN2dH0KLQkke18rX31AJHtFQ0hPRElSfSAiPT09 PiAke190b29sfSAob2JqLGRlcGVuZCxhbGwsaW5zdGFsbCkiOyBcCi0JCWNkICR7LkNVUkRJUn0v JHtfdG9vbH0gJiYgXAotCQkke01BS0V9IERJUlBSRlg9JHtfdG9vbH0vIG9iaiAmJiBcCi0JCSR7 TUFLRX0gRElSUFJGWD0ke190b29sfS8gZGVwZW5kICYmIFwKLQkJJHtNQUtFfSBESVJQUkZYPSR7 X3Rvb2x9LyBhbGwgJiYgXAotCQkke01BS0V9IERJUlBSRlg9JHtfdG9vbH0vIERFU1RESVI9JHtN QUtFT0JKRElSUFJFRklYfS9sZWdhY3kgaW5zdGFsbAorICAgICR7X3Z0Zm9udGN2dH0gXAorCisu Zm9yIFggaW4gJHtCVF9ESVJTfQorIyBHZW5lcmF0ZSBydWxlcworJHtYfTogLlBIT05ZCisJJHtf K199QCR7RUNIT0RJUn0gIj09PT4gJHtYfSAob2JqLGRlcGVuZCxhbGwsaW5zdGFsbCkiOyBcCisJ CWNkICR7LkNVUkRJUn0vJHtYfSAmJiBcCisJCSR7TUFLRX0gRElSUFJGWD0ke1h9LyBvYmogJiYg XAorCQkke01BS0V9IERJUlBSRlg9JHtYfS8gZGVwZW5kICYmIFwKKwkJJHtNQUtFfSBESVJQUkZY PSR7WH0vIGFsbCAmJiBcCisJCSR7TUFLRX0gRElSUFJGWD0ke1h9LyBERVNURElSPSR7TUFLRU9C SkRJUlBSRUZJWH0vbGVnYWN5IGluc3RhbGwKIC5lbmRmb3IKIAorIyAke19jbGFuZ190YmxnZW59 IGRlcGVuZGVuY2llcwordXNyLmJpbi9jbGFuZy90YmxnZW46IGxpYi9jbGFuZy9saWJsbHZtdGFi bGVnZW4gbGliL2NsYW5nL2xpYmxsdm1zdXBwb3J0Cit1c3IuYmluL2NsYW5nL2NsYW5nLXRibGdl bjogbGliL2NsYW5nL2xpYmxsdm10YWJsZWdlbiBsaWIvY2xhbmcvbGlibGx2bXN1cHBvcnQKKwor IyAke19rZXJiZXJvczVfYm9vdHN0cmFwX3Rvb2xzfSBkZXBlbmRlbmNpZXMKK2tlcmJlcm9zNS90 b29scy9zbGM6IGtlcmJlcm9zNS9saWIvbGlicm9rZW4KK2tlcmJlcm9zNS90b29scy9hc24xX2Nv bXBpbGU6IGtlcmJlcm9zNS9saWIvbGlicm9rZW4KKworIyBiZWxvdyBkZXBlbmRlbmNpZXMgbm90 IHRlc3RlZCBidXQgcmVwbGljYXRlZCBhcy1pcworIyAke195YWNjfQordXNyLmJpbi95YWNjOiBs aWIvbGlieQorIyAke19tNH0KK3Vzci5iaW4vbTQ6IGxpYi9saWJvaGFzaAorIyAke19ubXRyZWV9 Cit1c3Iuc2Jpbi9ubXRyZWU6IGxpYi9saWJuZXRic2QKKworIwlQbGVhc2UgZG9jdW1lbnQgKGFk ZCBjb21tZW50KSB3aHkgc29tZXRoaW5nIGlzIGluICdib290c3RyYXAtdG9vbHMnLgorIwlUcnkg dG8gYm91bmQgdGhlIGJ1aWxkaW5nIG9mIHRoZSBib290c3RyYXAtdG9vbCB0byBqdXN0IHRoZQor IwlGcmVlQlNEIHZlcnNpb25zIHRoYXQgbmVlZCB0aGUgdG9vbCBidWlsdCBhdCB0aGlzIHN0YWdl IG9mIHRoZSBidWlsZC4KK2Jvb3RzdHJhcC10b29sczogJHtCVF9ESVJTfQorCiAjCiAjIGJ1aWxk LXRvb2xzOiBCdWlsZCBzcGVjaWFsIHB1cnBvc2UgYnVpbGQgdG9vbHMKICMKSW5kZXg6IGdudS91 c3IuYmluL2dyb2ZmL01ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGdudS91c3IuYmluL2dyb2ZmL01h a2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUvdXNyLmJpbi9ncm9mZi9NYWtlZmlsZQko d29ya2luZyBjb3B5KQpAQCAtMiw0ICsyLDYgQEAKIAogU1VCRElSPQkJY29udHJpYiBkb2MgZm9u dCBtYW4gc3JjIHRtYWMKIAorU1VCRElSX1BBUkFMTEVMPQorCiAuaW5jbHVkZSA8YnNkLnN1YmRp ci5taz4KSW5kZXg6IGdudS91c3IuYmluL2dyb2ZmL3NyYy9NYWtlZmlsZQo9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0t LSBnbnUvdXNyLmJpbi9ncm9mZi9zcmMvTWFrZWZpbGUJKHJldmlzaW9uIDI3NTY5OSkKKysrIGdu dS91c3IuYmluL2dyb2ZmL3NyYy9NYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMiw0ICsyLDEw IEBACiAKIFNVQkRJUj0JCWxpYnMgZGV2aWNlcyBwcmVwcm9jIHJvZmYgdXRpbHMKIAorU1VCRElS X1BBUkFMTEVMPQorU1VCRElSX0RFUEVORF9kZXZpY2VzPSBsaWJzCitTVUJESVJfREVQRU5EX3By ZXByb2M9IGxpYnMKK1NVQkRJUl9ERVBFTkRfcm9mZj0gbGlicworU1VCRElSX0RFUEVORF91dGls cz0gbGlicworCiAuaW5jbHVkZSA8YnNkLnN1YmRpci5taz4KSW5kZXg6IGdudS91c3IuYmluL2dy b2ZmL3NyYy9kZXZpY2VzL01ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGdudS91c3IuYmluL2dyb2Zm L3NyYy9kZXZpY2VzL01ha2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUvdXNyLmJpbi9n cm9mZi9zcmMvZGV2aWNlcy9NYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMiw0ICsyLDYgQEAK IAogU1VCRElSPQkJZ3JvZHZpIGdyb2h0bWwgZ3JvbGJwIGdyb2xqNCBncm9wcyBncm90dHkKIAor U1VCRElSX1BBUkFMTEVMPQorCiAuaW5jbHVkZSA8YnNkLnN1YmRpci5taz4KSW5kZXg6IGdudS91 c3IuYmluL2dyb2ZmL3NyYy9wcmVwcm9jL01ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGdudS91c3Iu YmluL2dyb2ZmL3NyYy9wcmVwcm9jL01ha2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUv dXNyLmJpbi9ncm9mZi9zcmMvcHJlcHJvYy9NYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMiw0 ICsyLDYgQEAKIAogU1VCRElSPQkJZXFuIGdybiBodG1sIHBpYyByZWZlciBzb2VsaW0gdGJsCiAK K1NVQkRJUl9QQVJBTExFTD0KKwogLmluY2x1ZGUgPGJzZC5zdWJkaXIubWs+CkluZGV4OiBnbnUv dXNyLmJpbi9ncm9mZi9zcmMvcm9mZi9NYWtlZmlsZQo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBnbnUvdXNyLmJp bi9ncm9mZi9zcmMvcm9mZi9NYWtlZmlsZQkocmV2aXNpb24gMjc1Njk5KQorKysgZ251L3Vzci5i aW4vZ3JvZmYvc3JjL3JvZmYvTWFrZWZpbGUJKHdvcmtpbmcgY29weSkKQEAgLTIsNCArMiw2IEBA CiAKIFNVQkRJUj0JCWdyb2ZmIGdyb2cgbnJvZmYgcHNyb2ZmIHRyb2ZmCiAKK1NVQkRJUl9QQVJB TExFTD0KKwogLmluY2x1ZGUgPGJzZC5zdWJkaXIubWs+CkluZGV4OiBnbnUvdXNyLmJpbi9ncm9m Zi9zcmMvdXRpbHMvTWFrZWZpbGUKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gZ251L3Vzci5iaW4vZ3JvZmYvc3Jj L3V0aWxzL01ha2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUvdXNyLmJpbi9ncm9mZi9z cmMvdXRpbHMvTWFrZWZpbGUJKHdvcmtpbmcgY29weSkKQEAgLTIsNCArMiw2IEBACiAKIFNVQkRJ Uj0JCWFkZGZ0aW5mbyBhZm10b2RpdCBocGZ0b2RpdCBpbmR4YmliIGxrYmliIGxvb2tiaWIgcGZi dG9wcyB0Zm10b2RpdAogCitTVUJESVJfUEFSQUxMRUw9CisKIC5pbmNsdWRlIDxic2Quc3ViZGly Lm1rPgo= --20cf3011e059414030050a4e3ec9--