Date: Tue, 16 Dec 2014 13:00:21 +0800 From: Jia-Shiun Li <jiashiun@gmail.com> To: Ian Lepore <ian@freebsd.org> Cc: freebsd-arch <freebsd-arch@freebsd.org> Subject: _bootstrap-tools parallel build (was: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds Message-ID: <CAHNYxxO0F%2BHojpq2As--PeDn0WP6AVew5%2B-4c24bG7TC4hzsrQ@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
--20cf3011e059414030050a4e3ec9 Content-Type: text/plain; charset=UTF-8 On Sat, May 31, 2014 at 10:04 PM, Ian Lepore <ian@freebsd.org> wrote: > > The parallelism in the bootstrap stuff in Makefile.inc1 is done with a > different-but-similar mechanism, and it would be nice to fix that to > just use bsd.subdir.mk instead of almost-duplicating it inline. That's > on a longer-term to-do list for me. > Hi Ian & all, I did it manually for _bootstrap-tools anyway. Patches attached. It reduces 'make -j24 _botostrap-tools' time from ~55 sec. to ~25 sec. on a 12C IVB-EP. It scales more linearly in comparison to single-job ~250 sec on the same machine. Main blockers were tblgen & groff. On -j4 it reduces time from ~82 sec. to ~60 sec. too. It passed 'make -j24 universe' except the seemingly broken i386 kernels. Any comments? -Jia-Shiun --20cf3011e059414030050a4e3ec9 Content-Type: application/octet-stream; name="bootstrap.patch" Content-Disposition: attachment; filename="bootstrap.patch" Content-Transfer-Encoding: base64 X-Attachment-Id: f_i3qsoqf70 SW5kZXg6IE1ha2VmaWxlLmluYzEKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gTWFrZWZpbGUuaW5jMQkocmV2aXNp b24gMjc1Njk5KQorKysgTWFrZWZpbGUuaW5jMQkod29ya2luZyBjb3B5KQpAQCAtMTMxMywxMSAr MTMxMyw3IEBACiAJdXNyLmJpbi9jb21waWxlX2V0CiAuZW5kaWYKIAotIwlQbGVhc2UgZG9jdW1l bnQgKGFkZCBjb21tZW50KSB3aHkgc29tZXRoaW5nIGlzIGluICdib290c3RyYXAtdG9vbHMnLgot IwlUcnkgdG8gYm91bmQgdGhlIGJ1aWxkaW5nIG9mIHRoZSBib290c3RyYXAtdG9vbCB0byBqdXN0 IHRoZQotIwlGcmVlQlNEIHZlcnNpb25zIHRoYXQgbmVlZCB0aGUgdG9vbCBidWlsdCBhdCB0aGlz IHN0YWdlIG9mIHRoZSBidWlsZC4KLWJvb3RzdHJhcC10b29sczogLk1BS0UKLS5mb3IgX3Rvb2wg aW4gXAorQlRfRElSUz0gXAogICAgICR7X2NsYW5nX3RibGdlbn0gXAogICAgICR7X2tlcmJlcm9z NV9ib290c3RyYXBfdG9vbHN9IFwKICAgICAke19kdHJhY2VfdG9vbHN9IFwKQEAgLTEzNDAsMTUg KzEzMzYsNDAgQEAKICAgICB1c3Iuc2Jpbi9jb25maWcgXAogICAgICR7X2NydW5jaH0gXAogICAg ICR7X25tdHJlZX0gXAotICAgICR7X3Z0Zm9udGN2dH0KLQkke18rX31AJHtFQ0hPRElSfSAiPT09 PiAke190b29sfSAob2JqLGRlcGVuZCxhbGwsaW5zdGFsbCkiOyBcCi0JCWNkICR7LkNVUkRJUn0v JHtfdG9vbH0gJiYgXAotCQkke01BS0V9IERJUlBSRlg9JHtfdG9vbH0vIG9iaiAmJiBcCi0JCSR7 TUFLRX0gRElSUFJGWD0ke190b29sfS8gZGVwZW5kICYmIFwKLQkJJHtNQUtFfSBESVJQUkZYPSR7 X3Rvb2x9LyBhbGwgJiYgXAotCQkke01BS0V9IERJUlBSRlg9JHtfdG9vbH0vIERFU1RESVI9JHtN QUtFT0JKRElSUFJFRklYfS9sZWdhY3kgaW5zdGFsbAorICAgICR7X3Z0Zm9udGN2dH0gXAorCisu Zm9yIFggaW4gJHtCVF9ESVJTfQorIyBHZW5lcmF0ZSBydWxlcworJHtYfTogLlBIT05ZCisJJHtf K199QCR7RUNIT0RJUn0gIj09PT4gJHtYfSAob2JqLGRlcGVuZCxhbGwsaW5zdGFsbCkiOyBcCisJ CWNkICR7LkNVUkRJUn0vJHtYfSAmJiBcCisJCSR7TUFLRX0gRElSUFJGWD0ke1h9LyBvYmogJiYg XAorCQkke01BS0V9IERJUlBSRlg9JHtYfS8gZGVwZW5kICYmIFwKKwkJJHtNQUtFfSBESVJQUkZY PSR7WH0vIGFsbCAmJiBcCisJCSR7TUFLRX0gRElSUFJGWD0ke1h9LyBERVNURElSPSR7TUFLRU9C SkRJUlBSRUZJWH0vbGVnYWN5IGluc3RhbGwKIC5lbmRmb3IKIAorIyAke19jbGFuZ190YmxnZW59 IGRlcGVuZGVuY2llcwordXNyLmJpbi9jbGFuZy90YmxnZW46IGxpYi9jbGFuZy9saWJsbHZtdGFi bGVnZW4gbGliL2NsYW5nL2xpYmxsdm1zdXBwb3J0Cit1c3IuYmluL2NsYW5nL2NsYW5nLXRibGdl bjogbGliL2NsYW5nL2xpYmxsdm10YWJsZWdlbiBsaWIvY2xhbmcvbGlibGx2bXN1cHBvcnQKKwor IyAke19rZXJiZXJvczVfYm9vdHN0cmFwX3Rvb2xzfSBkZXBlbmRlbmNpZXMKK2tlcmJlcm9zNS90 b29scy9zbGM6IGtlcmJlcm9zNS9saWIvbGlicm9rZW4KK2tlcmJlcm9zNS90b29scy9hc24xX2Nv bXBpbGU6IGtlcmJlcm9zNS9saWIvbGlicm9rZW4KKworIyBiZWxvdyBkZXBlbmRlbmNpZXMgbm90 IHRlc3RlZCBidXQgcmVwbGljYXRlZCBhcy1pcworIyAke195YWNjfQordXNyLmJpbi95YWNjOiBs aWIvbGlieQorIyAke19tNH0KK3Vzci5iaW4vbTQ6IGxpYi9saWJvaGFzaAorIyAke19ubXRyZWV9 Cit1c3Iuc2Jpbi9ubXRyZWU6IGxpYi9saWJuZXRic2QKKworIwlQbGVhc2UgZG9jdW1lbnQgKGFk ZCBjb21tZW50KSB3aHkgc29tZXRoaW5nIGlzIGluICdib290c3RyYXAtdG9vbHMnLgorIwlUcnkg dG8gYm91bmQgdGhlIGJ1aWxkaW5nIG9mIHRoZSBib290c3RyYXAtdG9vbCB0byBqdXN0IHRoZQor IwlGcmVlQlNEIHZlcnNpb25zIHRoYXQgbmVlZCB0aGUgdG9vbCBidWlsdCBhdCB0aGlzIHN0YWdl IG9mIHRoZSBidWlsZC4KK2Jvb3RzdHJhcC10b29sczogJHtCVF9ESVJTfQorCiAjCiAjIGJ1aWxk LXRvb2xzOiBCdWlsZCBzcGVjaWFsIHB1cnBvc2UgYnVpbGQgdG9vbHMKICMKSW5kZXg6IGdudS91 c3IuYmluL2dyb2ZmL01ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGdudS91c3IuYmluL2dyb2ZmL01h a2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUvdXNyLmJpbi9ncm9mZi9NYWtlZmlsZQko d29ya2luZyBjb3B5KQpAQCAtMiw0ICsyLDYgQEAKIAogU1VCRElSPQkJY29udHJpYiBkb2MgZm9u dCBtYW4gc3JjIHRtYWMKIAorU1VCRElSX1BBUkFMTEVMPQorCiAuaW5jbHVkZSA8YnNkLnN1YmRp ci5taz4KSW5kZXg6IGdudS91c3IuYmluL2dyb2ZmL3NyYy9NYWtlZmlsZQo9PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0t LSBnbnUvdXNyLmJpbi9ncm9mZi9zcmMvTWFrZWZpbGUJKHJldmlzaW9uIDI3NTY5OSkKKysrIGdu dS91c3IuYmluL2dyb2ZmL3NyYy9NYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMiw0ICsyLDEw IEBACiAKIFNVQkRJUj0JCWxpYnMgZGV2aWNlcyBwcmVwcm9jIHJvZmYgdXRpbHMKIAorU1VCRElS X1BBUkFMTEVMPQorU1VCRElSX0RFUEVORF9kZXZpY2VzPSBsaWJzCitTVUJESVJfREVQRU5EX3By ZXByb2M9IGxpYnMKK1NVQkRJUl9ERVBFTkRfcm9mZj0gbGlicworU1VCRElSX0RFUEVORF91dGls cz0gbGlicworCiAuaW5jbHVkZSA8YnNkLnN1YmRpci5taz4KSW5kZXg6IGdudS91c3IuYmluL2dy b2ZmL3NyYy9kZXZpY2VzL01ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGdudS91c3IuYmluL2dyb2Zm L3NyYy9kZXZpY2VzL01ha2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUvdXNyLmJpbi9n cm9mZi9zcmMvZGV2aWNlcy9NYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMiw0ICsyLDYgQEAK IAogU1VCRElSPQkJZ3JvZHZpIGdyb2h0bWwgZ3JvbGJwIGdyb2xqNCBncm9wcyBncm90dHkKIAor U1VCRElSX1BBUkFMTEVMPQorCiAuaW5jbHVkZSA8YnNkLnN1YmRpci5taz4KSW5kZXg6IGdudS91 c3IuYmluL2dyb2ZmL3NyYy9wcmVwcm9jL01ha2VmaWxlCj09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KLS0tIGdudS91c3Iu YmluL2dyb2ZmL3NyYy9wcmVwcm9jL01ha2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUv dXNyLmJpbi9ncm9mZi9zcmMvcHJlcHJvYy9NYWtlZmlsZQkod29ya2luZyBjb3B5KQpAQCAtMiw0 ICsyLDYgQEAKIAogU1VCRElSPQkJZXFuIGdybiBodG1sIHBpYyByZWZlciBzb2VsaW0gdGJsCiAK K1NVQkRJUl9QQVJBTExFTD0KKwogLmluY2x1ZGUgPGJzZC5zdWJkaXIubWs+CkluZGV4OiBnbnUv dXNyLmJpbi9ncm9mZi9zcmMvcm9mZi9NYWtlZmlsZQo9PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09Ci0tLSBnbnUvdXNyLmJp bi9ncm9mZi9zcmMvcm9mZi9NYWtlZmlsZQkocmV2aXNpb24gMjc1Njk5KQorKysgZ251L3Vzci5i aW4vZ3JvZmYvc3JjL3JvZmYvTWFrZWZpbGUJKHdvcmtpbmcgY29weSkKQEAgLTIsNCArMiw2IEBA CiAKIFNVQkRJUj0JCWdyb2ZmIGdyb2cgbnJvZmYgcHNyb2ZmIHRyb2ZmCiAKK1NVQkRJUl9QQVJB TExFTD0KKwogLmluY2x1ZGUgPGJzZC5zdWJkaXIubWs+CkluZGV4OiBnbnUvdXNyLmJpbi9ncm9m Zi9zcmMvdXRpbHMvTWFrZWZpbGUKPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PQotLS0gZ251L3Vzci5iaW4vZ3JvZmYvc3Jj L3V0aWxzL01ha2VmaWxlCShyZXZpc2lvbiAyNzU2OTkpCisrKyBnbnUvdXNyLmJpbi9ncm9mZi9z cmMvdXRpbHMvTWFrZWZpbGUJKHdvcmtpbmcgY29weSkKQEAgLTIsNCArMiw2IEBACiAKIFNVQkRJ Uj0JCWFkZGZ0aW5mbyBhZm10b2RpdCBocGZ0b2RpdCBpbmR4YmliIGxrYmliIGxvb2tiaWIgcGZi dG9wcyB0Zm10b2RpdAogCitTVUJESVJfUEFSQUxMRUw9CisKIC5pbmNsdWRlIDxic2Quc3ViZGly Lm1rPgo= --20cf3011e059414030050a4e3ec9--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAHNYxxO0F%2BHojpq2As--PeDn0WP6AVew5%2B-4c24bG7TC4hzsrQ>