From owner-freebsd-arch@FreeBSD.ORG Fri May 30 05:54:49 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 866461DB; Fri, 30 May 2014 05:54:49 +0000 (UTC) Received: from mail-oa0-x231.google.com (mail-oa0-x231.google.com [IPv6:2607:f8b0:4003:c02::231]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 45AE72195; Fri, 30 May 2014 05:54:49 +0000 (UTC) Received: by mail-oa0-f49.google.com with SMTP id eb12so1389962oac.36 for ; Thu, 29 May 2014 22:54:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:from:date:message-id:subject:to :cc:content-type; bh=PO7PiTqYVxgt6ZpmB7N5BkKabOcaCRKanhlBlJv0NPA=; b=WgwG+NTYe0QS2mcSvIXQT5A9H/hd/ct1R79QH8uo1OB55ow7G/PAXjnKKvBJJzl0Yn 0Mm+zJf3U2rodifVEHt+v5ImHz3jtxjpbPMev1quKTXN1QGlZECIM1hDqlBxXU/K6X7g ZdVmS+j40ivbV47lhL9fOtVYmddLCZSzzz4u3/RJMkNPw8HtFC2bvDUiwfzJ3N1AVA+n KPxJJBQYgavME/kokoBjfkdfzM1sYfiGxnfMnRY5Btn9JBtf6OrcBsqAIh50VnQRhSPf fYtW791FRojF+t7QcfSG0jUE5uMKxynJhQq2RkMViM1rI5ylPnHnFsaQVyOg8tzJ27d5 Z/OQ== X-Received: by 10.60.54.228 with SMTP id m4mr14022213oep.29.1401429288579; Thu, 29 May 2014 22:54:48 -0700 (PDT) MIME-Version: 1.0 Received: by 10.76.123.178 with HTTP; Thu, 29 May 2014 22:54:18 -0700 (PDT) In-Reply-To: <1401280050.1152.377.camel@revolution.hippie.lan> References: <1401115733.1152.339.camel@revolution.hippie.lan> <1401280050.1152.377.camel@revolution.hippie.lan> From: Jia-Shiun Li Date: Fri, 30 May 2014 13:54:18 +0800 Message-ID: Subject: Re: CFR, CFT: Fine-grained SUBDIR dependencies for parallel builds To: Ian Lepore Content-Type: text/plain; charset=UTF-8 Cc: freebsd-arch X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 30 May 2014 05:54:49 -0000 Updated patch works fine for me surviving -j64 buildworld on quad-core. Compile time did not differ much, ~35 mins. I 'd guess that's because build has already been well paralleled inside each subdirs for ~4 jobs. It would be even better if someone can stress it on multicore system, say 12+ cores, to explore higher hw concurrency to see the difference. Looks other targets using .for loops can be translated to this finer dependency rules as well? e.g. _bootstrap-tools, probably already on your todo list? -Jia-Shiun. On Wed, May 28, 2014 at 8:27 PM, Ian Lepore wrote: > On Wed, 2014-05-28 at 12:58 +0800, Jia-Shiun Li wrote: >> It failed cleandir at libmb with -j4. Test script attached and log >> snippet below. >> >> Tested with: >> - HW: i5-3450 CPU w/ 8GB memory >> - /usr/obj & src mounted on tmpfs. >> >> src uses ~1GB without .svn dir. /usr/obj uses another ~2GB for >> buildworld, not including buildkernel. If memory is not constrained I >> think it is easier to use tmpfs to uncover parallel timing/race issues >> hidden by slower I/O. >> >> >> -Jia-shiun. > > Doh! There was a typo, libmb should have been libmd. More importantly, > it shows that my testing didn't test anything at all. I think I applied > the patch in one sandbox and then ran make universe in a different one. > Here's an updated patch (this time I'll try to test it correctly myself > too). > > Thanks for testing. > > -- Ian >