Skip site navigation (1)Skip section navigation (2)
Date:      Mon, 08 Jul 2013 17:01:00 -0700
From:      Alfred Perlstein <alfred@freebsd.org>
To:        Garrett Cooper <yaneurabeya@gmail.com>
Cc:        "svn-src-head@freebsd.org" <svn-src-head@freebsd.org>, "svn-src-all@freebsd.org" <svn-src-all@freebsd.org>, "src-committers@freebsd.org" <src-committers@freebsd.org>, John Baldwin <jhb@freebsd.org>
Subject:   Re: svn commit: r253002 - head
Message-ID:  <51DB52BC.2030802@freebsd.org>
In-Reply-To: <CAGHfRMDBB37pF-FKRvY==SM7A9oNQzMFSaPHXsHF6wfnN4J=Xg@mail.gmail.com>
References:  <201307072039.r67KdCdR028908@svn.freebsd.org> <D3E9FF9F-E53A-40DE-8DFF-C4A4F05566D4@gmail.com> <CAGHfRMDmRXR1jTF_CO4T8NXOPW5Soxjrq1y_eb41mKZG2yQiiw@mail.gmail.com> <201307081713.51892.jhb@freebsd.org> <CAGHfRMDBB37pF-FKRvY==SM7A9oNQzMFSaPHXsHF6wfnN4J=Xg@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help
On 7/8/13 4:24 PM, Garrett Cooper wrote:
> On Mon, Jul 8, 2013 at 2:13 PM, John Baldwin <jhb@freebsd.org> wrote:
>> On Monday, July 08, 2013 2:23:31 am Garrett Cooper wrote:
>>> On Sun, Jul 7, 2013 at 7:25 PM, Garrett Cooper <yaneurabeya@gmail.com>
>> wrote:
>>>> On Jul 7, 2013, at 2:15 PM, Alfred Perlstein <alfred@freebsd.org> wrote:
>>>>
>>>>> On 7/7/13 2:01 PM, Garrett Cooper wrote:
>>>>>> Why the magic number 12?
>>>>> Numbers higher seem to result in worse performance as reported by some
>> members of my team.
>>>> The suggestion is good in spirit, but this doesn't justify the reasoning
>> for this recommendation for all cases.
>>>> Please revert this change and add a doc page or notes to the dev handbook
>> discussing what the empirical process and results were for determining this
>> value so people can come up with their own values that work best with their
>> hardware and software config. This recommendation is prone to bitrot like some
>> of the recommendations in tuning(7).
>>>> Misinformation is sometimes more harmful than no information.
>>> I spoke with Alfred over the phone and did some more careful thought
>>> about this and I'm rescinding this request.
>>>
>>> Alfred did a good job at documenting how JFLAG works (it was
>>> previously undocumented). My concern over -j12 was performance
>>> related, and after giving things more careful thought it actually
>>> makes sense why -j12 was chosen because Westmere and newer processors
>>> have issues with NUMA and cache locality between multiple processor
>>> packages as we've seen non-empirically and empirically at Isilon with
>>> FreeBSD 7 and 10 (it's a known issue that jeffr@ and jhb@ are aware
>>> of).
>>>
>>> I'll come up with a concise patch that does what Alfred was trying to
>>> achieve and have Alfred review it.
>>>
>>> Thanks (and thank you Alfred for the contribution!!!)!
>> Westmere is fine, it's post-Westmere that is more troublesome.
> Even the 6-core Westmeres (I'm being completely dumb here as you and
> Jeff know a lot more about the NUMA issue than I do as I just caught
> the tail end of the conversation at BSDCan)? I'm asking because they
> (iX) are using build.ix as the primary build machine and it has 2
> Westmere dies with (IIRC -- please correct me if I'm wrong
> Alfred/Xin/etc) 6 cores each and are SMT enabled. It also has a
> boatload of RAM and disks hooked up to an mfi(4) controller (which
> could be a contributing factor in the performance degradation issue).
>
>> I think the comment is not super useful, but don't object enough to want
>> it to be removed.  I always use 'make tinderbox' instead of
>> 'make universe' though as I want build failures to be obvious.  For the
>> described use case of "checking if kernels build", 'tinderbox' certainly
>> seems to be the more appropriate target.
> Changing it from universe to tinderbox seems like a better idea --
> I'll put a short note in my proposed patch for that.
>
> Thanks!
> -Garrett
>
Just to clarify, the passing of jflag in the comments was there because 
it seems like most of the targets default to "-j1" which out of the box 
is somewhat ludicrous these days.  It was only a guide such that someone 
who knows what "-j" does would be like "oh that's absurd, I know a 
better value" rather than just being oblivious to it (like me) or 
stupidly assume that some level of auto-tuning was done (also like me) 
and wind up wondering why "make universe" is taking as long as an actual 
real live universe to build.

;)

-Alfred



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?51DB52BC.2030802>