From owner-svn-src-head@FreeBSD.ORG Tue Jul 9 00:01:09 2013 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 7E697BA6; Tue, 9 Jul 2013 00:01:09 +0000 (UTC) (envelope-from alfred@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.freebsd.org (Postfix) with ESMTP id 5FA941FFB; Tue, 9 Jul 2013 00:01:09 +0000 (UTC) Received: from Alfreds-MacBook-Pro-9.local (c-67-180-208-218.hsd1.ca.comcast.net [67.180.208.218]) by elvis.mu.org (Postfix) with ESMTPSA id 3A75B1A3C7A; Mon, 8 Jul 2013 17:01:02 -0700 (PDT) Message-ID: <51DB52BC.2030802@freebsd.org> Date: Mon, 08 Jul 2013 17:01:00 -0700 From: Alfred Perlstein User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130620 Thunderbird/17.0.7 MIME-Version: 1.0 To: Garrett Cooper Subject: Re: svn commit: r253002 - head References: <201307072039.r67KdCdR028908@svn.freebsd.org> <201307081713.51892.jhb@freebsd.org> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , "src-committers@freebsd.org" , John Baldwin X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 00:01:09 -0000 On 7/8/13 4:24 PM, Garrett Cooper wrote: > On Mon, Jul 8, 2013 at 2:13 PM, John Baldwin wrote: >> On Monday, July 08, 2013 2:23:31 am Garrett Cooper wrote: >>> On Sun, Jul 7, 2013 at 7:25 PM, Garrett Cooper >> wrote: >>>> On Jul 7, 2013, at 2:15 PM, Alfred Perlstein wrote: >>>> >>>>> On 7/7/13 2:01 PM, Garrett Cooper wrote: >>>>>> Why the magic number 12? >>>>> Numbers higher seem to result in worse performance as reported by some >> members of my team. >>>> The suggestion is good in spirit, but this doesn't justify the reasoning >> for this recommendation for all cases. >>>> Please revert this change and add a doc page or notes to the dev handbook >> discussing what the empirical process and results were for determining this >> value so people can come up with their own values that work best with their >> hardware and software config. This recommendation is prone to bitrot like some >> of the recommendations in tuning(7). >>>> Misinformation is sometimes more harmful than no information. >>> I spoke with Alfred over the phone and did some more careful thought >>> about this and I'm rescinding this request. >>> >>> Alfred did a good job at documenting how JFLAG works (it was >>> previously undocumented). My concern over -j12 was performance >>> related, and after giving things more careful thought it actually >>> makes sense why -j12 was chosen because Westmere and newer processors >>> have issues with NUMA and cache locality between multiple processor >>> packages as we've seen non-empirically and empirically at Isilon with >>> FreeBSD 7 and 10 (it's a known issue that jeffr@ and jhb@ are aware >>> of). >>> >>> I'll come up with a concise patch that does what Alfred was trying to >>> achieve and have Alfred review it. >>> >>> Thanks (and thank you Alfred for the contribution!!!)! >> Westmere is fine, it's post-Westmere that is more troublesome. > Even the 6-core Westmeres (I'm being completely dumb here as you and > Jeff know a lot more about the NUMA issue than I do as I just caught > the tail end of the conversation at BSDCan)? I'm asking because they > (iX) are using build.ix as the primary build machine and it has 2 > Westmere dies with (IIRC -- please correct me if I'm wrong > Alfred/Xin/etc) 6 cores each and are SMT enabled. It also has a > boatload of RAM and disks hooked up to an mfi(4) controller (which > could be a contributing factor in the performance degradation issue). > >> I think the comment is not super useful, but don't object enough to want >> it to be removed. I always use 'make tinderbox' instead of >> 'make universe' though as I want build failures to be obvious. For the >> described use case of "checking if kernels build", 'tinderbox' certainly >> seems to be the more appropriate target. > Changing it from universe to tinderbox seems like a better idea -- > I'll put a short note in my proposed patch for that. > > Thanks! > -Garrett > Just to clarify, the passing of jflag in the comments was there because it seems like most of the targets default to "-j1" which out of the box is somewhat ludicrous these days. It was only a guide such that someone who knows what "-j" does would be like "oh that's absurd, I know a better value" rather than just being oblivious to it (like me) or stupidly assume that some level of auto-tuning was done (also like me) and wind up wondering why "make universe" is taking as long as an actual real live universe to build. ;) -Alfred