From owner-svn-src-all@FreeBSD.ORG Tue Jul 9 18:48:44 2013 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 4695EFC8; Tue, 9 Jul 2013 18:48:44 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigwig.baldwin.cx [IPv6:2001:470:1f11:75::1]) by mx1.freebsd.org (Postfix) with ESMTP id 2060E1339; Tue, 9 Jul 2013 18:48:44 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 02110B977; Tue, 9 Jul 2013 14:48:43 -0400 (EDT) From: John Baldwin To: Garrett Cooper Subject: Re: svn commit: r253002 - head Date: Tue, 9 Jul 2013 14:10:34 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p25; KDE/4.5.5; amd64; ; ) References: <201307072039.r67KdCdR028908@svn.freebsd.org> <201307081713.51892.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201307091410.35120.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Tue, 09 Jul 2013 14:48:43 -0400 (EDT) Cc: "svn-src-head@freebsd.org" , "svn-src-all@freebsd.org" , Alfred Perlstein , "src-committers@freebsd.org" X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Jul 2013 18:48:44 -0000 On Monday, July 08, 2013 7:24:04 pm Garrett Cooper wrote: > On Mon, Jul 8, 2013 at 2:13 PM, John Baldwin wrote: > > On Monday, July 08, 2013 2:23:31 am Garrett Cooper wrote: > >> On Sun, Jul 7, 2013 at 7:25 PM, Garrett Cooper > > wrote: > >> > On Jul 7, 2013, at 2:15 PM, Alfred Perlstein wrote: > >> > > >> >> On 7/7/13 2:01 PM, Garrett Cooper wrote: > >> >>> Why the magic number 12? > >> >> > >> >> Numbers higher seem to result in worse performance as reported by some > > members of my team. > >> > > >> > The suggestion is good in spirit, but this doesn't justify the reasoning > > for this recommendation for all cases. > >> > > >> > Please revert this change and add a doc page or notes to the dev handbook > > discussing what the empirical process and results were for determining this > > value so people can come up with their own values that work best with their > > hardware and software config. This recommendation is prone to bitrot like some > > of the recommendations in tuning(7). > >> > > >> > Misinformation is sometimes more harmful than no information. > >> > >> I spoke with Alfred over the phone and did some more careful thought > >> about this and I'm rescinding this request. > >> > >> Alfred did a good job at documenting how JFLAG works (it was > >> previously undocumented). My concern over -j12 was performance > >> related, and after giving things more careful thought it actually > >> makes sense why -j12 was chosen because Westmere and newer processors > >> have issues with NUMA and cache locality between multiple processor > >> packages as we've seen non-empirically and empirically at Isilon with > >> FreeBSD 7 and 10 (it's a known issue that jeffr@ and jhb@ are aware > >> of). > >> > >> I'll come up with a concise patch that does what Alfred was trying to > >> achieve and have Alfred review it. > >> > >> Thanks (and thank you Alfred for the contribution!!!)! > > > > Westmere is fine, it's post-Westmere that is more troublesome. > > Even the 6-core Westmeres (I'm being completely dumb here as you and > Jeff know a lot more about the NUMA issue than I do as I just caught > the tail end of the conversation at BSDCan)? Yes. NUMA can matter some on Nehalem and Westmere, but for workloads that involve I/O, the SB/IB generation where the I/O hub moves onto the processor itself is much more finicky. -- John Baldwin