From owner-freebsd-mips@FreeBSD.ORG Mon Nov 5 17:38:36 2012 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 1853D4E1 for ; Mon, 5 Nov 2012 17:38:36 +0000 (UTC) (envelope-from freebsd@damnhippie.dyndns.org) Received: from duck.symmetricom.us (duck.symmetricom.us [206.168.13.214]) by mx1.freebsd.org (Postfix) with ESMTP id F27E58FC0C for ; Mon, 5 Nov 2012 17:38:34 +0000 (UTC) Received: from damnhippie.dyndns.org (daffy.symmetricom.us [206.168.13.218]) by duck.symmetricom.us (8.14.5/8.14.5) with ESMTP id qA5HcKtH018185 for ; Mon, 5 Nov 2012 10:38:27 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Received: from [172.22.42.240] (revolution.hippie.lan [172.22.42.240]) by damnhippie.dyndns.org (8.14.3/8.14.3) with ESMTP id qA5Hc7GX012470; Mon, 5 Nov 2012 10:38:07 -0700 (MST) (envelope-from freebsd@damnhippie.dyndns.org) Subject: Re: CACHE_LINE_SIZE macro. From: Ian Lepore To: Warner Losh In-Reply-To: References: <201211041828.qA4ISomC076058@pdx.rh.CN85.ChatUSA.com> Content-Type: text/plain; charset="us-ascii" Date: Mon, 05 Nov 2012 10:38:07 -0700 Message-ID: <1352137087.1120.180.camel@revolution.hippie.lan> Mime-Version: 1.0 X-Mailer: Evolution 2.32.1 FreeBSD GNOME Team Port Content-Transfer-Encoding: 7bit Cc: "Rodney W. Grimes" , Juli Mallett , "freebsd-mips@FreeBSD.org" X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Nov 2012 17:38:36 -0000 On Mon, 2012-11-05 at 10:11 -0700, Warner Losh wrote: > On Nov 5, 2012, at 10:01 AM, Eitan Adler wrote: > > > On 5 November 2012 11:49, Warner Losh wrote: > >>> There has been some discussion recently about padding lock mutexs to > >>> the cache line size in order to avoid false sharing of CPUs. Some have > >>> claimed to see significant performance increases as a result. > >> > >> Is that an out-of-kernel interface? > >> > >> If we did that, we'd have to make it run-time settable, because there's no one right answer for arm and MIPS cpus: they are all different. > > > > The discussion ended up with using a special parameter > > CACHE_LINE_SIZE_LOCKS which is different than CACHE_LINE_SIZE. This is > > necessary for other reasons as well (CACHE_LINE_SIZE_LOCKS may take > > into account prefetching of cache lines, but CACHE_LINE_SIZE > > wouldn't). > > > > I think the "correct" thing to do here is choose a reasonable, but > > not-always-correct CACHE_LINE_SIZE_LOCKS and make CACHE_LINE_SIZE a > > per-board constant (or run time setting, or whatever works). You > > can't make it run-time settable as the padding is part of the ABI: > > > > For more details see > > http://comments.gmane.org/gmane.os.freebsd.devel.cvs/483696 > > which contains the original discussion. > > > > Note - I was not involved. > > this is a kernel-only interface, so compile time constants are fine there. What user-land visible interfaces are affected by this setting? The answer should be 'none' > > Warner When I commented on Attilio's recent checkins concerning padding of locks to cache line size and the fact that the value changes per-cpu and we're not well-positioned to handle that right now, his main concern was modules matching the kernel. I had suggested making the padding conditional on SMP (because apparently there's no benefit to the padding in a UP kernel), but then a module compiled for UP wouldn't work right on an SMP kernel, and vice versa. I'm not sure why that's a problem, my solution to that would be "So then don't do that." What scares me the most is the mushy definition of what CACHE_LINE_SIZE really means. There's nothing about the name that says "This may not be the actual cache line size but it's probably close," but increasingly I see people talking about it as if it had such a malleable meaning. Is that consistant with the existing uses in the code? Is it a good idea? -- Ian