From owner-freebsd-arch@FreeBSD.ORG Thu Oct 25 21:33:38 2012 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 141F37F4; Thu, 25 Oct 2012 21:33:38 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id D99F38FC17; Thu, 25 Oct 2012 21:33:37 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 3F893B924; Thu, 25 Oct 2012 17:33:37 -0400 (EDT) From: John Baldwin To: Andre Oppermann Subject: Re: CACHE_LINE_SIZE on x86 Date: Thu, 25 Oct 2012 17:32:31 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <201210250918.00602.jhb@freebsd.org> <5089690A.8070503@networx.ch> In-Reply-To: <5089690A.8070503@networx.ch> MIME-Version: 1.0 Content-Type: Text/Plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Message-Id: <201210251732.31631.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Thu, 25 Oct 2012 17:33:37 -0400 (EDT) Cc: Jim Harris , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 25 Oct 2012 21:33:38 -0000 On Thursday, October 25, 2012 12:30:02 pm Andre Oppermann wrote: > On 25.10.2012 15:18, John Baldwin wrote: > > On Wednesday, October 24, 2012 3:13:38 pm Jim Harris wrote: > >> While investigating padding of the ULE scheduler locks (r242014), I > >> recently discovered that CACHE_LINE_SIZE on x86 is defined as 128 (not > >> 64). From what I can tell from svn logs, this was to account for 128 > >> byte cache "sectors" that existed on the NetBurst micro architecture > >> CPUs. > >> > >> I'm curious if there's been consideration in changing this back to 64? > >> With maybe a kernel config option to modify it? On 2S systems (but > >> not on 1S systems), I see a benefit using CACHE_LINE_SIZE=128 for the > >> scheduler locks. I suspect this is related to data prefetching but am > >> still running experiments to verify. > > > > All the i7 and later systems I've seen (maybe even Penryn?) have a BIOS option > > (typically enabled by default) to enable adjacent cache line prefetching (my > > understanding is that this only affects the LLC, and it seems to always fetch > > an aligned 128 bytes, so if your miss is in the "second" line it fetches N-1 > > and N, not always fetching N and N+1). That is why I thought we still use 128 > > bytes on x86. > > As long as the additionally prefetched cache line has its own MOESI > state and gets marked as shared there is not problem with using only > 64B alignment and padding. It would be good to know though if there are performance benefits from avoiding sharing across paired lines in this manner. Even if it has its own MOESI state, there might still be negative effects from sharing the pair. -- John Baldwin