From owner-svn-src-user@FreeBSD.ORG Sun Oct 21 11:07:20 2012 Return-Path: Delivered-To: svn-src-user@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E9611ADA for ; Sun, 21 Oct 2012 11:07:20 +0000 (UTC) (envelope-from andre@freebsd.org) Received: from c00l3r.networx.ch (c00l3r.networx.ch [62.48.2.2]) by mx1.freebsd.org (Postfix) with ESMTP id 41B5B8FC0C for ; Sun, 21 Oct 2012 11:07:19 +0000 (UTC) Received: (qmail 48487 invoked from network); 21 Oct 2012 12:45:42 -0000 Received: from c00l3r.networx.ch (HELO [127.0.0.1]) ([62.48.2.2]) (envelope-sender ) by c00l3r.networx.ch (qmail-ldap-1.03) with SMTP for ; 21 Oct 2012 12:45:42 -0000 Message-ID: <5083D75F.6050603@freebsd.org> Date: Sun, 21 Oct 2012 13:07:11 +0200 From: Andre Oppermann User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:16.0) Gecko/20121010 Thunderbird/16.0.1 MIME-Version: 1.0 To: Gleb Smirnoff Subject: Re: svn commit: r241798 - in user/andre/tcp_workqueue/sys: kern sys References: <201210210846.q9L8kGX7029653@svn.freebsd.org> <20121021102155.GA64905@FreeBSD.org> In-Reply-To: <20121021102155.GA64905@FreeBSD.org> Content-Type: text/plain; charset=KOI8-R; format=flowed Content-Transfer-Encoding: 7bit Cc: src-committers@FreeBSD.org, svn-src-user@FreeBSD.org X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 21 Oct 2012 11:07:21 -0000 On 21.10.2012 12:21, Gleb Smirnoff wrote: > On Sun, Oct 21, 2012 at 08:46:16AM +0000, Andre Oppermann wrote: > A> Author: andre > A> Date: Sun Oct 21 08:46:15 2012 > A> New Revision: 241798 > A> URL: http://svn.freebsd.org/changeset/base/241798 > A> > A> Log: > A> Make sure that global locks have their own CPU cache line in > A> the .bss section of the kernel and do not end up sharing one > A> by chance or the place they are defined in. > A> > A> Three new macros are added: > A> MTX_GLOBAL(name) defining the mutex > A> MTX_GLOBAL_STATIC(name) defining the mutex as static > A> MTX_GLB(name) accessing the mutex in mtx_lock() > A> > A> This makes global locks cache deterministic. Changes to the > A> .bss layout and ordering due to differences in completely > A> unrelated parts of the kernel can no longer cause global > A> locks to share the same CPU cache line. > A> > A> This is a proof of concept with only one global lock converted. > A> > A> I'm open to suggestions and improvements on the macros and > A> their naming. > > My fault, I have confused you yesterday in our discussion. Actually > we don't need to embrace declaration into additional struct. > This code works: > > static struct mtx __aligned(CACHE_LINE_SIZE) foo_mtx; OK, I see. That makes it simpler. I'll still do it with a macro to do the SMP/UP differentiation. > When speaking about embracing, I actually meant that we can embrace > into anonymous struct a mutex and a structures it protects, not a > mutex only. In most cases the global locks serve as a serialization point protecting more than one structure. So it's difficult to chose where to put it. > P.S. Before merging such changes to head we need to do some benchmarking > proving that this isn't a nop. Ideally it is a nop. It's about being deterministic and preventing bad things should cache line sharing happen. -- Andre