From owner-svn-src-user@FreeBSD.ORG Wed Oct 24 18:16:18 2012 Return-Path: Delivered-To: svn-src-user@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 22913366; Wed, 24 Oct 2012 18:16:18 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from bigwig.baldwin.cx (bigknife-pt.tunnel.tserv9.chi1.ipv6.he.net [IPv6:2001:470:1f10:75::2]) by mx1.freebsd.org (Postfix) with ESMTP id BC9558FC0A; Wed, 24 Oct 2012 18:16:17 +0000 (UTC) Received: from jhbbsd.localnet (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 08BB4B97D; Wed, 24 Oct 2012 14:16:17 -0400 (EDT) From: John Baldwin To: attilio@freebsd.org Subject: Re: svn commit: r241889 - in user/andre/tcp_workqueue/sys: arm/arm cddl/compat/opensolaris/kern cddl/contrib/opensolaris/uts/common/dtrace cddl/contrib/opensolaris/uts/common/fs/zfs ddb dev/acpica dev/... Date: Wed, 24 Oct 2012 14:14:30 -0400 User-Agent: KMail/1.13.5 (FreeBSD/8.2-CBSD-20110714-p20; KDE/4.5.5; amd64; ; ) References: <201210221418.q9MEINkr026751@svn.freebsd.org> <201210241136.06154.jhb@freebsd.org> In-Reply-To: MIME-Version: 1.0 Content-Type: Text/Plain; charset="utf-8" Content-Transfer-Encoding: 7bit Message-Id: <201210241414.30723.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.7 (bigwig.baldwin.cx); Wed, 24 Oct 2012 14:16:17 -0400 (EDT) Cc: mdf@freebsd.org, src-committers@freebsd.org, Andre Oppermann , Bruce Evans , svn-src-user@freebsd.org X-BeenThere: svn-src-user@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: "SVN commit messages for the experimental " user" src tree" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 24 Oct 2012 18:16:18 -0000 On Wednesday, October 24, 2012 11:41:24 am Attilio Rao wrote: > On Wed, Oct 24, 2012 at 4:36 PM, John Baldwin wrote: > > On Wednesday, October 24, 2012 11:24:22 am Attilio Rao wrote: > >> On Wed, Oct 24, 2012 at 4:09 PM, Attilio Rao wrote: > >> > On Wed, Oct 24, 2012 at 3:45 PM, John Baldwin wrote: > >> >> On Wednesday, October 24, 2012 10:34:34 am Attilio Rao wrote: > >> >>> On Wed, Oct 24, 2012 at 3:05 PM, John Baldwin wrote: > >> >>> > On Tuesday, October 23, 2012 7:20:04 pm Andre Oppermann wrote: > >> >>> >> On 24.10.2012 00:15, mdf@FreeBSD.org wrote: > >> >>> >> > On Tue, Oct 23, 2012 at 7:41 AM, Andre Oppermann > >> >> wrote: > >> >>> >> >> Struct mtx and MTX_SYSINIT always occur as pair next to each other. > >> >>> >> > > >> >>> >> > That doesn't matter. Language basics like variable definitions should > >> >>> >> > not be obscured by macros. It either takes longer to figure out what > >> >>> >> > a variable is (because one needs to look up the definition of the > >> >>> >> > macro) or makes it almost impossible (because now e.g. cscope doesn't > >> >>> >> > know this is a variable definition. > >> >>> >> > >> >>> >> Sigh, cscope doesn't expand macros? > >> >>> >> > >> >>> >> Is there a way to do the cache line alignment in a sane way without > >> >>> >> littering __aligned(CACHE_LINE_SIZE) all over the place? > >> >>> > > >> >>> > I was hoping to do something with an anonymous union or some such like: > >> >>> > > >> >>> > union mtx_aligned { > >> >>> > struct mtx; > >> >>> > char[roundup2(sizeof(struct mtx), CACHE_LINE_SIZE)]; > >> >>> > } > >> >>> > > >> >>> > I don't know if there is a useful way to define an 'aligned mutex' type > >> >>> > that will transparently map to a 'struct mtx', e.g.: > >> >>> > > >> >>> > typedef struct mtx __aligned(CACHE_LINE_SIZE) aligned_mtx_t; > >> >>> > >> >>> Unfortunately that doesn't work as I've verified with alc@ few months ago. > >> >>> The __aligned() attribute only works with structures definition, not > >> >>> objects declaration. > >> >> > >> >> Are you saying that the typedef doesn't (I expect it doesn't), or that this > >> >> doesn't: > >> >> > >> >> struct mtx foo __aligned(CACHE_LINE_SIZE); > >> > > >> > I meant to say that such notation won't address the padding issue > >> > which is as import as the alignment. Infact, for sensitive locks, > >> > having just an aligned object is not really useful if the cacheline > >> > gets shared. > >> > In the end you will need to use explicit padding or use __aligned in > >> > the struct definition, which cannot be used as a general pattern. > >> > >> The quickest way I see this can be made general is to have a specific > >> struct defined in sys/_mutex.h like that > >> > >> struct mtx_unshare { > >> struct mtx lock; > >> char _pad[CACHE_LINE_SIZE - sizeof(struct mtx)]; > >> } __aligned(CACHE_LINE_SIZE); > > > > I think instead you want my union above that uses roundup2 in case a lock > > eats up multiple cache lines: > > Do you think locks can eat more than one cacheline? This would be > absolutely killer for performance. Not the lock cookie, but 'struct lock_object', etc. aren't entirely trivial. If you had a 32-bit platform with a 16-byte cache line size I wouldn't be surprised if the entire structure spilled over a cacheline. > > union mtx_foo { > > struct mtx lock; > > char junk[roundup2(sizeof(struct mtx), CACHE_LINE_SIZE)]; > > } __aligned_CACHE_LINE_SIZE; > > > >> then let mtx_* functions to accept void ptrs and cast them to struct > >> mtx as long as the functions enter. > > > > Eh, that removes all compile time type checks. That seems very dubious to me. > > Well right now fast path already has a fair amount of macros wrapping > the operations, which don't really enforce any type checks. Sure they do. They still call a function that takes a 'struct mtx *' even if it isn't called in the fast path. If you pass a 'struct sx *' to mtx_lock() it will fail to compile. That needs to stay that way. -- John Baldwin