From owner-freebsd-arch@FreeBSD.ORG Thu May 14 04:47:51 2009 Return-Path: Delivered-To: arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E8D3E106566C for ; Thu, 14 May 2009 04:47:51 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from mail11.syd.optusnet.com.au (mail11.syd.optusnet.com.au [211.29.132.192]) by mx1.freebsd.org (Postfix) with ESMTP id 861D68FC0A for ; Thu, 14 May 2009 04:47:50 +0000 (UTC) (envelope-from brde@optusnet.com.au) Received: from besplex.bde.org (c122-107-117-19.carlnfd1.nsw.optusnet.com.au [122.107.117.19]) by mail11.syd.optusnet.com.au (8.13.1/8.13.1) with ESMTP id n4E4lgXH016169 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Thu, 14 May 2009 14:47:44 +1000 Date: Thu, 14 May 2009 14:47:41 +1000 (EST) From: Bruce Evans X-X-Sender: bde@besplex.bde.org To: Jeff Roberson In-Reply-To: Message-ID: <20090514131613.T1224@besplex.bde.org> References: <86bppy60ti.fsf@ds4.des.no> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: =?ISO-8859-15?Q?Dag-Erling_Sm=F8rgrav?= , arch@freebsd.org Subject: Re: lockless file descriptor lookup X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 May 2009 04:47:52 -0000 On Tue, 12 May 2009, Jeff Roberson wrote: > On Tue, 12 May 2009, Dag-Erling Sm?rgrav wrote: > >> Jeff Roberson writes: >>> I'd also appreciate it if someone could look at my volatile cast and >>> make sure I'm actually forcing the compiler to refresh the fd_ofiles >>> array here: >>> >>> + if (fp == ((struct file *volatile*)fdp->fd_ofiles)[fd]) This has 2 style bugs (missing space after first '*' and missing space before second '*'. It isn't clear whether you want to refresh the fd_ofiles pointer to the (first element of) the array, or the fd'th element. It is clear that you don't want to refresh the whole array. The above refreshes the fd'th element. Strangely, in my tests gcc refreshes the fd'th element even without the cast. E.g., test(fdp->fd_ofiles[fd], fdp->fd_ofiles[fd]); results in 1 memory access for each of the [fd]'s. >> The problem is that since it is not declared as volatile, some other >> piece of code may have modified it but not yet flushed it to RAM. > > That is an acceptable race due to other guarantees. If it hasn't been > committed to memory yet, the old table still contains valid data. I only > need to be certain that the compiler doesn't cache the original ofiles value. > It can't anyway because atomics use inline assembly on all platforms but I'd > like it to be explicit anyway. It shouldn't matter that atomics use inline asm. Non-broken inline asm declares all its inputs and outputs, so compilers can see what it changes just as easily as for C code (and more easily than for non- inline asm or C). Anyway, you probably need atomics that have suitable memory barriers. Memory barriers must affect the compiler and make it perform refreshes for them to work, so you shouldn't need any volatile casts. E.g., all atomic store operations (including cmpset) have release semantics even if they aren't spelled with "_rel" or implemented using inline asm. On amd64 and i386, they happen to be implemented using inline asm with "memory" clobbers. The "memory" clobbers force refreshes of all non-local variables. Bruce