From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 19:15:42 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 31135106566B; Tue, 29 Sep 2009 19:15:42 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f210.google.com (mail-bw0-f210.google.com [209.85.218.210]) by mx1.freebsd.org (Postfix) with ESMTP id 598608FC36; Tue, 29 Sep 2009 19:15:41 +0000 (UTC) Received: by mail-bw0-f210.google.com with SMTP id 6so2267243bwz.37 for ; Tue, 29 Sep 2009 12:15:41 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=vqn7gWCxCd9ic9U1E6fckFGgXcCxbJDMfdPW0a6y8to=; b=cFWjTz4NbgZn+quPU9ufVLcoDgFcAlmDwpvGk8neatyqokW/JccIiES7sNABBQwjMV CsqCn3to3krGE5a3Kmv4pIiKXFXL8Sa9Eq39tWB1gYP/69VvldLu9kkuEEuh81GFVCUu LV3ZepubrlDtcFp4yhALeBchvXKsyrcEgLWow= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=XaIJvlEW41yd1zEa8txTYV3C+xwPLV4h9k5JBP58OvB2xwcCrQHq7YNZXjuz4Jf6Ln bapLUT/OOyX37He8vBEI1O8X7gX6m3C75/w4xxqToVbx1tdDOjZ9G//YUiCsMtyU81E+ LjHe4+Xbeex2+2I4U2Zh11/DsoiWCh8xUqmW0= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.60.68 with SMTP id o4mr1395543fah.102.1254251740833; Tue, 29 Sep 2009 12:15:40 -0700 (PDT) In-Reply-To: <200909291425.46134.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291425.46134.jhb@freebsd.org> Date: Tue, 29 Sep 2009 21:15:40 +0200 X-Google-Sender-Auth: 5e2c84c90e02b721 Message-ID: <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 19:15:42 -0000 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >> 2009/9/25 Fabio Checconi : >> > Hi all, >> > looking at sys/sx.h I have some troubles understanding this comment: >> > >> > * A note about memory barriers. Exclusive locks need to use the same >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> > * and _rel when releasing an exclusive lock. On the other side, >> > * shared lock needs to use an _acq barrier when acquiring the lock >> > * but, since they don't update any locked data, no memory barrier is >> > * needed when releasing a shared lock. >> > >> > In particular, I'm not understanding what prevents the following sequence >> > from happening: >> > >> > CPU A CPU B >> > >> > sx_slock(&data->lock); >> > >> > sx_sunlock(&data->lock); >> > >> > /* reordered after the unlock >> > by the cpu */ >> > if (data->buffer) >> > sx_xlock(&data->lock); >> > free(data->buffer); >> > data->buffer = NULL; >> > sx_xunlock(&data->lock); >> > >> > a = *data->buffer; >> > >> > IOW, even if readers do not modify the data protected by the lock, >> > without a release barrier a memory access may leak past the unlock (as >> > the cpu won't notice any dependency between the unlock and the fetch, >> > feeling free to reorder them), thus potentially racing with an exclusive >> > writer accessing the data. >> > >> > On architectures where atomic ops serialize memory accesses this would >> > never happen, otherwise the sequence above seems possible; am I missing >> > something? >> >> I think your concerns are right, possibly we need this patch: >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff > > Actually, since you are only worried about reads, I think this should be > an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, so we > should prefer the cheapest barrier that provides what we need. You would > still need to keep some language about the memory barriers since using "acq" > for shared unlocking is different from exclusive unlocking. Actually, I don't think that an acq barrier ensures enough protection against the reordering of 'earlier' operation thus not fixing the architecture ordering problem reported by Fabio. Also, I don't think we just have to care about reads (or I don't understand what you mean here). However, I'm not even sure that we have faster read barriers than the write one. As long as it should be true in theory I don't think that's what happen in practice. > The memory clobber is quite heavyweight. It actually forces gcc to forget any > cached memory items in registers and reload everything again. What I really > want is just a barrier to tell GCC to not reorder things. If I read a value > in the program before acquiring a lock it is in theory fine to keep that > cached across the barrier. However, there isn't a way to do this sort of > thing with GCC currently. Yes, that's the only tool we have right now with GCC. I will try to look for another way, but it sounds difficult to discover. Attilio -- Peace can only be achieved by understanding - A. Einstein