From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:54:28 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B7C16106568D; Tue, 29 Sep 2009 20:54:28 +0000 (UTC) (envelope-from marius@nuenneri.ch) Received: from mail-ew0-f209.google.com (mail-ew0-f209.google.com [209.85.219.209]) by mx1.freebsd.org (Postfix) with ESMTP id C92038FC22; Tue, 29 Sep 2009 20:54:27 +0000 (UTC) Received: by ewy5 with SMTP id 5so4528719ewy.36 for ; Tue, 29 Sep 2009 13:54:26 -0700 (PDT) MIME-Version: 1.0 Received: by 10.216.53.205 with SMTP id g55mr1179248wec.160.1254256016832; Tue, 29 Sep 2009 13:26:56 -0700 (PDT) In-Reply-To: <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> Date: Tue, 29 Sep 2009 22:26:56 +0200 Message-ID: From: =?ISO-8859-1?Q?Marius_N=FCnnerich?= To: Attilio Rao Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:54:28 -0000 On Tue, Sep 29, 2009 at 21:15, Attilio Rao wrote: > 2009/9/29 John Baldwin : >> On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >>> 2009/9/25 Fabio Checconi : >>> > Hi all, >>> > =A0looking at sys/sx.h I have some troubles understanding this commen= t: >>> > >>> > =A0* A note about memory barriers. =A0Exclusive locks need to use the= same >>> > =A0* memory barriers as mutexes: _acq when acquiring an exclusive loc= k >>> > =A0* and _rel when releasing an exclusive lock. =A0On the other side, >>> > =A0* shared lock needs to use an _acq barrier when acquiring the lock >>> > =A0* but, since they don't update any locked data, no memory barrier = is >>> > =A0* needed when releasing a shared lock. >>> > >>> > In particular, I'm not understanding what prevents the following sequ= ence >>> > from happening: >>> > >>> > CPU A =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0= =A0 CPU B >>> > >>> > sx_slock(&data->lock); >>> > >>> > sx_sunlock(&data->lock); >>> > >>> > /* reordered after the unlock >>> > =A0 by the cpu */ >>> > if (data->buffer) >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0sx_xlock(&data->lock); >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0free(data->buffer); >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0data->buffer =3D NULL; >>> > =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 = =A0 =A0 =A0sx_xunlock(&data->lock); >>> > >>> > =A0 =A0 =A0 =A0a =3D *data->buffer; >>> > >>> > IOW, even if readers do not modify the data protected by the lock, >>> > without a release barrier a memory access may leak past the unlock (a= s >>> > the cpu won't notice any dependency between the unlock and the fetch, >>> > feeling free to reorder them), thus potentially racing with an exclus= ive >>> > writer accessing the data. >>> > >>> > On architectures where atomic ops serialize memory accesses this woul= d >>> > never happen, otherwise the sequence above seems possible; am I missi= ng >>> > something? >>> >>> I think your concerns are right, possibly we need this patch: >>> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> >> Actually, since you are only worried about reads, I think this should be >> an "acq" barrier rather than a "rel". =A0In some cases "acq" is cheaper,= so we >> should prefer the cheapest barrier that provides what we need. =A0You wo= uld >> still need to keep some language about the memory barriers since using "= acq" >> for shared unlocking is different from exclusive unlocking. > > Actually, I don't think that an acq barrier ensures enough protection > against the reordering of 'earlier' operation thus not fixing the > architecture ordering problem reported by Fabio. Also, I don't think > we just have to care about reads (or =A0I don't understand what you mean > here). > However, I'm not even sure that we have faster read barriers than the > write one. As long as it should be true in theory I don't think that's > what happen in practice. > >> The memory clobber is quite heavyweight. =A0It actually forces gcc to fo= rget any >> cached memory items in registers and reload everything again. =A0What I = really >> want is just a barrier to tell GCC to not reorder things. =A0If I read a= value >> in the program before acquiring a lock it is in theory fine to keep that >> cached across the barrier. =A0However, there isn't a way to do this sort= of >> thing with GCC currently. > > Yes, that's the only tool we have right now with GCC. I will try to > look for another way, but it sounds difficult to discover. Even if we would have a mechanism to tell GCC to not reorder the instructions the CPU itself would still be free to reorder if there are no barriers. Or am I missing something?