From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:58:21 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EB4321065672; Tue, 29 Sep 2009 20:58:21 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-fx0-f222.google.com (mail-fx0-f222.google.com [209.85.220.222]) by mx1.freebsd.org (Postfix) with ESMTP id 27AA38FC2D; Tue, 29 Sep 2009 20:58:20 +0000 (UTC) Received: by fxm22 with SMTP id 22so1288874fxm.36 for ; Tue, 29 Sep 2009 13:58:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=JOz4+6q9qPXQ73j2H0FXSmtQzV6dZR/V2ET4kGQU/uI=; b=kevww5ZYTkWQ0hFx8Qf8uZHGZ1I4jnqqHJ5RUs4bItgKpgA9Q8AatCocdj7eXKySCI kU2Ud9y6R0M7LE+tLWp2iNvo9iklYySQkoYvGOrzRejKXLMqpbFOdllosfQOJWJpYEGe I3LUfBdxjev90SC2ZID/evIuFTUSRDzsj7Yag= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=m/VFIvHLoYnjIBK4Z+JY5eKvzLNTiUnusacSlFOde0NfwyFsl6h6TXQpk/uxmfjmVs Zf86ZmfUIsc9hqWWNFbb44IoyHAibS+4cX+Vaxq6OJ0rTSrocmd+zPZfNRpU8jhUVD8t FTM3wUoWM7V9VV/kU05hTqEnBWTbz1aJo1hLY= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.64.133 with SMTP id e5mr1444413fai.31.1254257899985; Tue, 29 Sep 2009 13:58:19 -0700 (PDT) In-Reply-To: References: <20090924224935.GW473@gandalf.sssup.it> <3bbf2fe10909290839w305c85c3t1532bd7733c39a6a@mail.gmail.com> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> Date: Tue, 29 Sep 2009 22:58:19 +0200 X-Google-Sender-Auth: d59edbff119ef1da Message-ID: <3bbf2fe10909291358q3063f763md4ccba88c3b1d0be@mail.gmail.com> From: Attilio Rao To: =?UTF-8?Q?Marius_N=C3=BCnnerich?= Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:58:22 -0000 2009/9/29 Marius N=C3=BCnnerich : > On Tue, Sep 29, 2009 at 21:15, Attilio Rao wrote: >> 2009/9/29 John Baldwin : >>> On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >>>> 2009/9/25 Fabio Checconi : >>>> > Hi all, >>>> > looking at sys/sx.h I have some troubles understanding this comment= : >>>> > >>>> > * A note about memory barriers. Exclusive locks need to use the sa= me >>>> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >>>> > * and _rel when releasing an exclusive lock. On the other side, >>>> > * shared lock needs to use an _acq barrier when acquiring the lock >>>> > * but, since they don't update any locked data, no memory barrier i= s >>>> > * needed when releasing a shared lock. >>>> > >>>> > In particular, I'm not understanding what prevents the following seq= uence >>>> > from happening: >>>> > >>>> > CPU A CPU B >>>> > >>>> > sx_slock(&data->lock); >>>> > >>>> > sx_sunlock(&data->lock); >>>> > >>>> > /* reordered after the unlock >>>> > by the cpu */ >>>> > if (data->buffer) >>>> > sx_xlock(&data->lock); >>>> > free(data->buffer); >>>> > data->buffer =3D NULL; >>>> > sx_xunlock(&data->lock); >>>> > >>>> > a =3D *data->buffer; >>>> > >>>> > IOW, even if readers do not modify the data protected by the lock, >>>> > without a release barrier a memory access may leak past the unlock (= as >>>> > the cpu won't notice any dependency between the unlock and the fetch= , >>>> > feeling free to reorder them), thus potentially racing with an exclu= sive >>>> > writer accessing the data. >>>> > >>>> > On architectures where atomic ops serialize memory accesses this wou= ld >>>> > never happen, otherwise the sequence above seems possible; am I miss= ing >>>> > something? >>>> >>>> I think your concerns are right, possibly we need this patch: >>>> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >>> >>> Actually, since you are only worried about reads, I think this should b= e >>> an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, = so we >>> should prefer the cheapest barrier that provides what we need. You wou= ld >>> still need to keep some language about the memory barriers since using = "acq" >>> for shared unlocking is different from exclusive unlocking. >> >> Actually, I don't think that an acq barrier ensures enough protection >> against the reordering of 'earlier' operation thus not fixing the >> architecture ordering problem reported by Fabio. Also, I don't think >> we just have to care about reads (or I don't understand what you mean >> here). >> However, I'm not even sure that we have faster read barriers than the >> write one. As long as it should be true in theory I don't think that's >> what happen in practice. >> >>> The memory clobber is quite heavyweight. It actually forces gcc to for= get any >>> cached memory items in registers and reload everything again. What I r= eally >>> want is just a barrier to tell GCC to not reorder things. If I read a = value >>> in the program before acquiring a lock it is in theory fine to keep tha= t >>> cached across the barrier. However, there isn't a way to do this sort = of >>> thing with GCC currently. >> >> Yes, that's the only tool we have right now with GCC. I will try to >> look for another way, but it sounds difficult to discover. > > Even if we would have a mechanism to tell GCC to not reorder the > instructions the CPU itself would still be free to reorder if there > are no barriers. Or am I missing something? Our code already takes care of that case in our barriers. Attilio --=20 Peace can only be achieved by understanding - A. Einstein