From owner-freebsd-hackers@FreeBSD.ORG Tue Sep 29 20:39:45 2009 Return-Path: Delivered-To: hackers@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D52061065679; Tue, 29 Sep 2009 20:39:45 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f227.google.com (mail-bw0-f227.google.com [209.85.218.227]) by mx1.freebsd.org (Postfix) with ESMTP id EAD148FC0C; Tue, 29 Sep 2009 20:39:44 +0000 (UTC) Received: by bwz27 with SMTP id 27so4194805bwz.43 for ; Tue, 29 Sep 2009 13:39:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type; bh=qW2U9d8YpA940/f8735p54sjdZcjtB8DtqOpOhb2JRo=; b=nsgsjMVhC90V7AkgFv2XdrMq5MSRJrNNkbWHOCVdzA3ydhedn2lFzTRI61DC+j9g2b driLt8OuPamhNMEROwoqobvLgFEBYfg4NQ8sPtEFh/IgLUZv5EpZ7kR4XCikKrxsmKUp gv68lMlMDv8qV/vwRLT2fLldfhpWvh6EY1xPs= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type; b=Sr7sO7JO0eXjJYukcSVV9Nbkipsna83JP9QR5KJjIA7fNLPTCeykNZDVUfpky+/LVc p94TJLeDiJ5d81/YntLkLEyaRq+jpcoI9A26/XX9qgDdzF1EDspAjkz+efe8QEnaf/H1 Wl+GuDR3Ty85S34QTQI0tDjm6EM+h53dloWqQ= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.6.9 with SMTP id 9mr1469442fax.84.1254256783571; Tue, 29 Sep 2009 13:39:43 -0700 (PDT) In-Reply-To: <200909291628.15516.jhb@freebsd.org> References: <20090924224935.GW473@gandalf.sssup.it> <200909291425.46134.jhb@freebsd.org> <3bbf2fe10909291215i2bdd73aj13c1ac433152cab4@mail.gmail.com> <200909291628.15516.jhb@freebsd.org> Date: Tue, 29 Sep 2009 22:39:43 +0200 X-Google-Sender-Auth: 2055cafbe55f7990 Message-ID: <3bbf2fe10909291339s5705a9bendb4c9331293b45a4@mail.gmail.com> From: Attilio Rao To: John Baldwin Content-Type: text/plain; charset=UTF-8 Cc: hackers@freebsd.org, Fabio Checconi Subject: Re: sx locks and memory barriers X-BeenThere: freebsd-hackers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Technical Discussions relating to FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2009 20:39:46 -0000 2009/9/29 John Baldwin : > On Tuesday 29 September 2009 3:15:40 pm Attilio Rao wrote: >> 2009/9/29 John Baldwin : >> > On Tuesday 29 September 2009 11:39:37 am Attilio Rao wrote: >> >> 2009/9/25 Fabio Checconi : >> >> > Hi all, >> >> > looking at sys/sx.h I have some troubles understanding this comment: >> >> > >> >> > * A note about memory barriers. Exclusive locks need to use the same >> >> > * memory barriers as mutexes: _acq when acquiring an exclusive lock >> >> > * and _rel when releasing an exclusive lock. On the other side, >> >> > * shared lock needs to use an _acq barrier when acquiring the lock >> >> > * but, since they don't update any locked data, no memory barrier is >> >> > * needed when releasing a shared lock. >> >> > >> >> > In particular, I'm not understanding what prevents the following sequence >> >> > from happening: >> >> > >> >> > CPU A CPU B >> >> > >> >> > sx_slock(&data->lock); >> >> > >> >> > sx_sunlock(&data->lock); >> >> > >> >> > /* reordered after the unlock >> >> > by the cpu */ >> >> > if (data->buffer) >> >> > sx_xlock(&data->lock); >> >> > free(data->buffer); >> >> > data->buffer = NULL; >> >> > sx_xunlock(&data->lock); >> >> > >> >> > a = *data->buffer; >> >> > >> >> > IOW, even if readers do not modify the data protected by the lock, >> >> > without a release barrier a memory access may leak past the unlock (as >> >> > the cpu won't notice any dependency between the unlock and the fetch, >> >> > feeling free to reorder them), thus potentially racing with an exclusive >> >> > writer accessing the data. >> >> > >> >> > On architectures where atomic ops serialize memory accesses this would >> >> > never happen, otherwise the sequence above seems possible; am I missing >> >> > something? >> >> >> >> I think your concerns are right, possibly we need this patch: >> >> http://www.freebsd.org/~attilio/sxrw_unlockb.diff >> > >> > Actually, since you are only worried about reads, I think this should be >> > an "acq" barrier rather than a "rel". In some cases "acq" is cheaper, so we >> > should prefer the cheapest barrier that provides what we need. You would >> > still need to keep some language about the memory barriers since using "acq" >> > for shared unlocking is different from exclusive unlocking. >> >> Actually, I don't think that an acq barrier ensures enough protection >> against the reordering of 'earlier' operation thus not fixing the >> architecture ordering problem reported by Fabio. Also, I don't think >> we just have to care about reads (or I don't understand what you mean >> here). > > Hmmm, it might not on certain archs. It does on x86 (i.e. an lfence would > work here on x86), but probably not on ia64 and sparc64. Also, we certainly > only care about reads. A read/share lock cannot resolve any races where the > lock holder is writing data, it can only ensure that the lock holder can > safely read shared data without the data changing out from underneath it. > >> However, I'm not even sure that we have faster read barriers than the >> write one. As long as it should be true in theory I don't think that's >> what happen in practice. > > bde@ found that sfence was generally much more expensive than lfence on x86. > However, since x86 guarantees the order of all stores we don't actually need > to use sfence at all on x86 anyway. Yes, x86 guarantees that the stores are strong ordered so I don't think acq to be faster than rel. Can I assume the patch I already sent as reviewed by you and commit then, right? Attilio -- Peace can only be achieved by understanding - A. Einstein