From owner-freebsd-arch@FreeBSD.ORG Tue Oct 28 13:43:04 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9194EDEF; Tue, 28 Oct 2014 13:43:04 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 34110D26; Tue, 28 Oct 2014 13:43:04 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id s9SDgto5027853 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 28 Oct 2014 15:42:55 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua s9SDgto5027853 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id s9SDgtSQ027852; Tue, 28 Oct 2014 15:42:55 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 28 Oct 2014 15:42:54 +0200 From: Konstantin Belousov To: Mateusz Guzik Subject: Re: atomic ops Message-ID: <20141028134254.GD1877@kib.kiev.ua> References: <20141028025222.GA19223@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20141028025222.GA19223@dft-labs.eu> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: Attilio Rao , adrian@freebsd.org, Alan Cox , freebsd-arch@freebsd.org X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Oct 2014 13:43:04 -0000 On Tue, Oct 28, 2014 at 03:52:22AM +0100, Mateusz Guzik wrote: > As was mentioned sometime ago, our situation related to atomic ops is > not ideal. > > atomic_load_acq_* and atomic_store_rel_* (at least on amd64) provide > full memory barriers, which is stronger than needed. x86 atomic_store_rel() does not establish any cpu barrier, due to the already provided guarantees of the architecture. > > Moreover, load is implemented as lock cmpchg on var address, so it is > addditionally slower especially when cpus compete. > > On amd64 it is sufficient to place a compiler barrier in such cases. > > Next, we lack some atomic ops in the first place. > > Let's define some useful terms: > smp_wmb - no writes can be reordered past this point > smp_rmb - no reads can be reordered past this point > > With this in mind, we lack ops which would guarantee only the following: > > 1. var = tmp; smp_wmb(); > 2. tmp = var; smp_rmb(); > 3. smp_rmb(); tmp = var; > > This matters since what we can use already to emulate this is way > heavier than needed on aforementioned amd64 and most likely other archs. > > It is unclear to me whether it makes sense to alter what > atomic_load_acq_* are currently doing. I still think that our load/stores, comparing with the classic definition of the operations, are ordered, i.e. what is called sequential consistent in the C standard. I have no idea if we want this property, or is it used really. The kern_intr.c (ab)uses load in this way. > > The simplest thing would be to just introduce aforementioned macros. > > Unfortunately I don't have any ideas for new function names. > > I was considering stealing consumer/producer wording instead of acq/rel, > but that does not help with case 1. > > Also there is no common header for atomic ops. > > I propose adding sys/atomic.h which includes machine/atomic.h. Then it > would provide atomic ops missing from md header implemented using what > is already there. > > For an example where it could be useful see > https://svnweb.freebsd.org/base/head/sys/sys/seq.h?view=markup > > Comments? > > And yes, I know that: > - atomic_load_acq_rmb_int is a terrible name and I'm trying to get rid > of it > - seq_consistent misses a read memory barrier, but in worst case this > will result in spurious ENOTCAPABLE returned. security problem of > circumventing capabilities is plugged since seq is properly re-checked > before we return > > -- > Mateusz Guzik