From owner-freebsd-arch@FreeBSD.ORG Tue Oct 28 02:52:28 2014 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id A52EEBB1; Tue, 28 Oct 2014 02:52:28 +0000 (UTC) Received: from mail-wg0-x22a.google.com (mail-wg0-x22a.google.com [IPv6:2a00:1450:400c:c00::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G2" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id C39D4D54; Tue, 28 Oct 2014 02:52:27 +0000 (UTC) Received: by mail-wg0-f42.google.com with SMTP id k14so5028055wgh.13 for ; Mon, 27 Oct 2014 19:52:26 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=date:from:to:cc:subject:message-id:mime-version:content-type :content-disposition:user-agent; bh=xjJ6BspOw1GD30+/rPxX3KdG4RukQlnlA2cnx0tgkoI=; b=GcltBRlOf1fjMv4YV/+OEDg9P8zVdTizH8ZDcqIGFgAs1QMqo79OFJaGPZdSZTnhjc MytewOutgwS5DGlPcYoAJAC7YkyKWPXLRttXE7JO51vVEjTmTAX/3N7FOHRCzPkvtbxZ gEdxFgaKvMmnzsyTRWgWgkl+2Jb3B+hjAEHuUbRyChGsLJ0hH5ZQA9nGhy3NSxiqEcu4 643hAeQCd9YNXsLXO/T2JPyCBpHtVVAWIvTHIX9ZYGPGMOcyOYRH0pNjB0iFYB0DcHs9 N2lplH6PiVAqj+6osCZUOKAZ0VdTA9vPrMg+K+SGAsqp/w/ruqxswtzSsdCLz83tWtfr MhSg== X-Received: by 10.180.90.65 with SMTP id bu1mr1089117wib.71.1414464746012; Mon, 27 Oct 2014 19:52:26 -0700 (PDT) Received: from dft-labs.eu (n1x0n-1-pt.tunnel.tserv5.lon1.ipv6.he.net. [2001:470:1f08:1f7::2]) by mx.google.com with ESMTPSA id dc8sm593915wib.7.2014.10.27.19.52.24 for (version=TLSv1.2 cipher=RC4-SHA bits=128/128); Mon, 27 Oct 2014 19:52:25 -0700 (PDT) Date: Tue, 28 Oct 2014 03:52:22 +0100 From: Mateusz Guzik To: freebsd-arch@freebsd.org Subject: atomic ops Message-ID: <20141028025222.GA19223@dft-labs.eu> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Disposition: inline User-Agent: Mutt/1.5.21 (2010-09-15) Cc: Attilio Rao , adrian@freebsd.org, Konstantin Belousov , Alan Cox X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Oct 2014 02:52:28 -0000 As was mentioned sometime ago, our situation related to atomic ops is not ideal. atomic_load_acq_* and atomic_store_rel_* (at least on amd64) provide full memory barriers, which is stronger than needed. Moreover, load is implemented as lock cmpchg on var address, so it is addditionally slower especially when cpus compete. On amd64 it is sufficient to place a compiler barrier in such cases. Next, we lack some atomic ops in the first place. Let's define some useful terms: smp_wmb - no writes can be reordered past this point smp_rmb - no reads can be reordered past this point With this in mind, we lack ops which would guarantee only the following: 1. var = tmp; smp_wmb(); 2. tmp = var; smp_rmb(); 3. smp_rmb(); tmp = var; This matters since what we can use already to emulate this is way heavier than needed on aforementioned amd64 and most likely other archs. It is unclear to me whether it makes sense to alter what atomic_load_acq_* are currently doing. The simplest thing would be to just introduce aforementioned macros. Unfortunately I don't have any ideas for new function names. I was considering stealing consumer/producer wording instead of acq/rel, but that does not help with case 1. Also there is no common header for atomic ops. I propose adding sys/atomic.h which includes machine/atomic.h. Then it would provide atomic ops missing from md header implemented using what is already there. For an example where it could be useful see https://svnweb.freebsd.org/base/head/sys/sys/seq.h?view=markup Comments? And yes, I know that: - atomic_load_acq_rmb_int is a terrible name and I'm trying to get rid of it - seq_consistent misses a read memory barrier, but in worst case this will result in spurious ENOTCAPABLE returned. security problem of circumventing capabilities is plugged since seq is properly re-checked before we return -- Mateusz Guzik