From owner-svn-src-head@freebsd.org Mon Jul 27 04:07:40 2015 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4E2F99AC08D; Mon, 27 Jul 2015 04:07:40 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 3045D1E7; Mon, 27 Jul 2015 04:07:39 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-248-13.lns20.per4.internode.on.net [121.45.248.13]) (authenticated bits=0) by vps1.elischer.org (8.14.9/8.14.9) with ESMTP id t6R47RBe049244 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 26 Jul 2015 21:07:30 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: svn commit: r285854 - head/sys/amd64/include To: Alan Cox , John-Mark Gurney , Alan Cox References: <201507241943.t6OJhJaq090500@repo.freebsd.org> <20150724211532.GO78154@funkthat.com> <55B3CF86.200@rice.edu> Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org From: Julian Elischer Message-ID: <55B5AE79.3060505@freebsd.org> Date: Mon, 27 Jul 2015 12:07:21 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.10; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <55B3CF86.200@rice.edu> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Jul 2015 04:07:40 -0000 On 7/26/15 2:03 AM, Alan Cox wrote: > On 07/24/2015 16:15, John-Mark Gurney wrote: >> Alan Cox wrote this message on Fri, Jul 24, 2015 at 19:43 +0000: >>> Author: alc >>> Date: Fri Jul 24 19:43:18 2015 >>> New Revision: 285854 >>> URL: https://svnweb.freebsd.org/changeset/base/285854 >>> >>> Log: >>> Add a comment discussing the appropriate use of the atomic_*() functions >>> with acquire and release semantics versus the *mb() functions on amd64 >>> processors. >> Please put this documentation in the atomic(9) man page where it is >> easier to read and access... it's probably best to just move it >> there and reference atomic(9) here... >> >> Also, this advice isn't amd64 specific is it? If it isn't, why is it >> in an amd64 include file? > > While the first sentence is not amd64 specific, the core of this > paragraph, the third, four, and fifth sentences, is very amd64 specific. > In particular, the redundancy of the rmb() and wmb() functions for > ordinary cases of interprocessor memory ordering is not generally true > across architectures that we support. For example, on arm64 or powerpc, > these functions do provide non-redundant ordering. > > But, I do agree that the first sentence also belongs in a man page, like > atomic(9). Today, however, we have no man page documenting the *mb() > functions. > > >>> Modified: >>> head/sys/amd64/include/atomic.h >>> >>> Modified: head/sys/amd64/include/atomic.h >>> ============================================================================== >>> --- head/sys/amd64/include/atomic.h Fri Jul 24 19:37:30 2015 (r285853) >>> +++ head/sys/amd64/include/atomic.h Fri Jul 24 19:43:18 2015 (r285854) >>> @@ -32,6 +32,25 @@ >>> #error this file needs sys/cdefs.h as a prerequisite >>> #endif >>> >>> +/* >>> + * To express interprocessor (as opposed to processor and device) memory >>> + * ordering constraints, use the atomic_*() functions with acquire and release >>> + * semantics rather than the *mb() functions. An architecture's memory >>> + * ordering (or memory consistency) model governs the order in which a >>> + * program's accesses to different locations may be performed by an >>> + * implementation of that architecture. In general, for memory regions >>> + * defined as writeback cacheable, the memory ordering implemented by amd64 >>> + * processors preserves the program ordering of a load followed by a load, a >>> + * load followed by a store, and a store followed by a store. Only a store >>> + * followed by a load to a different memory location may be reordered. >>> + * Therefore, except for special cases, like non-temporal memory accesses or >>> + * memory regions defined as write combining, the memory ordering effects >>> + * provided by the sfence instruction in the wmb() function and the lfence >>> + * instruction in the rmb() function are redundant. In contrast, the >>> + * atomic_*() functions with acquire and release semantics do not perform >>> + * redundant instructions for ordinary cases of interprocessor memory >>> + * ordering on any architecture. >>> + */ >>> #define mb() __asm __volatile("mfence;" : : : "memory") >>> #define wmb() __asm __volatile("sfence;" : : : "memory") >>> #define rmb() __asm __volatile("lfence;" : : : "memory") It's getting long past the time when someone who understands the VM system writes a document about it. I think we have maybe 6 people who 'get it', and that pool needs to be expanded.. Maybe someone who has students can set an assignment for extra credit for someone to write a start and we cna keep fleshing it out :-) What there is, is now getting rather out of date, and much of the MACH documentation no longer applies. I'm willing to proof read and help but I don't know enough to actually write it. > > > >