From owner-freebsd-current@FreeBSD.ORG Tue May 17 11:56:08 2011 Return-Path: Delivered-To: freebsd-current@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 23C0A106566B; Tue, 17 May 2011 11:56:08 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [65.122.17.42]) by mx1.freebsd.org (Postfix) with ESMTP id EC7808FC08; Tue, 17 May 2011 11:56:07 +0000 (UTC) Received: from bigwig.baldwin.cx (66.111.2.69.static.nyinternet.net [66.111.2.69]) by cyrus.watson.org (Postfix) with ESMTPSA id 8A7CC46B0A; Tue, 17 May 2011 07:56:07 -0400 (EDT) Received: from jhbmac.hudson-trading.com (unknown [209.249.190.124]) by bigwig.baldwin.cx (Postfix) with ESMTPSA id 172D68A04F; Tue, 17 May 2011 07:56:07 -0400 (EDT) Message-ID: <4DD26256.2070008@FreeBSD.org> Date: Tue, 17 May 2011 07:56:06 -0400 From: John Baldwin User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.17) Gecko/20110414 Thunderbird/3.1.10 MIME-Version: 1.0 To: Andriy Gapon References: <4DCD357D.6000109@FreeBSD.org> <201105161421.27665.jhb@freebsd.org> <4DD17AB3.1070606@FreeBSD.org> <201105161609.21898.jhb@freebsd.org> <4DD22BD9.6070504@FreeBSD.org> In-Reply-To: <4DD22BD9.6070504@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH, not delayed by milter-greylist-4.2.6 (bigwig.baldwin.cx); Tue, 17 May 2011 07:56:07 -0400 (EDT) Cc: Max Laier , FreeBSD current , neel@FreeBSD.org, Peter Grehan Subject: Re: proposed smp_rendezvous change X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 17 May 2011 11:56:08 -0000 On 5/17/11 4:03 AM, Andriy Gapon wrote: > on 16/05/2011 23:09 John Baldwin said the following: >> is probably just cut and pasted to match the other uses of values in >> the smp_rv_waiters[] array. >> >> (atomic_add_acq_int() could spin on architectures where it is implemented >> using compare-and-swap (e.g. sparc64) or locked-load conditional-store (e.g. >> Alpha).) > > > When you say "not strictly necessary", do you mean "not necessary"? > If you do not mean that, then when could it be (non-strictly) necessary? :) > > Couldn't [Shouldn't] the whole: > >>>> /* Ensure we have up-to-date values. */ >>>> atomic_add_acq_int(&smp_rv_waiters[0], 1); >>>> while (smp_rv_waiters[0]< smp_rv_ncpus) >>>> cpu_spinwait(); > > be just replaced with: > > rmb(); > > Or a proper MI function that does just a read memory barrier, if rmb() is not that. No, you could replace it with: atomic_add_acq_int(&smp_rv_waiters[0], 1); The key being that atomic_add_acq_int() will block (either in hardware or software) until it can safely perform the atomic operation. That means waiting until the write to set smp_rv_waiters[0] to 0 by the rendezvous initiator is visible to the current CPU. On some platforms a write by one CPU may not post instantly to other CPUs (e.g. it may sit in a store buffer). That is fine so long as an attempt to update that value atomically (using cas or a conditional-store, etc.) fails. For those platforms, the atomic(9) API is required to spin until it succeeds. This is why the mtx code spins if it can't set MTX_CONTESTED for example. -- John Baldwin