From owner-svn-src-head@FreeBSD.ORG Mon May 25 12:30:39 2015 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 9D8A1CF4; Mon, 25 May 2015 12:30:39 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from kif.fubar.geek.nz (kif.fubar.geek.nz [178.62.119.249]) by mx1.freebsd.org (Postfix) with ESMTP id 693DCD4A; Mon, 25 May 2015 12:30:39 +0000 (UTC) (envelope-from andrew@fubar.geek.nz) Received: from bender.Home (97e07b11.skybroadband.com [151.224.123.17]) by kif.fubar.geek.nz (Postfix) with ESMTPSA id 8429FD7A6F; Mon, 25 May 2015 12:21:49 +0000 (UTC) Date: Mon, 25 May 2015 13:21:48 +0100 From: Andrew Turner To: John Baldwin Cc: Andrew Turner , src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r283331 - head/sys/arm/arm Message-ID: <20150525132148.3d5adb40@bender.Home> In-Reply-To: <1484557.5zjnsBffEc@ralph.baldwin.cx> References: <201505232228.t4NMSxs2032365@svn.freebsd.org> <1484557.5zjnsBffEc@ralph.baldwin.cx> X-Mailer: Claws Mail 3.11.1 (GTK+ 2.24.27; amd64-portbld-freebsd10.0) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 May 2015 12:30:39 -0000 On Mon, 25 May 2015 07:23:28 -0400 John Baldwin wrote: > On Saturday, May 23, 2015 10:28:59 PM Andrew Turner wrote: > > Author: andrew > > Date: Sat May 23 22:28:59 2015 > > New Revision: 283331 > > URL: https://svnweb.freebsd.org/changeset/base/283331 > > > > Log: > > Use the wait-for-event instruction to put the core we have just > > enabled to sleep while it waits to start scheduling. The boot core > > can then use the send-event instruction to wake the cores when they > > should enter the scheduler. > > > > MFC after: 1 week > > > > Modified: > > head/sys/arm/arm/mp_machdep.c > > > > Modified: head/sys/arm/arm/mp_machdep.c > > ============================================================================== > > --- head/sys/arm/arm/mp_machdep.c Sat May 23 21:58:41 > > 2015 (r283330) +++ head/sys/arm/arm/mp_machdep.c Sat > > May 23 22:28:59 2015 (r283331) @@ -185,8 +185,11 @@ > > init_secondary(int cpu) atomic_add_rel_32(&mp_naps, 1); > > > > /* Spin until the BSP releases the APs */ > > - while (!aps_ready) > > - ; > > + while (!atomic_load_acq_int(&aps_ready)) { > > +#if __ARM_ARCH >= 7 > > + __asm __volatile("wfe"); > > +#endif > > + } > > I don't know that this atomic load acquire is really changing > anything here? Since aps_ready is volatile reading it should > already be "atomic" on each check around the loop. It's also adding acquire semantics to ensure we don't incorrectly reorder memory operations across the call. > > > /* Initialize curthread */ > > KASSERT(PCPU_GET(idlethread) != NULL, ("no idle thread")); > > @@ -353,6 +356,10 @@ release_aps(void *dummy __unused) > > arm_unmask_irq(i); > > } > > atomic_store_rel_int(&aps_ready, 1); > > + /* Wake the other threads up */ > > +#if __ARM_ARCH >= 7 > > + armv7_sev(); > > +#endif > > So I'm not at all familiar with these instructions or what they do, > but are the events level triggered? In particular, is there any > sort of race where the sev might arrive in between the check of > aps_ready and the wfe on an AP? (For example, if wfe/sev were > similar to using mwait on x86 for wfe and a memory write for sev, > x86 would require a call to monitor before doing a check of > aps_ready to handle the race like so: > > while (!aps_ready) { > monitor(&aps_ready); > if (!aps_ready) > mwait(); > } > The armv7_sev function includes a barrier to ensure any previous memory operations have been flushed to cache before we send the event. The sev instruction then sets the event register in every processor. The wfe instruction will check this event register and, if it is unset, it the processor can then enter a low power mode. From my reading of the documentation, if an event has been signalled before executing the wfe then the instruction is a nop so will exit the loop as the new value of aps_ready will be visible on all processors. Andrew