From owner-svn-src-all@FreeBSD.ORG Mon May 25 15:02:40 2015
Return-Path: <owner-svn-src-all@FreeBSD.ORG>
Delivered-To: svn-src-all@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
(using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
(No client certificate requested)
by hub.freebsd.org (Postfix) with ESMTPS id 0A6A6C65
for <svn-src-all@freebsd.org>; Mon, 25 May 2015 15:02:40 +0000 (UTC)
(envelope-from ian@freebsd.org)
Received: from pmta2.delivery6.ore.mailhop.org
(pmta2.delivery6.ore.mailhop.org [54.200.129.228])
by mx1.freebsd.org (Postfix) with SMTP id BE589908
for <svn-src-all@freebsd.org>; Mon, 25 May 2015 15:02:39 +0000 (UTC)
(envelope-from ian@freebsd.org)
Received: from ilsoft.org (unknown [73.34.117.227])
by outbound2.ore.mailhop.org (Halon Mail Gateway) with ESMTPSA;
Mon, 25 May 2015 15:02:51 +0000 (UTC)
Received: from revolution.hippie.lan (revolution.hippie.lan [172.22.42.240])
by ilsoft.org (8.14.9/8.14.9) with ESMTP id t4PF2WtV010640;
Mon, 25 May 2015 09:02:32 -0600 (MDT) (envelope-from ian@freebsd.org)
Message-ID: <1432566152.1200.34.camel@freebsd.org>
Subject: Re: svn commit: r283331 - head/sys/arm/arm
From: Ian Lepore <ian@freebsd.org>
To: John Baldwin <jhb@freebsd.org>
Cc: Andrew Turner <andrew@freebsd.org>, src-committers@freebsd.org,
svn-src-all@freebsd.org, svn-src-head@freebsd.org
Date: Mon, 25 May 2015 09:02:32 -0600
In-Reply-To: <1484557.5zjnsBffEc@ralph.baldwin.cx>
References: <201505232228.t4NMSxs2032365@svn.freebsd.org>
<1484557.5zjnsBffEc@ralph.baldwin.cx>
Content-Type: text/plain; charset="us-ascii"
X-Mailer: Evolution 3.12.10 FreeBSD GNOME Team Port
Mime-Version: 1.0
Content-Transfer-Encoding: 7bit
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.20
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for "
user" and " projects" \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/svn-src-all>,
<mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all/>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
<mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 25 May 2015 15:02:40 -0000
On Mon, 2015-05-25 at 07:23 -0400, John Baldwin wrote:
> On Saturday, May 23, 2015 10:28:59 PM Andrew Turner wrote:
> > Author: andrew
> > Date: Sat May 23 22:28:59 2015
> > New Revision: 283331
> > URL: https://svnweb.freebsd.org/changeset/base/283331
> >
> > Log:
> > Use the wait-for-event instruction to put the core we have just enabled
> > to sleep while it waits to start scheduling. The boot core can then use
> > the send-event instruction to wake the cores when they should enter the
> > scheduler.
> >
> > MFC after: 1 week
> >
> > Modified:
> > head/sys/arm/arm/mp_machdep.c
> >
> > Modified: head/sys/arm/arm/mp_machdep.c
> > ==============================================================================
> > --- head/sys/arm/arm/mp_machdep.c Sat May 23 21:58:41 2015 (r283330)
> > +++ head/sys/arm/arm/mp_machdep.c Sat May 23 22:28:59 2015 (r283331)
> > @@ -185,8 +185,11 @@ init_secondary(int cpu)
> > atomic_add_rel_32(&mp_naps, 1);
> >
> > /* Spin until the BSP releases the APs */
> > - while (!aps_ready)
> > - ;
> > + while (!atomic_load_acq_int(&aps_ready)) {
> > +#if __ARM_ARCH >= 7
> > + __asm __volatile("wfe");
> > +#endif
> > + }
>
> I don't know that this atomic load acquire is really changing
> anything here? Since aps_ready is volatile reading it should
> already be "atomic" on each check around the loop.
>
> > /* Initialize curthread */
> > KASSERT(PCPU_GET(idlethread) != NULL, ("no idle thread"));
> > @@ -353,6 +356,10 @@ release_aps(void *dummy __unused)
> > arm_unmask_irq(i);
> > }
> > atomic_store_rel_int(&aps_ready, 1);
> > + /* Wake the other threads up */
> > +#if __ARM_ARCH >= 7
> > + armv7_sev();
> > +#endif
>
> So I'm not at all familiar with these instructions or what they do,
> but are the events level triggered? In particular, is there any
> sort of race where the sev might arrive in between the check of
> aps_ready and the wfe on an AP? (For example, if wfe/sev were
> similar to using mwait on x86 for wfe and a memory write for sev,
> x86 would require a call to monitor before doing a check of
> aps_ready to handle the race like so:
>
> while (!aps_ready) {
> monitor(&aps_ready);
> if (!aps_ready)
> mwait();
> }
>
The arm send-event/wait-for-event system includes a 1-bit event latch
per core that indicates whether an event arrived since the prior wait.
The latch is checked atomically by the hardware as part of going into
low-power mode, so you can't get stuck waiting by an event-arrival race,
you just need to be prepared to handle spurious wakeups.
-- Ian