From owner-svn-src-head@freebsd.org Tue Oct 3 02:10:39 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 40118E2D567 for ; Tue, 3 Oct 2017 02:10:39 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: from asp.reflexion.net (outbound-mail-210-28.reflexion.net [208.70.210.28]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id EA0AF83FDB for ; Tue, 3 Oct 2017 02:10:38 +0000 (UTC) (envelope-from markmi@dsl-only.net) Received: (qmail 15568 invoked from network); 3 Oct 2017 02:10:36 -0000 Received: from unknown (HELO mail-cs-01.app.dca.reflexion.local) (10.81.19.1) by 0 (rfx-qmail) with SMTP; 3 Oct 2017 02:10:36 -0000 Received: by mail-cs-01.app.dca.reflexion.local (Reflexion email security v8.40.3) with SMTP; Mon, 02 Oct 2017 22:10:36 -0400 (EDT) Received: (qmail 2206 invoked from network); 3 Oct 2017 02:10:36 -0000 Received: from unknown (HELO iron2.pdx.net) (69.64.224.71) by 0 (rfx-qmail) with (AES256-SHA encrypted) SMTP; 3 Oct 2017 02:10:36 -0000 Received: from [192.168.1.26] (c-76-115-7-162.hsd1.or.comcast.net [76.115.7.162]) by iron2.pdx.net (Postfix) with ESMTPSA id 03010EC9411; Mon, 2 Oct 2017 19:10:35 -0700 (PDT) From: Mark Millard Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\)) Subject: Re: svn commit: r324207 - head/sys/arm64/arm64 Message-Id: Date: Mon, 2 Oct 2017 19:10:35 -0700 To: andrew@freebsd.org, svn-src-head@freebsd.org, freebsd-arm X-Mailer: Apple Mail (2.3273) X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 03 Oct 2017 02:10:39 -0000 > Author: andrew > Date: Mon Oct 2 14:22:35 2017 > New Revision: 324207 > URL:=20 > https://svnweb.freebsd.org/changeset/base/324207 >=20 >=20 > Log: > Add a memory barrier to ensure the atomic write is visible to the = other > CPUs before waking them up. > =20 > Sponsored by: DARPA, AFRL >=20 > Modified: > head/sys/arm64/arm64/mp_machdep.c >=20 > Modified: head/sys/arm64/arm64/mp_machdep.c > = =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D > --- head/sys/arm64/arm64/mp_machdep.c Mon Oct 2 14:19:31 2017 = (r324206) > +++ head/sys/arm64/arm64/mp_machdep.c Mon Oct 2 14:22:35 2017 = (r324207) > @@ -236,7 +236,10 @@ release_aps(void *dummy __unused) > =20 > atomic_store_rel_int(&aps_ready, 1); > /* Wake up the other CPUs */ > - __asm __volatile("sev"); > + __asm __volatile( > + "dsb ishst \n" > + "sev \n" > + ::: "memory"); > =20 > printf("Release APs\n"); There is another sev instruction without a prior memory barrier: Index: /usr/src/sys/arm64/arm64/identcpu.c =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D --- /usr/src/sys/arm64/arm64/identcpu.c (revision 323676) +++ /usr/src/sys/arm64/arm64/identcpu.c (working copy) @@ -1109,6 +1109,9 @@ =20 /* Wake up the other CPUs */ atomic_store_rel_int(&ident_lock, 0); - __asm __volatile("sev" ::: "memory"); + __asm __volatile( + "dsb ish \n" + "sev \n" + ::: "memory"); } } (No claim that the dsb form is optimal. Arm documentation says that they recommend every sev be preceded by a dsb and I picked ish to experiment with.) See comments and extra attachment in bugzilla 222234 (which you have fixed with your change). [Specifically your change should make non-debug kernels boot Pine64+ 2GB's again. (I had used just "dsb ish" get get such to boot.)] =3D=3D=3D Mark Millard markmi at dsl-only.net