From owner-svn-src-all@freebsd.org  Mon Aug 21 18:12:33 2017
Return-Path: <owner-svn-src-all@freebsd.org>
Delivered-To: svn-src-all@mailman.ysv.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 by mailman.ysv.freebsd.org (Postfix) with ESMTP id 8CC35DEC267;
 Mon, 21 Aug 2017 18:12:33 +0000 (UTC)
 (envelope-from andrew@FreeBSD.org)
Received: from repo.freebsd.org (repo.freebsd.org
 [IPv6:2610:1c1:1:6068::e6a:0])
 (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id 5BE7B7EB5B;
 Mon, 21 Aug 2017 18:12:33 +0000 (UTC)
 (envelope-from andrew@FreeBSD.org)
Received: from repo.freebsd.org ([127.0.1.37])
 by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v7LICW8t039996;
 Mon, 21 Aug 2017 18:12:32 GMT (envelope-from andrew@FreeBSD.org)
Received: (from andrew@localhost)
 by repo.freebsd.org (8.15.2/8.15.2/Submit) id v7LICWEO039995;
 Mon, 21 Aug 2017 18:12:32 GMT (envelope-from andrew@FreeBSD.org)
Message-Id: <201708211812.v7LICWEO039995@repo.freebsd.org>
X-Authentication-Warning: repo.freebsd.org: andrew set sender to
 andrew@FreeBSD.org using -f
From: Andrew Turner <andrew@FreeBSD.org>
Date: Mon, 21 Aug 2017 18:12:32 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
 svn-src-head@freebsd.org
Subject: svn commit: r322769 - head/sys/arm64/arm64
X-SVN-Group: head
X-SVN-Commit-Author: andrew
X-SVN-Commit-Paths: head/sys/arm64/arm64
X-SVN-Commit-Revision: 322769
X-SVN-Commit-Repository: base
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.23
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
 user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all/>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/svn-src-all>,
 <mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Mon, 21 Aug 2017 18:12:33 -0000

Author: andrew
Date: Mon Aug 21 18:12:32 2017
New Revision: 322769
URL: https://svnweb.freebsd.org/changeset/base/322769

Log:
  Improve the performance of the arm64 thread switching code.
  
  The full system memory barrier around a TLB invalidation is stricter than
  required. It needs to wait on accesses to main memory, with just the weaker
  store variant before the invalidate. As such use the dsb istst, tlbi, dlb
  ish sequence already used in pmap.
  
  The tlbi instruction in this sequence is also unnecessarily using a
  broadcast invalidate when it just needs to invalidate the local CPUs TLB.
  Switch to a non-broadcast variant of this instruction.
  
  Sponsored by:	DARPA, AFRL

Modified:
  head/sys/arm64/arm64/swtch.S

Modified: head/sys/arm64/arm64/swtch.S
==============================================================================
--- head/sys/arm64/arm64/swtch.S	Mon Aug 21 18:00:26 2017	(r322768)
+++ head/sys/arm64/arm64/swtch.S	Mon Aug 21 18:12:32 2017	(r322769)
@@ -91,9 +91,9 @@ ENTRY(cpu_throw)
 	isb
 
 	/* Invalidate the TLB */
-	dsb	sy
-	tlbi	vmalle1is
-	dsb	sy
+	dsb	ishst
+	tlbi	vmalle1
+	dsb	ish
 	isb
 
 	/* If we are single stepping, enable it */
@@ -192,9 +192,9 @@ ENTRY(cpu_switch)
 	isb
 
 	/* Invalidate the TLB */
-	dsb	sy
-	tlbi	vmalle1is
-	dsb	sy
+	dsb	ishst
+	tlbi	vmalle1
+	dsb	ish
 	isb
 
 	/*