From nobody Wed May 25 10:38:47 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 1D5F71B4B6E6; Wed, 25 May 2022 10:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4L7SGc0JPfz4klb; Wed, 25 May 2022 10:38:48 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653475128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1t/jlh7KNiix7KrnY81OHSdxTYXmo1/YH/jXCG0hkls=; b=erN/QPx3vHZJoAB2AaR1iVOwGDh3eikI+/IJpYj7YT436dk9E+USdKiEv1XvKayevtytk8 u0k3sG2fpJi/FUZcfvEI8w5cwzHI/L+e8z56RwERx9rLz2kgYwzmECHcnuVna74zZmOJl3 C6ZZumOzTK9QvpFq/AjF3oqwdmARX+bWIkGII81/FpqeE7YkiliDyW8jDOzI90Dvi3LWF4 kI+UalQ8tdhGpMWSXBtVce0FQQLhu+fmMjMtOFHgA6WryuHYzPvp6zzOqG+IExHfKXdvyk YbFkendAVLn2GEzBo966pYml31CaUOywS+Y2sVunNXQKlkQ3DseWkfYdZ4rPZQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DB60617C91; Wed, 25 May 2022 10:38:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 24PAclwp042334; Wed, 25 May 2022 10:38:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 24PAcljj042333; Wed, 25 May 2022 10:38:47 GMT (envelope-from git) Date: Wed, 25 May 2022 10:38:47 GMT Message-Id: <202205251038.24PAcljj042333@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: 9e0716f46101 - main - Add arm64 CnP support List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 9e0716f4610158936c0b9cec1a9930c4ebe924fb Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1653475128; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=1t/jlh7KNiix7KrnY81OHSdxTYXmo1/YH/jXCG0hkls=; b=UN7hrhMgSrrD70u9/nK4zz/cWWAAOGt4ma6MaK708XdTR3WJ6sas6RxQQsPOpxsbXYKNL9 tcLjBpvCDvfh2qGw4i7XT/7U1KFGSFaBmfedzeMxUepe532iQm3ol++xUBvY+T5YnYunY1 eW7GVz+13U6p2gpIDk0HddiV+5UULb1WX00HSaD/0pt/RoPAAg/yhHLTOnM+G3/0tRI3V+ s+B32vGX4yZg9d+SIyuii4m5pfkxkETEph4ViM1Ucl/xIq6nirdvuCF0v3abUr10X6EqRB 1uzCbW1njnDM02YZHbZH20jLQrOTJy4vXnHCshH7RbPs2Gl8nVV3R2Ef5i1+ow== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1653475128; a=rsa-sha256; cv=none; b=Qvp5p4udRzGCDc87ebqCBYHtNV2QvtxJMbK9F3uuOxv14kFWkrSM/o9fO9yQuX8aCl0bOj 10Apk0827qWcsUEKG9D2dCv5xG06U0MvAkIxfQ16JpWAvLrK15EEYFn8A0o7d5F0hNjxqr Qw6OLatIqI/CXoINR+Bi0Hap2aSuN25tjqdpgTdQVfpn0wKFhYlsNn/pf4oYZC6bIQgm5d Hg6GzaeQRgnyE5aD37kl7UEB/8fAZbpQb6t6Nt6kasxJkvBvplNwNUhJzXFzrPcl9cUBNk eUbF6mqjLzz3PCu8AavpaufWH0yZCWaUTMo0/Oqyop81RN3oJwtfv+RatLW1iw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=9e0716f4610158936c0b9cec1a9930c4ebe924fb commit 9e0716f4610158936c0b9cec1a9930c4ebe924fb Author: Andrew Turner AuthorDate: 2022-05-25 09:55:40 +0000 Commit: Andrew Turner CommitDate: 2022-05-25 09:56:13 +0000 Add arm64 CnP support Set the Common not Private bit in the ttbr registers when supported on arm64. This tells the hardware it can share the translation table entries on multiple CPUs. Reviewed by: alc, kib Sponsored by: The FreeBSD Foundation --- sys/arm64/arm64/pmap.c | 66 ++++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 64 insertions(+), 2 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 3a356e8b720b..3b37ce214664 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -6724,6 +6724,8 @@ out: mtx_unlock_spin(&set->asid_set_mutex); } +static uint64_t __read_mostly ttbr_flags; + /* * Compute the value that should be stored in ttbr0 to activate the specified * pmap. This value may change from time to time. @@ -6731,10 +6733,70 @@ out: uint64_t pmap_to_ttbr0(pmap_t pmap) { + uint64_t ttbr; + + ttbr = pmap->pm_ttbr; + ttbr |= ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); + ttbr |= ttbr_flags; + + return (ttbr); +} + +static void +pmap_set_cnp(void *arg) +{ + uint64_t ttbr0, ttbr1; + u_int cpuid; + + cpuid = *(u_int *)arg; + if (cpuid == curcpu) { + /* + * Set the flags while all CPUs are handling the + * smp_rendezvous so will not call pmap_to_ttbr0. Any calls + * to pmap_to_ttbr0 after this will have the CnP flag set. + * The dsb after invalidating the TLB will act as a barrier + * to ensure all CPUs can observe this change. + */ + ttbr_flags |= TTBR_CnP; + } + + ttbr0 = READ_SPECIALREG(ttbr0_el1); + ttbr0 |= TTBR_CnP; + + ttbr1 = READ_SPECIALREG(ttbr1_el1); + ttbr1 |= TTBR_CnP; + + /* Update ttbr{0,1}_el1 with the CnP flag */ + WRITE_SPECIALREG(ttbr0_el1, ttbr0); + WRITE_SPECIALREG(ttbr1_el1, ttbr1); + isb(); + __asm __volatile("tlbi vmalle1is"); + dsb(ish); + isb(); +} + +/* + * Defer enabling CnP until we have read the ID registers to know if it's + * supported on all CPUs. + */ +static void +pmap_init_cnp(void *dummy __unused) +{ + uint64_t reg; + u_int cpuid; + + if (!get_kernel_reg(ID_AA64MMFR2_EL1, ®)) + return; + + if (ID_AA64MMFR2_CnP_VAL(reg) != ID_AA64MMFR2_CnP_NONE) { + if (bootverbose) + printf("Enabling CnP\n"); + cpuid = curcpu; + smp_rendezvous(NULL, pmap_set_cnp, NULL, &cpuid); + } - return (ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)) | - pmap->pm_ttbr); } +SYSINIT(pmap_init_cnp, SI_SUB_SMP, SI_ORDER_ANY, pmap_init_cnp, NULL); static bool pmap_activate_int(pmap_t pmap)