From nobody Fri Jan 24 12:11:47 2025 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Yfc9v6RMqz5lKpD; Fri, 24 Jan 2025 12:11:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Yfc9v2bYCz47mv; Fri, 24 Jan 2025 12:11:47 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1737720707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MiWu0irlzU2KZ+8mtAH3///2SwovsXne73pDg8komhY=; b=xGG2ZeNpYlvTAW6OjOYqkL9QEtlY6HsS24g3/ml9AS8rqHkuEWBPgqkFZi/w7VZ/6Icqqb uBOV3jB5G9zLeJdzGej7pkcPqaYs/OgFoKuf5bqKxesq+KvDCfe3L9+1OgbjO1SWqSNJ5n Sm3jj7B/3wug4CySW5ccReH6HxKT5KZEQSKQgbuwe4blLxD9b4j92hLUduK1utSHefUzuM RJzcP5gjc3cu0zuyPArd5aq7Ajyeadbtg4VRtmWb4brjdCAovmxMOWM1E39unCJfPqKJfQ v+HV3my8oS3sBMBkcD1RKPsW01KQvADQYQXnRd1fj3kb1eVxnoBI1rnR0qJAOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1737720707; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=MiWu0irlzU2KZ+8mtAH3///2SwovsXne73pDg8komhY=; b=bwB9duKqBeEa2mLUH13zj5M/IAmj+SGAMgspW2eq7bRL5/ihqkzJgn7gPcJrtB+/tDxVIL kPOOcCKNjXTMAgHfpj2TQvw3gq3Aaw3YQH8WS66NPWKl3ez+AdzXx/SXJMOLdS2CIEITdi I71q6+GZfj4WjJCwelOZnOD1X/BFV2uN7axCiVQWoVmYG+n5U7EvVcCAeylFaVyDGUtfLC FACIW4xTTRv3/M5zfbgb6Cavt63YK3s/clFgyU8tXw1GKdJLLR9VGnAq3dIn+txQ1JzQmQ Ie1ioIYkJ3+qoSgpgl6l3aYWWwPdVQFM0WSKpakKSZeolvnUQqhUz578XBZhCQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1737720707; a=rsa-sha256; cv=none; b=yLKFPFNxm+ZqoxTt99QtqlPMPT9VnV22JgEhKtMA1i8JIgFCfdYFwgYKcA/hkaS3cuZ5lt UNjr3Vx6ivsIAO2v0jMwSaQjDbnXUhqrWEOIK5lkLc1seKBSAh0VXViJshoEklvvPMM5QH 0VvcI20KSkwp52+7z0aFsXH3EEDJDudpcfio1Kutu3vABXEZC2/yFNKnPD8hF3QWGwMkND HEvsCjpEui3+I4s8MRKIrVLN7xhxxt6xYzYJ5XMwmM28dNP1efVH1I4yFyNxnYZE7yXtB0 QQ/3s5wD6xN1u92G8CCOkNvUP9XMKiRLV6HhZ2Df7fLAlaYLpXiSZd3jkU6UZA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Yfc9v1y70z8jT; Fri, 24 Jan 2025 12:11:47 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 50OCBlb4040266; Fri, 24 Jan 2025 12:11:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 50OCBl47040263; Fri, 24 Jan 2025 12:11:47 GMT (envelope-from git) Date: Fri, 24 Jan 2025 12:11:47 GMT Message-Id: <202501241211.50OCBl47040263@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: 3041b636463d - main - arm64: Support mapping a 52-bit physical adddress List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 3041b636463d521e3c2bbea7673da6afceec34e5 Auto-Submitted: auto-generated The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=3041b636463d521e3c2bbea7673da6afceec34e5 commit 3041b636463d521e3c2bbea7673da6afceec34e5 Author: Andrew Turner AuthorDate: 2025-01-24 11:36:18 +0000 Commit: Andrew Turner CommitDate: 2025-01-24 12:09:27 +0000 arm64: Support mapping a 52-bit physical adddress When FEAT_LPA2 is enabled the physical address space increases from 48-bits to 52-bits. The top two address bits are moved to the now unused shareability field. Update the kernel to support this new larger address space. Reviewed by: alc, kib Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D46624 --- sys/arm64/arm64/pmap.c | 2 +- sys/arm64/include/armreg.h | 2 +- sys/arm64/include/hypervisor.h | 1 + sys/arm64/include/pmap.h | 2 ++ sys/arm64/include/pte.h | 52 ++++++++++++++++++++++++++++++++++++------ sys/arm64/vmm/vmm_arm64.c | 10 +++++--- 6 files changed, 57 insertions(+), 12 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 23eedff2e7cd..5c1e5bb63e4d 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -355,7 +355,7 @@ static u_int physmap_idx; static SYSCTL_NODE(_vm, OID_AUTO, pmap, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "VM/pmap parameters"); -static bool pmap_lpa_enabled __read_mostly = false; +bool pmap_lpa_enabled __read_mostly = false; pt_entry_t pmap_sh_attr __read_mostly = ATTR_SH(ATTR_SH_IS); #if PAGE_SIZE == PAGE_SIZE_4K diff --git a/sys/arm64/include/armreg.h b/sys/arm64/include/armreg.h index d586d3568bd7..2a2c8b23e0a4 100644 --- a/sys/arm64/include/armreg.h +++ b/sys/arm64/include/armreg.h @@ -2066,7 +2066,7 @@ #define PAR_NS_SHIFT 9 #define PAR_NS_MASK (0x3 << PAR_NS_SHIFT) #define PAR_PA_SHIFT 12 -#define PAR_PA_MASK 0x0000fffffffff000 +#define PAR_PA_MASK 0x000ffffffffff000 #define PAR_ATTR_SHIFT 56 #define PAR_ATTR_MASK (0xff << PAR_ATTR_SHIFT) /* When PAR_F == 1 (aborted) */ diff --git a/sys/arm64/include/hypervisor.h b/sys/arm64/include/hypervisor.h index 15fc36014626..a32e1000d911 100644 --- a/sys/arm64/include/hypervisor.h +++ b/sys/arm64/include/hypervisor.h @@ -281,6 +281,7 @@ #define VTCR_EL2_PS_42BIT (0x3UL << VTCR_EL2_PS_SHIFT) #define VTCR_EL2_PS_44BIT (0x4UL << VTCR_EL2_PS_SHIFT) #define VTCR_EL2_PS_48BIT (0x5UL << VTCR_EL2_PS_SHIFT) +#define VTCR_EL2_PS_52BIT (0x6UL << VTCR_EL2_PS_SHIFT) #define VTCR_EL2_DS_SHIFT 32 #define VTCR_EL2_DS (0x1UL << VTCR_EL2_DS_SHIFT) diff --git a/sys/arm64/include/pmap.h b/sys/arm64/include/pmap.h index d92069ee42fd..75de9e342c72 100644 --- a/sys/arm64/include/pmap.h +++ b/sys/arm64/include/pmap.h @@ -101,6 +101,8 @@ extern struct pmap kernel_pmap_store; #define kernel_pmap (&kernel_pmap_store) #define pmap_kernel() kernel_pmap +extern bool pmap_lpa_enabled; + #define PMAP_ASSERT_LOCKED(pmap) \ mtx_assert(&(pmap)->pm_mtx, MA_OWNED) #define PMAP_LOCK(pmap) mtx_lock(&(pmap)->pm_mtx) diff --git a/sys/arm64/include/pte.h b/sys/arm64/include/pte.h index 02eba21448ba..ae6a8694f6c4 100644 --- a/sys/arm64/include/pte.h +++ b/sys/arm64/include/pte.h @@ -54,13 +54,6 @@ typedef uint64_t pt_entry_t; /* page table entry */ #define ATTR_MASK_L UINT64_C(0x0000000000000fff) #define ATTR_MASK (ATTR_MASK_H | ATTR_MASK_L) -#define BASE_MASK ~ATTR_MASK -#define BASE_ADDR(x) ((x) & BASE_MASK) - -#define PTE_TO_PHYS(pte) BASE_ADDR(pte) -/* Convert a phys addr to the output address field of a PTE */ -#define PHYS_TO_PTE(pa) (pa) - /* Bits 58:55 are reserved for software */ #define ATTR_SW_UNUSED1 (1UL << 58) #define ATTR_SW_NO_PROMOTE (1UL << 57) @@ -81,13 +74,35 @@ typedef uint64_t pt_entry_t; /* page table entry */ #define ATTR_CONTIGUOUS (1UL << 52) #define ATTR_DBM (1UL << 51) #define ATTR_S1_GP (1UL << 50) + +/* + * Largest possible output address field for a level 3 page. Block + * entries will use fewer low address bits, but these are res0 so + * should be safe to include. + * + * This is also safe to use for the next-level table address for + * table entries as they encode a physical address in the same way. + */ +#if PAGE_SIZE == PAGE_SIZE_4K +#define ATTR_ADDR UINT64_C(0x0003fffffffff000) +#elif PAGE_SIZE == PAGE_SIZE_16K +#define ATTR_ADDR UINT64_C(0x0003ffffffffc000) +#else +#error Unsupported page size +#endif + #define ATTR_S1_nG (1 << 11) #define ATTR_AF (1 << 10) +/* When TCR_EL1.DS == 0 */ #define ATTR_SH(x) ((x) << 8) #define ATTR_SH_MASK ATTR_SH(3) #define ATTR_SH_NS 0 /* Non-shareable */ #define ATTR_SH_OS 2 /* Outer-shareable */ #define ATTR_SH_IS 3 /* Inner-shareable */ +/* When TCR_EL1.DS == 1 */ +#define ATTR_OA_51_50_SHIFT 8 +#define ATTR_OA_51_50_MASK (3 << ATTR_OA_51_50_SHIFT) +#define ATTR_OA_51_50_DELTA (50 - 8) /* Delta from address to pte */ #define ATTR_S1_AP_RW_BIT (1 << 7) #define ATTR_S1_AP(x) ((x) << 6) @@ -124,6 +139,29 @@ typedef uint64_t pt_entry_t; /* page table entry */ */ #define ATTR_PROMOTE (ATTR_MASK & ~(ATTR_CONTIGUOUS | ATTR_AF)) +/* Read the output address or next-level table address from a PTE */ +#define PTE_TO_PHYS(x) ({ \ + pt_entry_t _pte = (x); \ + vm_paddr_t _pa; \ + _pa = _pte & ATTR_ADDR; \ + if (pmap_lpa_enabled) \ + _pa |= (_pte & ATTR_OA_51_50_MASK) << ATTR_OA_51_50_DELTA; \ + _pa; \ +}) + +/* + * Convert a physical address to an output address or next-level + * table address in a PTE + */ +#define PHYS_TO_PTE(x) ({ \ + vm_paddr_t _pa = (x); \ + pt_entry_t _pte; \ + _pte = _pa & ATTR_ADDR; \ + if (pmap_lpa_enabled) \ + _pte |= (_pa >> ATTR_OA_51_50_DELTA) & ATTR_OA_51_50_MASK; \ + _pte; \ +}) + #if PAGE_SIZE == PAGE_SIZE_4K #define L0_SHIFT 39 #define L1_SHIFT 30 diff --git a/sys/arm64/vmm/vmm_arm64.c b/sys/arm64/vmm/vmm_arm64.c index 80d985241c69..43b2ba7802d7 100644 --- a/sys/arm64/vmm/vmm_arm64.c +++ b/sys/arm64/vmm/vmm_arm64.c @@ -381,8 +381,6 @@ vmmops_modinit(int ipinum) * shareable */ el2_regs.vtcr_el2 = VTCR_EL2_RES1; - el2_regs.vtcr_el2 |= - min(pa_range_bits << VTCR_EL2_PS_SHIFT, VTCR_EL2_PS_48BIT); el2_regs.vtcr_el2 |= VTCR_EL2_IRGN0_WBWA | VTCR_EL2_ORGN0_WBWA; el2_regs.vtcr_el2 |= VTCR_EL2_T0SZ(64 - vmm_virt_bits); el2_regs.vtcr_el2 |= vmm_vtcr_el2_sl(vmm_pmap_levels); @@ -402,8 +400,14 @@ vmmops_modinit(int ipinum) * the shareability field changes to become address bits when this * is set. */ - if ((READ_SPECIALREG(tcr_el1) & TCR_DS) != 0) + if ((READ_SPECIALREG(tcr_el1) & TCR_DS) != 0) { el2_regs.vtcr_el2 |= VTCR_EL2_DS; + el2_regs.vtcr_el2 |= + min(pa_range_bits << VTCR_EL2_PS_SHIFT, VTCR_EL2_PS_52BIT); + } else { + el2_regs.vtcr_el2 |= + min(pa_range_bits << VTCR_EL2_PS_SHIFT, VTCR_EL2_PS_48BIT); + } smp_rendezvous(NULL, arm_setup_vectors, NULL, &el2_regs);