From nobody Thu Feb 8 16:36:40 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TW2gY1cBfz5B0DJ; Thu, 8 Feb 2024 16:36:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TW2gY0yNxz4NDR; Thu, 8 Feb 2024 16:36:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707410201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3fG3HDF3yVIisj3Qj1iWWwMPeAzFoFHLdUxFvZEoer4=; b=xzXeaLE5nLTSuue4rsDTEccucxz8nwu30+ZlSu0xKBG4ml51hGihqepWW21z4ZNV1CCfYz iIFSFz/yERUs91o2ainbtW8NrWWRiz8orNfzUc837JC6dLSQujsE0Y+5ZCZxe2rliyB3wu nfrpqKOPZo8+tGjqo8+gUFIMqEAHwCgWCUQnZ9ZQdtRQ36q0ynT5mKMRdf6cTtHY4G6raY xz0XAWnyaO5NT7zkOeNednD8zkCYTtL/GC1Y7stUbJhJAzHX/vsCExNHspot1cpRxyLsJS xzAb+TP+FBzaQWhnS0CGo1tMJeGo6BRM9BBYCIu/6Dg+df3veazCI6kpN1hbgQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1707410201; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3fG3HDF3yVIisj3Qj1iWWwMPeAzFoFHLdUxFvZEoer4=; b=SweHY9+DnCowBTBfSwFwP0LuncJ4MdKblWckvvCCF64xQcXR0c9aBhy4zx9DeIJdfRp1dS QtAj3oWIF4ce7JTbr8A0ub83bKW1hw3GmXfQwrGFQYv/1rM3vtN2LyamdDxr+Qjvlw7bcz d94Ipi6uihnVCXkAKOEy0/RmKgqMkxWWwq+n7k2JbMSsqolSQoB7YxfZMbm0J/9mSifNhN vWEsaTR0KTSS1cHZW0mFZZ9DDnGPdjzQTujZeuQukqEb5B+7x0WiNVDxEZL3Zer/an/jZP rXiOT4kzLgOq1omOTApOcfBJlFJfQ8RdD+YaYk8i8ip3VE7MEE6YAMsfD+b/2g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1707410201; a=rsa-sha256; cv=none; b=Z0+GzuI8OqGyDmdyqx66598jzlWrq7wYuwzmIJ5wQ5Ij0luHSfiwcRQ/MQGbUmVeJt33m/ yMFyX/IZiCjzkRIee6eGJosdkhmS4wnEhurKfqRv6UTQRZ3b7TumhMgZw1v6WnjZ6KYyxB D539Eo9/Bk3xEId+Oid/7nEP+CfWhn0g15qP4Ac6rFvOlkiG+Pkfq0vGnDhCsZIpkpxxfj JhDDKqjSNnQLVLB23q5b9YUz4AlwBeBHDwiHz3iz4zHDcHJK71LoYL36KUwxh8NWF9PH7E 3bpKHLexmUBNHme4xpixuQrX95zcRnHNdv1RkbKq+NPwYe1rRH1oqXtVlh6GaA== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TW2gY04YFzxdY; Thu, 8 Feb 2024 16:36:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 418Gae4Y021077; Thu, 8 Feb 2024 16:36:40 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 418GaeNW021074; Thu, 8 Feb 2024 16:36:40 GMT (envelope-from git) Date: Thu, 8 Feb 2024 16:36:40 GMT Message-Id: <202402081636.418GaeNW021074@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: fe05296fc3d0 - main - arm64: Add pmap integration for KMSAN List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: fe05296fc3d0ec4079a51ecf1250fcde0f54fc5f Auto-Submitted: auto-generated The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=fe05296fc3d0ec4079a51ecf1250fcde0f54fc5f commit fe05296fc3d0ec4079a51ecf1250fcde0f54fc5f Author: Mark Johnston AuthorDate: 2024-02-08 16:11:02 +0000 Commit: Mark Johnston CommitDate: 2024-02-08 16:35:11 +0000 arm64: Add pmap integration for KMSAN - In pmap_bootstrap_san(), allocate the root PTPs for the shadow maps. (For KASAN, this is done earlier since we need to do some special bootstrapping for the kernel stack.) - Adjust ifdefs to include KMSAN. - Expand the shadow maps when pmap_growkernel() is called. MFC after: 2 weeks Sponsored by: Klara, Inc. Sponsored by: Juniper Networks, Inc. Differential Revision: https://reviews.freebsd.org/D43405 --- sys/arm64/arm64/machdep.c | 2 +- sys/arm64/arm64/pmap.c | 44 ++++++++++++++++++++++++++++++++++++-------- 2 files changed, 37 insertions(+), 9 deletions(-) diff --git a/sys/arm64/arm64/machdep.c b/sys/arm64/arm64/machdep.c index c543f0764e3b..92f9e5692be4 100644 --- a/sys/arm64/arm64/machdep.c +++ b/sys/arm64/arm64/machdep.c @@ -972,7 +972,7 @@ initarm(struct arm64_bootparams *abp) * we'll end up searching for segments that we can safely use. Those * segments also get excluded from phys_avail. */ -#if defined(KASAN) +#if defined(KASAN) || defined(KMSAN) pmap_bootstrap_san(); #endif diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 93f513832b97..d9b6425cc20e 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -114,6 +114,7 @@ #include #include #include +#include #include #include #include @@ -1321,9 +1322,9 @@ pmap_bootstrap(vm_size_t kernlen) cpu_tlb_flushID(); } -#ifdef KASAN +#if defined(KASAN) || defined(KMSAN) static void -pmap_bootstrap_allocate_kasan_l2(vm_paddr_t start_pa, vm_paddr_t end_pa, +pmap_bootstrap_allocate_san_l2(vm_paddr_t start_pa, vm_paddr_t end_pa, vm_offset_t *vap, vm_offset_t eva) { vm_paddr_t pa; @@ -1392,7 +1393,7 @@ pmap_bootstrap_san1(vm_offset_t va, int scale) if (kernstart >= plow && kernstart < phigh) phigh = kernstart; if (phigh - plow >= L2_SIZE) { - pmap_bootstrap_allocate_kasan_l2(plow, phigh, &va, eva); + pmap_bootstrap_allocate_san_l2(plow, phigh, &va, eva); if (va >= eva) break; } @@ -1403,7 +1404,7 @@ pmap_bootstrap_san1(vm_offset_t va, int scale) /* * Done. We should now have a valid shadow address mapped for all KVA * that has been mapped so far, i.e., KERNBASE to virtual_avail. Thus, - * shadow accesses by the kasan(9) runtime will succeed for this range. + * shadow accesses by the sanitizer runtime will succeed for this range. * When the kernel virtual address range is later expanded, as will * happen in vm_mem_init(), the shadow map will be grown as well. This * is handled by pmap_san_enter(). @@ -1413,7 +1414,32 @@ pmap_bootstrap_san1(vm_offset_t va, int scale) void pmap_bootstrap_san(void) { +#ifdef KASAN pmap_bootstrap_san1(KASAN_MIN_ADDRESS, KASAN_SHADOW_SCALE); +#else + static uint8_t kmsan_shad_ptp[PAGE_SIZE * 2] __aligned(PAGE_SIZE); + static uint8_t kmsan_orig_ptp[PAGE_SIZE * 2] __aligned(PAGE_SIZE); + pd_entry_t *l0, *l1; + + if (virtual_avail - VM_MIN_KERNEL_ADDRESS > L1_SIZE) + panic("initial kernel map is too large"); + + l0 = pmap_l0(kernel_pmap, KMSAN_SHAD_MIN_ADDRESS); + pmap_store(l0, L0_TABLE | PHYS_TO_PTE( + pmap_early_vtophys((vm_offset_t)kmsan_shad_ptp))); + l1 = pmap_l0_to_l1(l0, KMSAN_SHAD_MIN_ADDRESS); + pmap_store(l1, L1_TABLE | PHYS_TO_PTE( + pmap_early_vtophys((vm_offset_t)kmsan_shad_ptp + PAGE_SIZE))); + pmap_bootstrap_san1(KMSAN_SHAD_MIN_ADDRESS, 1); + + l0 = pmap_l0(kernel_pmap, KMSAN_ORIG_MIN_ADDRESS); + pmap_store(l0, L0_TABLE | PHYS_TO_PTE( + pmap_early_vtophys((vm_offset_t)kmsan_orig_ptp))); + l1 = pmap_l0_to_l1(l0, KMSAN_ORIG_MIN_ADDRESS); + pmap_store(l1, L1_TABLE | PHYS_TO_PTE( + pmap_early_vtophys((vm_offset_t)kmsan_orig_ptp + PAGE_SIZE))); + pmap_bootstrap_san1(KMSAN_ORIG_MIN_ADDRESS, 1); +#endif } #endif @@ -2717,8 +2743,10 @@ pmap_growkernel(vm_offset_t addr) addr = roundup2(addr, L2_SIZE); if (addr - 1 >= vm_map_max(kernel_map)) addr = vm_map_max(kernel_map); - if (kernel_vm_end < addr) + if (kernel_vm_end < addr) { kasan_shadow_map(kernel_vm_end, addr - kernel_vm_end); + kmsan_shadow_map(kernel_vm_end, addr - kernel_vm_end); + } while (kernel_vm_end < addr) { l0 = pmap_l0(kernel_pmap, kernel_vm_end); KASSERT(pmap_load(l0) != 0, @@ -7856,7 +7884,7 @@ pmap_pte_bti(pmap_t pmap, vm_offset_t va __diagused) return (0); } -#if defined(KASAN) +#if defined(KASAN) || defined(KMSAN) static pd_entry_t *pmap_san_early_l2; #define SAN_BOOTSTRAP_L2_SIZE (1 * L2_SIZE) @@ -7930,7 +7958,7 @@ pmap_san_enter_alloc_l2(void) Ln_ENTRIES, 0, ~0ul, L2_SIZE, 0, VM_MEMATTR_DEFAULT)); } -void __nosanitizeaddress +void __nosanitizeaddress __nosanitizememory pmap_san_enter(vm_offset_t va) { pd_entry_t *l1, *l2; @@ -7992,7 +8020,7 @@ pmap_san_enter(vm_offset_t va) PMAP_SAN_PTE_BITS | L3_PAGE); dmb(ishst); } -#endif /* KASAN */ +#endif /* KASAN || KMSAN */ /* * Track a range of the kernel's virtual address space that is contiguous