From nobody Sat Aug 9 20:07:48 2025 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4bzsQF1dL1z64HXQ; Sat, 09 Aug 2025 20:07:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R10" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4bzsQF0sZzz4MCF; Sat, 09 Aug 2025 20:07:49 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1754770069; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jlFBKgVm/YgDefxTGpUvTGGcNLvlBlvc+UCKZ4hWbRA=; b=rEZSSleeFDO22PJALfWcG5bTSJ9YkTOw6kQWaabCGYXvmOeqC3qihBRKie+qDpmacB/6Nf jIZ4xNZxPDHp3B+/1jopD3cjl8DQqcQl9aUpLMHZOL4aUt5Op4RPaivy5TLG0PssFQkRxq ei+p/pdbVKcTA6XmkDR7zAWfg2K+LbGb/DolM5s7x8hRSM0EXrrkUes/UL6sBMe5Lc9AoR B/cH8pM1jlvOZ1D92whs7VVqFxH/Z+iXWl06Z/7AWuCcEhAHR6ialLW2BfrZr6ZTWS2ZrI Cq6otkroapARvQWCWsdNeQvLFDML5lh0x++lxloo4tQL/UlGR0TKbmjhXAuk+Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1754770069; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=jlFBKgVm/YgDefxTGpUvTGGcNLvlBlvc+UCKZ4hWbRA=; b=vl+KS0gf+4wALr470sGuhRE3IjyqIk/7Q1UA1Jpctn+eO44ffsIU/MR8sY0CfiNej4nI4y qmbOuPO7bv0cvrukDMzPZiz4PiyIOtDSxorymCo1v+VLLOWIdvApEZSId3gVqfXrhA4TRM WFhmRflIfWSYw9Xd9ryHbj8W3foaWr2wUlETVS0XF7T7eo2mY9pL3Ht/WrLnppypaSgqsg TwTcEp75P4fuC1QWU5wKZpQ8/uewLwDrXIZAH327WOzRZOXAWB68sFPVmXweekKpDBqGQN OY/fqHUDknrAVR93YaY5uLoNrzTYb43vkW+Ix5siCipDPCyR3fto62KwRD+Dyw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1754770069; a=rsa-sha256; cv=none; b=AuvxNl42ul4Yv6FB2wo0vRufvVKzme2YMcWh62EN1A7irnYvwVppnkme0uedWyznnRVQV1 rqFY8rZLgRnmybLsAYZIvfB33rQb2MWjlBV3Bs/XRJ94fuxZK23CEznjU+y8dVhM55W7Id XdbJQJgqM13ZTxxH99KMdCddm4MPL8dFzdIXjwReKYjNH25B522P0N3NnMT7wjUtmxJ2EQ ZRiRuc7sI+5/oECE0qROW70XKKlYvIHvmtRpQzfDdJO7CLiw7ENkkwmmpgf5RD2jSfMzgO eJOQ9OEjLZ32tPki7wZWtPGSRMRrNbn25hUmLpTpjBkFdz3ma9doDMWACjYf2A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4bzsQF0TDszbjG; Sat, 09 Aug 2025 20:07:49 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 579K7nh8068097; Sat, 9 Aug 2025 20:07:49 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 579K7mCL068094; Sat, 9 Aug 2025 20:07:48 GMT (envelope-from git) Date: Sat, 9 Aug 2025 20:07:48 GMT Message-Id: <202508092007.579K7mCL068094@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Alan Cox Subject: git: 702224d7e082 - stable/14 - amd64 pmap: preallocate another page table page in pmap_demote_DMAP() List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: alc X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 702224d7e0824094fa3f33f7fdc5a34be626d774 Auto-Submitted: auto-generated The branch stable/14 has been updated by alc: URL: https://cgit.FreeBSD.org/src/commit/?id=702224d7e0824094fa3f33f7fdc5a34be626d774 commit 702224d7e0824094fa3f33f7fdc5a34be626d774 Author: Alan Cox AuthorDate: 2025-06-30 07:36:12 +0000 Commit: Alan Cox CommitDate: 2025-08-09 20:04:16 +0000 amd64 pmap: preallocate another page table page in pmap_demote_DMAP() To avoid a possible panic in pmap_demote_DMAP(), preallocate the page table page that may be needed by pmap_demote_pde() before acquiring the kernel pmap lock, so that we can wait until the allocation succeeds. (cherry picked from commit 88c8cba0439599c00d9d3d586a66a035004e6e84) --- sys/amd64/amd64/pmap.c | 113 +++++++++++++++++++++++++++++++++---------------- 1 file changed, 77 insertions(+), 36 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index b9681da13d3a..69d4008dda91 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -1303,6 +1303,8 @@ static int pmap_change_props_locked(vm_offset_t va, vm_size_t size, static boolean_t pmap_demote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va); static boolean_t pmap_demote_pde_locked(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, struct rwlock **lockp); +static bool pmap_demote_pde_mpte(pmap_t pmap, pd_entry_t *pde, + vm_offset_t va, struct rwlock **lockp, vm_page_t mpte); static bool pmap_demote_pdpe(pmap_t pmap, pdp_entry_t *pdpe, vm_offset_t va, vm_page_t m); static int pmap_enter_2mpage(pmap_t pmap, vm_offset_t va, vm_page_t m, @@ -6007,12 +6009,18 @@ pmap_demote_pde_abort(pmap_t pmap, vm_offset_t va, pd_entry_t *pde, static boolean_t pmap_demote_pde_locked(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, struct rwlock **lockp) +{ + return (pmap_demote_pde_mpte(pmap, pde, va, lockp, NULL)); +} + +static bool +pmap_demote_pde_mpte(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, + struct rwlock **lockp, vm_page_t mpte) { pd_entry_t newpde, oldpde; pt_entry_t *firstpte, newpte; pt_entry_t PG_A, PG_G, PG_M, PG_PKU_MASK, PG_RW, PG_V; vm_paddr_t mptepa; - vm_page_t mpte; int PG_PTE_CACHE; bool in_kernel; @@ -6025,10 +6033,10 @@ pmap_demote_pde_locked(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, PG_PKU_MASK = pmap_pku_mask_bit(pmap); PMAP_LOCK_ASSERT(pmap, MA_OWNED); - in_kernel = va >= VM_MAXUSER_ADDRESS; oldpde = *pde; KASSERT((oldpde & (PG_PS | PG_V)) == (PG_PS | PG_V), ("pmap_demote_pde: oldpde is missing PG_PS and/or PG_V")); +<<<<<<< HEAD /* * Invalidate the 2MB page mapping and return "failure" if the @@ -6042,44 +6050,64 @@ pmap_demote_pde_locked(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, } mpte = pmap_remove_pt_page(pmap, va); +======= + KASSERT((oldpde & PG_MANAGED) == 0 || lockp != NULL, + ("pmap_demote_pde: lockp for a managed mapping is NULL")); + in_kernel = va >= VM_MAXUSER_ADDRESS; +>>>>>>> 88c8cba04395 (amd64 pmap: preallocate another page table page in pmap_demote_DMAP()) if (mpte == NULL) { - KASSERT((oldpde & PG_W) == 0, - ("pmap_demote_pde: page table page for a wired mapping" - " is missing")); - - /* - * If the page table page is missing and the mapping - * is for a kernel address, the mapping must belong to - * the direct map. Page table pages are preallocated - * for every other part of the kernel address space, - * so the direct map region is the only part of the - * kernel address space that must be handled here. - */ - KASSERT(!in_kernel || (va >= DMAP_MIN_ADDRESS && - va < DMAP_MAX_ADDRESS), - ("pmap_demote_pde: No saved mpte for va %#lx", va)); - - /* - * If the 2MB page mapping belongs to the direct map - * region of the kernel's address space, then the page - * allocation request specifies the highest possible - * priority (VM_ALLOC_INTERRUPT). Otherwise, the - * priority is normal. - */ - mpte = pmap_alloc_pt_page(pmap, pmap_pde_pindex(va), - (in_kernel ? VM_ALLOC_INTERRUPT : 0) | VM_ALLOC_WIRED); - /* - * If the allocation of the new page table page fails, - * invalidate the 2MB page mapping and return "failure". + * Invalidate the 2MB page mapping and return "failure" if the + * mapping was never accessed. */ - if (mpte == NULL) { + if ((oldpde & PG_A) == 0) { + KASSERT((oldpde & PG_W) == 0, + ("pmap_demote_pde: a wired mapping is missing PG_A")); pmap_demote_pde_abort(pmap, va, pde, oldpde, lockp); return (FALSE); } - if (!in_kernel) - mpte->ref_count = NPTEPG; + mpte = pmap_remove_pt_page(pmap, va); + if (mpte == NULL) { + KASSERT((oldpde & PG_W) == 0, + ("pmap_demote_pde: page table page for a wired mapping is missing")); + + /* + * If the page table page is missing and the mapping + * is for a kernel address, the mapping must belong to + * the direct map. Page table pages are preallocated + * for every other part of the kernel address space, + * so the direct map region is the only part of the + * kernel address space that must be handled here. + */ + KASSERT(!in_kernel || (va >= DMAP_MIN_ADDRESS && + va < DMAP_MAX_ADDRESS), + ("pmap_demote_pde: No saved mpte for va %#lx", va)); + + /* + * If the 2MB page mapping belongs to the direct map + * region of the kernel's address space, then the page + * allocation request specifies the highest possible + * priority (VM_ALLOC_INTERRUPT). Otherwise, the + * priority is normal. + */ + mpte = pmap_alloc_pt_page(pmap, pmap_pde_pindex(va), + (in_kernel ? VM_ALLOC_INTERRUPT : 0) | + VM_ALLOC_WIRED); + + /* + * If the allocation of the new page table page fails, + * invalidate the 2MB page mapping and return "failure". + */ + if (mpte == NULL) { + pmap_demote_pde_abort(pmap, va, pde, oldpde, + lockp); + return (false); + } + + if (!in_kernel) + mpte->ref_count = NPTEPG; + } } mptepa = VM_PAGE_TO_PHYS(mpte); firstpte = (pt_entry_t *)PHYS_TO_DMAP(mptepa); @@ -9965,8 +9993,8 @@ pmap_demote_DMAP(vm_paddr_t base, vm_size_t len, boolean_t invalidate) { pdp_entry_t *pdpe; pd_entry_t *pde; - vm_page_t m; vm_offset_t va; + vm_page_t m, mpte; bool changed; if (len == 0) @@ -9986,6 +10014,11 @@ pmap_demote_DMAP(vm_paddr_t base, vm_size_t len, boolean_t invalidate) * x86_mr_split_dmap() function. */ m = vm_page_alloc_noobj(VM_ALLOC_WIRED | VM_ALLOC_WAITOK); + if (len < NBPDR) { + mpte = vm_page_alloc_noobj(VM_ALLOC_WIRED | + VM_ALLOC_WAITOK); + } else + mpte = NULL; PMAP_LOCK(kernel_pmap); pdpe = pmap_pdpe(kernel_pmap, va); @@ -10002,9 +10035,13 @@ pmap_demote_DMAP(vm_paddr_t base, vm_size_t len, boolean_t invalidate) if ((*pde & X86_PG_V) == 0) panic("pmap_demote_DMAP: invalid PDE"); if ((*pde & PG_PS) != 0) { - if (!pmap_demote_pde(kernel_pmap, pde, va)) + mpte->pindex = pmap_pde_pindex(va); + pmap_pt_page_count_adj(kernel_pmap, 1); + if (!pmap_demote_pde_mpte(kernel_pmap, pde, va, + NULL, mpte)) panic("pmap_demote_DMAP: PDE failed"); - changed = TRUE; + changed = true; + mpte = NULL; } } if (changed && invalidate) @@ -10014,6 +10051,10 @@ pmap_demote_DMAP(vm_paddr_t base, vm_size_t len, boolean_t invalidate) vm_page_unwire_noq(m); vm_page_free(m); } + if (mpte != NULL) { + vm_page_unwire_noq(mpte); + vm_page_free(mpte); + } } }