From nobody Sun Sep 11 06:21:58 2022 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4MQKPy3MZHz4c50s; Sun, 11 Sep 2022 06:21:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4MQKPy2tGvz3LR7; Sun, 11 Sep 2022 06:21:58 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1662877318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=7IKcNWv+a8lIex81qEmTgDcxM7ZNrzu/2hNlNJJIUKg=; b=AgnfD0rOBlEBvUfDwc6c0BTUELJjyGmCm12c4FCtFsMRGjAR8p8Vvx4NlHNqTnmliG4ZqB 7040rdb3M7uHVbI32OEYRSC279dtOL9i/Vf7j4AgNBCbkVMVEBEPFyFjnfhOMHEcP7UHhU i5rUA3mLXSFcVp+Pd+oG8qmcmmTA9r+aUZIuea65HZLtMoEVtiJhsziNN1785VR1WNA5pm iJTvbButT0Ht492lKUqRqh/oTp8pFOhVM/UFfokGke3Cfl+Rzrqlz4pzzNXTXlf4EwkMMS pONWyMgqK+t7wSYua6Q9D0JSLdlHyfw+at/l78Zm3YNHJfWrMnEh50UPBxhLWw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4MQKPy1yDPzYqJ; Sun, 11 Sep 2022 06:21:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 28B6Lw1r027923; Sun, 11 Sep 2022 06:21:58 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 28B6LwjK027922; Sun, 11 Sep 2022 06:21:58 GMT (envelope-from git) Date: Sun, 11 Sep 2022 06:21:58 GMT Message-Id: <202209110621.28B6LwjK027922@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Cox Subject: git: 8d7ee2047c5e - main - pmap: don't recompute mpte during promotion List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-main@freebsd.org X-BeenThere: dev-commits-src-main@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: alc X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 8d7ee2047c5e8b4db51c682aee4161ebfd1238e5 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1662877318; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=7IKcNWv+a8lIex81qEmTgDcxM7ZNrzu/2hNlNJJIUKg=; b=LSzEBHHVw3nvicI12iizvOSMc9OAKriB1GfiuiZIGWC1zqyztsYtTxLpJCNDcBZZhmuoJw fzDov9Tj4D6/sSWaA5hc/4NOECl5SJMiQGkyf214GB56hWO1pOR8Mrowu1oDb8QEL9+Zji yDOnNIUGc5SWpudDBR1ck/sikMp3JF1pwGLquWo40oCTvnZEXQp7J7TkJSs90WUAr1hHAs 7b+tDuahJ1BqR/ZgMfIKDYLLX4rKNnXx31rQTBVbzcN6O5MRtRk1oklR5PPJTUYXQ/mf5y Ncz1GuPwlhK91SJ1Vsrd36jLUCgwlcw6y72aTaxm95vHS80eG1FBC1kRI1KifQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1662877318; a=rsa-sha256; cv=none; b=iCaR6OU1QMuYcdEsd8S8DKlqVgvqHRhCFXfQt77uMJkZpsGEtICLTvJf7/kd5eQZIT/kRV 7ViZScLz7rvlfK86sA7y+h+rVALxKSIcolyY+j+ANugEFkKDwCrGEr3pdzyDkRSrdhVNFG ewPtHB+y9YMFxVzEOpb8ocf8Daz9Tw7pZR1urdLZvLfZG4C2kpIF/Lg87INiVx+t2Wvztg FWAo54DCPYjCON0tTww3uN/DpXT5GZ8ZgBrjKJal1Az7IfX7+IQ51cM+22/JBylIBaJTT6 frIWKtJrUglBDwR7MUFbiYL8lmjQorHaUTdDzeaOm5fk65z1aw0Ye3nGzzFM5w== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by alc: URL: https://cgit.FreeBSD.org/src/commit/?id=8d7ee2047c5e8b4db51c682aee4161ebfd1238e5 commit 8d7ee2047c5e8b4db51c682aee4161ebfd1238e5 Author: Alan Cox AuthorDate: 2022-09-09 23:34:58 +0000 Commit: Alan Cox CommitDate: 2022-09-11 06:19:22 +0000 pmap: don't recompute mpte during promotion When attempting to promote 4KB user-space mappings to a 2MB user-space mapping, the address of the struct vm_page representing the page table page that contains the 4KB mappings is already known to the caller. Pass that address to the promotion function rather than making the promotion function recompute it, which on arm64 entails iteration over the vm_phys_segs array by PHYS_TO_VM_PAGE(). And, while I'm here, eliminate unnecessary arithmetic from the calculation of the first PTE's address on arm64. MFC after: 1 week --- sys/amd64/amd64/pmap.c | 12 ++++++------ sys/arm64/arm64/pmap.c | 14 ++++++-------- 2 files changed, 12 insertions(+), 14 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 326103a1affb..e3f281784893 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -1277,7 +1277,7 @@ static vm_page_t pmap_large_map_getptp_unlocked(void); static vm_paddr_t pmap_large_map_kextract(vm_offset_t va); #if VM_NRESERVLEVEL > 0 static void pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, - struct rwlock **lockp); + vm_page_t mpte, struct rwlock **lockp); #endif static boolean_t pmap_protect_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t sva, vm_prot_t prot); @@ -6737,13 +6737,12 @@ pmap_pde_ept_executable(pmap_t pmap, pd_entry_t pde) * identical characteristics. */ static void -pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, +pmap_promote_pde(pmap_t pmap, pd_entry_t *pde, vm_offset_t va, vm_page_t mpte, struct rwlock **lockp) { pd_entry_t newpde; pt_entry_t *firstpte, oldpte, pa, *pte; pt_entry_t PG_G, PG_A, PG_M, PG_RW, PG_V, PG_PKU_MASK; - vm_page_t mpte; int PG_PTE_CACHE; PG_A = pmap_accessed_bit(pmap); @@ -6823,7 +6822,8 @@ setpte: * mapping the superpage is demoted by pmap_demote_pde() or * destroyed by pmap_remove_pde(). */ - mpte = PHYS_TO_VM_PAGE(*pde & PG_FRAME); + if (mpte == NULL) + mpte = PHYS_TO_VM_PAGE(*pde & PG_FRAME); KASSERT(mpte >= vm_page_array && mpte < &vm_page_array[vm_page_array_size], ("pmap_promote_pde: page table page is out of range")); @@ -7237,7 +7237,7 @@ unchanged: pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) - pmap_promote_pde(pmap, pde, va, &lock); + pmap_promote_pde(pmap, pde, va, mpte, &lock); #endif rv = KERN_SUCCESS; @@ -10183,7 +10183,7 @@ pmap_emulate_accessed_dirty(pmap_t pmap, vm_offset_t va, int ftype) pmap_ps_enabled(pmap) && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) { - pmap_promote_pde(pmap, pde, va, &lock); + pmap_promote_pde(pmap, pde, va, mpte, &lock); #ifdef INVARIANTS atomic_add_long(&ad_emulation_superpage_promotions, 1); #endif diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index deea00bc5d13..c86e9f562729 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -3787,18 +3787,15 @@ pmap_pv_promote_l2(pmap_t pmap, vm_offset_t va, vm_paddr_t pa, * identical characteristics. */ static void -pmap_promote_l2(pmap_t pmap, pd_entry_t *l2, vm_offset_t va, +pmap_promote_l2(pmap_t pmap, pd_entry_t *l2, vm_offset_t va, vm_page_t mpte, struct rwlock **lockp) { pt_entry_t *firstl3, *l3, newl2, oldl3, pa; - vm_page_t mpte; - vm_offset_t sva; PMAP_LOCK_ASSERT(pmap, MA_OWNED); PMAP_ASSERT_STAGE1(pmap); - sva = va & ~L2_OFFSET; - firstl3 = pmap_l2_to_l3(l2, sva); + firstl3 = (pt_entry_t *)PHYS_TO_DMAP(pmap_load(l2) & ~ATTR_MASK); newl2 = pmap_load(firstl3); if (((newl2 & (~ATTR_MASK | ATTR_AF)) & L2_OFFSET) != ATTR_AF || @@ -3851,7 +3848,8 @@ setl3: * mapping the superpage is demoted by pmap_demote_l2() or * destroyed by pmap_remove_l3(). */ - mpte = PHYS_TO_VM_PAGE(pmap_load(l2) & ~ATTR_MASK); + if (mpte == NULL) + mpte = PHYS_TO_VM_PAGE(pmap_load(l2) & ~ATTR_MASK); KASSERT(mpte >= vm_page_array && mpte < &vm_page_array[vm_page_array_size], ("pmap_promote_l2: page table page is out of range")); @@ -3871,7 +3869,7 @@ setl3: newl2 &= ~ATTR_DESCR_MASK; newl2 |= L2_BLOCK; - pmap_update_entry(pmap, l2, newl2, sva, L2_SIZE); + pmap_update_entry(pmap, l2, newl2, va & ~L2_OFFSET, L2_SIZE); atomic_add_long(&pmap_l2_promotions, 1); CTR2(KTR_PMAP, "pmap_promote_l2: success for va %#lx in pmap %p", va, @@ -4295,7 +4293,7 @@ validate: pmap_ps_enabled(pmap) && pmap->pm_stage == PM_STAGE1 && (m->flags & PG_FICTITIOUS) == 0 && vm_reserv_level_iffullpop(m) == 0) { - pmap_promote_l2(pmap, pde, va, &lock); + pmap_promote_l2(pmap, pde, va, mpte, &lock); } #endif