From owner-dev-commits-src-branches@freebsd.org Tue Aug 31 19:09:48 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E64F6673A0F; Tue, 31 Aug 2021 19:09:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GzcFS2vS7z4sl6; Tue, 31 Aug 2021 19:09:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 0F0941748B; Tue, 31 Aug 2021 19:09:48 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 17VJ9lo4074248; Tue, 31 Aug 2021 19:09:47 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 17VJ9lPl074247; Tue, 31 Aug 2021 19:09:47 GMT (envelope-from git) Date: Tue, 31 Aug 2021 19:09:47 GMT Message-Id: <202108311909.17VJ9lPl074247@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: b2727ab88c8d - stable/13 - arm64: a few simplications to pmap_remove_{all, write} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: b2727ab88c8d2b2974124c3c2929b16e20949200 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2021 19:09:49 -0000 The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=b2727ab88c8d2b2974124c3c2929b16e20949200 commit b2727ab88c8d2b2974124c3c2929b16e20949200 Author: Alan Cox AuthorDate: 2021-06-29 02:57:04 +0000 Commit: Mark Johnston CommitDate: 2021-08-31 19:09:23 +0000 arm64: a few simplications to pmap_remove_{all,write} Eliminate some unnecessary unlocking and relocking when we have to retry the operation to avoid deadlock. (All of the other pmap functions that iterate over a PV list already implemented retries without these same unlocking and relocking operations.) Avoid a pointer dereference by using an existing local variable that already holds the desired value. Eliminate some unnecessary repetition of code on a failed fcmpset. Specifically, there is no point in retesting the DBM bit because it cannot change state while the pmap lock is held. Reviewed by: kib, markj (cherry picked from commit 26a357245f2197eea4dbbae0956d5c71ef8ba4f1) --- sys/arm64/arm64/pmap.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 4a5dff98f308..01156f2c1ccd 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -3151,8 +3151,8 @@ pmap_remove_all(vm_page_t m) SLIST_INIT(&free); lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : page_to_pvh(m); -retry: rw_wlock(lock); +retry: while ((pv = TAILQ_FIRST(&pvh->pv_list)) != NULL) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { @@ -3161,7 +3161,6 @@ retry: PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { - rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } @@ -3172,7 +3171,6 @@ retry: ("pmap_remove_all: no page table entry found")); KASSERT(lvl == 2, ("pmap_remove_all: invalid pte level %d", lvl)); - pmap_demote_l2_locked(pmap, pte, va, &lock); PMAP_UNLOCK(pmap); } @@ -3186,7 +3184,6 @@ retry: PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { - rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } @@ -5433,8 +5430,8 @@ pmap_remove_write(vm_page_t m) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : page_to_pvh(m); -retry_pv_loop: rw_wlock(lock); +retry: TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) { pmap = PV_PMAP(pv); PMAP_ASSERT_STAGE1(pmap); @@ -5445,12 +5442,11 @@ retry_pv_loop: rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); - rw_wunlock(lock); - goto retry_pv_loop; + goto retry; } } va = pv->pv_va; - pte = pmap_pte(pmap, pv->pv_va, &lvl); + pte = pmap_pte(pmap, va, &lvl); if ((pmap_load(pte) & ATTR_SW_DBM) != 0) (void)pmap_demote_l2_locked(pmap, pte, va, &lock); KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), @@ -5470,17 +5466,15 @@ retry_pv_loop: if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); - rw_wunlock(lock); - goto retry_pv_loop; + goto retry; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); oldpte = pmap_load(pte); -retry: if ((oldpte & ATTR_SW_DBM) != 0) { - if (!atomic_fcmpset_long(pte, &oldpte, + while (!atomic_fcmpset_64(pte, &oldpte, (oldpte | ATTR_S1_AP_RW_BIT) & ~ATTR_SW_DBM)) - goto retry; + cpu_spinwait(); if ((oldpte & ATTR_S1_AP_RW_BIT) == ATTR_S1_AP(ATTR_S1_AP_RW)) vm_page_dirty(m);