From owner-dev-commits-src-main@freebsd.org Tue Jun 29 03:23:11 2021 Return-Path: Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7930B642014; Tue, 29 Jun 2021 03:23:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GDVDH1gKGz3PTF; Tue, 29 Jun 2021 03:23:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 22F0211EF4; Tue, 29 Jun 2021 03:23:11 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 15T3NBXe048217; Tue, 29 Jun 2021 03:23:11 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 15T3NBK8048216; Tue, 29 Jun 2021 03:23:11 GMT (envelope-from git) Date: Tue, 29 Jun 2021 03:23:11 GMT Message-Id: <202106290323.15T3NBK8048216@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Alan Cox Subject: git: 26a357245f21 - main - arm64: a few simplications to pmap_remove_{all, write} MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: alc X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 26a357245f2197eea4dbbae0956d5c71ef8ba4f1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-main@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for the main branch of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jun 2021 03:23:11 -0000 The branch main has been updated by alc: URL: https://cgit.FreeBSD.org/src/commit/?id=26a357245f2197eea4dbbae0956d5c71ef8ba4f1 commit 26a357245f2197eea4dbbae0956d5c71ef8ba4f1 Author: Alan Cox AuthorDate: 2021-06-29 02:57:04 +0000 Commit: Alan Cox CommitDate: 2021-06-29 03:21:24 +0000 arm64: a few simplications to pmap_remove_{all,write} Eliminate some unnecessary unlocking and relocking when we have to retry the operation to avoid deadlock. (All of the other pmap functions that iterate over a PV list already implemented retries without these same unlocking and relocking operations.) Avoid a pointer dereference by using an existing local variable that already holds the desired value. Eliminate some unnecessary repetition of code on a failed fcmpset. Specifically, there is no point in retesting the DBM bit because it cannot change state while the pmap lock is held. Reviewed by: kib, markj MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D30931 --- sys/arm64/arm64/pmap.c | 20 +++++++------------- 1 file changed, 7 insertions(+), 13 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 76ca8eab70ff..79b9d20231aa 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -3130,8 +3130,8 @@ pmap_remove_all(vm_page_t m) SLIST_INIT(&free); lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : page_to_pvh(m); -retry: rw_wlock(lock); +retry: while ((pv = TAILQ_FIRST(&pvh->pv_list)) != NULL) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { @@ -3140,7 +3140,6 @@ retry: PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { - rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } @@ -3151,7 +3150,6 @@ retry: ("pmap_remove_all: no page table entry found")); KASSERT(lvl == 2, ("pmap_remove_all: invalid pte level %d", lvl)); - pmap_demote_l2_locked(pmap, pte, va, &lock); PMAP_UNLOCK(pmap); } @@ -3165,7 +3163,6 @@ retry: PMAP_LOCK(pmap); rw_wlock(lock); if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { - rw_wunlock(lock); PMAP_UNLOCK(pmap); goto retry; } @@ -5224,8 +5221,8 @@ pmap_remove_write(vm_page_t m) return; lock = VM_PAGE_TO_PV_LIST_LOCK(m); pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : page_to_pvh(m); -retry_pv_loop: rw_wlock(lock); +retry: TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) { pmap = PV_PMAP(pv); PMAP_ASSERT_STAGE1(pmap); @@ -5236,12 +5233,11 @@ retry_pv_loop: rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); - rw_wunlock(lock); - goto retry_pv_loop; + goto retry; } } va = pv->pv_va; - pte = pmap_pte(pmap, pv->pv_va, &lvl); + pte = pmap_pte(pmap, va, &lvl); if ((pmap_load(pte) & ATTR_SW_DBM) != 0) (void)pmap_demote_l2_locked(pmap, pte, va, &lock); KASSERT(lock == VM_PAGE_TO_PV_LIST_LOCK(m), @@ -5261,17 +5257,15 @@ retry_pv_loop: if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); - rw_wunlock(lock); - goto retry_pv_loop; + goto retry; } } pte = pmap_pte(pmap, pv->pv_va, &lvl); oldpte = pmap_load(pte); -retry: if ((oldpte & ATTR_SW_DBM) != 0) { - if (!atomic_fcmpset_long(pte, &oldpte, + while (!atomic_fcmpset_64(pte, &oldpte, (oldpte | ATTR_S1_AP_RW_BIT) & ~ATTR_SW_DBM)) - goto retry; + cpu_spinwait(); if ((oldpte & ATTR_S1_AP_RW_BIT) == ATTR_S1_AP(ATTR_S1_AP_RW)) vm_page_dirty(m);