From owner-dev-commits-src-branches@freebsd.org Tue Aug 31 19:09:51 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 99D16673A10; Tue, 31 Aug 2021 19:09:51 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4GzcFV5QkPz4sql; Tue, 31 Aug 2021 19:09:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 804461740E; Tue, 31 Aug 2021 19:09:50 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 17VJ9oZc074296; Tue, 31 Aug 2021 19:09:50 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 17VJ9oIJ074295; Tue, 31 Aug 2021 19:09:50 GMT (envelope-from git) Date: Tue, 31 Aug 2021 19:09:50 GMT Message-Id: <202108311909.17VJ9oIJ074295@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 380c653c7d4b - stable/13 - On a failed fcmpset don't pointlessly repeat tests MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 380c653c7d4b1b54cc0ea5f7f80a8ec8c71671f1 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Aug 2021 19:09:51 -0000 The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=380c653c7d4b1b54cc0ea5f7f80a8ec8c71671f1 commit 380c653c7d4b1b54cc0ea5f7f80a8ec8c71671f1 Author: Alan Cox AuthorDate: 2021-07-04 05:20:42 +0000 Commit: Mark Johnston CommitDate: 2021-08-31 19:09:24 +0000 On a failed fcmpset don't pointlessly repeat tests In a few places, on a failed compare-and-set, both the amd64 pmap and the arm64 pmap repeat tests on bits that won't change state while the pmap is locked. Eliminate some of these unnecessary tests. Reviewed by: andrew, kib, markj (cherry picked from commit e41fde3ed71c1e4fce81eac002c9f5b0926e6c49) --- sys/amd64/amd64/pmap.c | 11 +++++------ sys/arm64/arm64/pmap.c | 15 ++++++++------- 2 files changed, 13 insertions(+), 13 deletions(-) diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c index 16dcddc7786a..9122a368823b 100644 --- a/sys/amd64/amd64/pmap.c +++ b/sys/amd64/amd64/pmap.c @@ -8347,7 +8347,7 @@ pmap_remove_write(vm_page_t m) pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy : pa_to_pvh(VM_PAGE_TO_PHYS(m)); rw_wlock(lock); -retry_pv_loop: +retry: TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) { pmap = PV_PMAP(pv); if (!PMAP_TRYLOCK(pmap)) { @@ -8357,7 +8357,7 @@ retry_pv_loop: rw_wlock(lock); if (pvh_gen != pvh->pv_gen) { PMAP_UNLOCK(pmap); - goto retry_pv_loop; + goto retry; } } PG_RW = pmap_rw_bit(pmap); @@ -8381,7 +8381,7 @@ retry_pv_loop: if (pvh_gen != pvh->pv_gen || md_gen != m->md.pv_gen) { PMAP_UNLOCK(pmap); - goto retry_pv_loop; + goto retry; } } PG_M = pmap_modified_bit(pmap); @@ -8391,12 +8391,11 @@ retry_pv_loop: ("pmap_remove_write: found a 2mpage in page %p's pv list", m)); pte = pmap_pde_to_pte(pde, pv->pv_va); -retry: oldpte = *pte; if (oldpte & PG_RW) { - if (!atomic_cmpset_long(pte, oldpte, oldpte & + while (!atomic_fcmpset_long(pte, &oldpte, oldpte & ~(PG_RW | PG_M))) - goto retry; + cpu_spinwait(); if ((oldpte & PG_M) != 0) vm_page_dirty(m); pmap_invalidate_page(pmap, pv->pv_va); diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 01156f2c1ccd..35cbfc4c308c 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -3244,10 +3244,12 @@ pmap_protect_l2(pmap_t pmap, pt_entry_t *l2, vm_offset_t sva, pt_entry_t mask, * Return if the L2 entry already has the desired access restrictions * in place. */ -retry: if ((old_l2 & mask) == nbits) return; + while (!atomic_fcmpset_64(l2, &old_l2, (old_l2 & ~mask) | nbits)) + cpu_spinwait(); + /* * When a dirty read/write superpage mapping is write protected, * update the dirty field of each of the superpage's constituent 4KB @@ -3261,9 +3263,6 @@ retry: vm_page_dirty(mt); } - if (!atomic_fcmpset_64(l2, &old_l2, (old_l2 & ~mask) | nbits)) - goto retry; - /* * Since a promotion must break the 4KB page mappings before making * the 2MB page mapping, a pmap_invalidate_page() suffices. @@ -3355,7 +3354,7 @@ pmap_protect(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, vm_prot_t prot) for (l3p = pmap_l2_to_l3(l2, sva); sva != va_next; l3p++, sva += L3_SIZE) { l3 = pmap_load(l3p); -retry: + /* * Go to the next L3 entry if the current one is * invalid or already has the desired access @@ -3372,6 +3371,10 @@ retry: continue; } + while (!atomic_fcmpset_64(l3p, &l3, (l3 & ~mask) | + nbits)) + cpu_spinwait(); + /* * When a dirty read/write mapping is write protected, * update the page's dirty field. @@ -3381,8 +3384,6 @@ retry: pmap_pte_dirty(pmap, l3)) vm_page_dirty(PHYS_TO_VM_PAGE(l3 & ~ATTR_MASK)); - if (!atomic_fcmpset_64(l3p, &l3, (l3 & ~mask) | nbits)) - goto retry; if (va == va_next) va = sva; }