From owner-dev-commits-src-main@freebsd.org Tue Jul 6 02:09:55 2021
Return-Path: <owner-dev-commits-src-main@freebsd.org>
Delivered-To: dev-commits-src-main@mailman.nyi.freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1])
by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0B27466E670;
Tue, 6 Jul 2021 02:09:55 +0000 (UTC) (envelope-from git@FreeBSD.org)
Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org
[IPv6:2610:1c1:1:606c::19:3])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256
client-signature RSA-PSS (4096 bits) client-digest SHA256)
(Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK))
by mx1.freebsd.org (Postfix) with ESMTPS id 4GJmGV6j13z3H8s;
Tue, 6 Jul 2021 02:09:54 +0000 (UTC) (envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org (gitrepo.freebsd.org
[IPv6:2610:1c1:1:6068::e6a:5])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256)
(Client did not present a certificate)
by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CAF9011167;
Tue, 6 Jul 2021 02:09:54 +0000 (UTC) (envelope-from git@FreeBSD.org)
Received: from gitrepo.freebsd.org ([127.0.1.44])
by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16629sUG076821;
Tue, 6 Jul 2021 02:09:54 GMT (envelope-from git@gitrepo.freebsd.org)
Received: (from git@localhost)
by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16629sEk076820;
Tue, 6 Jul 2021 02:09:54 GMT (envelope-from git)
Date: Tue, 6 Jul 2021 02:09:54 GMT
Message-Id: <202107060209.16629sEk076820@gitrepo.freebsd.org>
To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org,
dev-commits-src-main@FreeBSD.org
From: Alan Cox <alc@FreeBSD.org>
Subject: git: e41fde3ed71c - main - On a failed fcmpset don't pointlessly
repeat tests
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Git-Committer: alc
X-Git-Repository: src
X-Git-Refname: refs/heads/main
X-Git-Reftype: branch
X-Git-Commit: e41fde3ed71c1e4fce81eac002c9f5b0926e6c49
Auto-Submitted: auto-generated
X-BeenThere: dev-commits-src-main@freebsd.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: Commit messages for the main branch of the src repository
<dev-commits-src-main.freebsd.org>
List-Unsubscribe: <https://lists.freebsd.org/mailman/options/dev-commits-src-main>,
<mailto:dev-commits-src-main-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/dev-commits-src-main/>
List-Post: <mailto:dev-commits-src-main@freebsd.org>
List-Help: <mailto:dev-commits-src-main-request@freebsd.org?subject=help>
List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/dev-commits-src-main>,
<mailto:dev-commits-src-main-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 06 Jul 2021 02:09:55 -0000
The branch main has been updated by alc:
URL: https://cgit.FreeBSD.org/src/commit/?id=e41fde3ed71c1e4fce81eac002c9f5b0926e6c49
commit e41fde3ed71c1e4fce81eac002c9f5b0926e6c49
Author: Alan Cox <alc@FreeBSD.org>
AuthorDate: 2021-07-04 05:20:42 +0000
Commit: Alan Cox <alc@FreeBSD.org>
CommitDate: 2021-07-06 02:07:40 +0000
On a failed fcmpset don't pointlessly repeat tests
In a few places, on a failed compare-and-set, both the amd64 pmap and
the arm64 pmap repeat tests on bits that won't change state while the
pmap is locked. Eliminate some of these unnecessary tests.
Reviewed by: andrew, kib, markj
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D31014
---
sys/amd64/amd64/pmap.c | 11 +++++------
sys/arm64/arm64/pmap.c | 15 ++++++++-------
2 files changed, 13 insertions(+), 13 deletions(-)
diff --git a/sys/amd64/amd64/pmap.c b/sys/amd64/amd64/pmap.c
index ea017b8a61a8..5e0b6d76ae0a 100644
--- a/sys/amd64/amd64/pmap.c
+++ b/sys/amd64/amd64/pmap.c
@@ -8459,7 +8459,7 @@ pmap_remove_write(vm_page_t m)
pvh = (m->flags & PG_FICTITIOUS) != 0 ? &pv_dummy :
pa_to_pvh(VM_PAGE_TO_PHYS(m));
rw_wlock(lock);
-retry_pv_loop:
+retry:
TAILQ_FOREACH_SAFE(pv, &pvh->pv_list, pv_next, next_pv) {
pmap = PV_PMAP(pv);
if (!PMAP_TRYLOCK(pmap)) {
@@ -8469,7 +8469,7 @@ retry_pv_loop:
rw_wlock(lock);
if (pvh_gen != pvh->pv_gen) {
PMAP_UNLOCK(pmap);
- goto retry_pv_loop;
+ goto retry;
}
}
PG_RW = pmap_rw_bit(pmap);
@@ -8493,7 +8493,7 @@ retry_pv_loop:
if (pvh_gen != pvh->pv_gen ||
md_gen != m->md.pv_gen) {
PMAP_UNLOCK(pmap);
- goto retry_pv_loop;
+ goto retry;
}
}
PG_M = pmap_modified_bit(pmap);
@@ -8503,12 +8503,11 @@ retry_pv_loop:
("pmap_remove_write: found a 2mpage in page %p's pv list",
m));
pte = pmap_pde_to_pte(pde, pv->pv_va);
-retry:
oldpte = *pte;
if (oldpte & PG_RW) {
- if (!atomic_cmpset_long(pte, oldpte, oldpte &
+ while (!atomic_fcmpset_long(pte, &oldpte, oldpte &
~(PG_RW | PG_M)))
- goto retry;
+ cpu_spinwait();
if ((oldpte & PG_M) != 0)
vm_page_dirty(m);
pmap_invalidate_page(pmap, pv->pv_va);
diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c
index 79b9d20231aa..bf476490b6be 100644
--- a/sys/arm64/arm64/pmap.c
+++ b/sys/arm64/arm64/pmap.c
@@ -3223,10 +3223,12 @@ pmap_protect_l2(pmap_t pmap, pt_entry_t *l2, vm_offset_t sva, pt_entry_t mask,
* Return if the L2 entry already has the desired access restrictions
* in place.
*/
-retry:
if ((old_l2 & mask) == nbits)
return;
+ while (!atomic_fcmpset_64(l2, &old_l2, (old_l2 & ~mask) | nbits))
+ cpu_spinwait();
+
/*
* When a dirty read/write superpage mapping is write protected,
* update the dirty field of each of the superpage's constituent 4KB
@@ -3240,9 +3242,6 @@ retry:
vm_page_dirty(mt);
}
- if (!atomic_fcmpset_64(l2, &old_l2, (old_l2 & ~mask) | nbits))
- goto retry;
-
/*
* Since a promotion must break the 4KB page mappings before making
* the 2MB page mapping, a pmap_invalidate_page() suffices.
@@ -3334,7 +3333,7 @@ pmap_protect(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, vm_prot_t prot)
for (l3p = pmap_l2_to_l3(l2, sva); sva != va_next; l3p++,
sva += L3_SIZE) {
l3 = pmap_load(l3p);
-retry:
+
/*
* Go to the next L3 entry if the current one is
* invalid or already has the desired access
@@ -3351,6 +3350,10 @@ retry:
continue;
}
+ while (!atomic_fcmpset_64(l3p, &l3, (l3 & ~mask) |
+ nbits))
+ cpu_spinwait();
+
/*
* When a dirty read/write mapping is write protected,
* update the page's dirty field.
@@ -3360,8 +3363,6 @@ retry:
pmap_pte_dirty(pmap, l3))
vm_page_dirty(PHYS_TO_VM_PAGE(l3 & ~ATTR_MASK));
- if (!atomic_fcmpset_64(l3p, &l3, (l3 & ~mask) | nbits))
- goto retry;
if (va == va_next)
va = sva;
}