From nobody Mon Jul 27 19:15:05 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h87Zy2SVrz6nQDK for ; Mon, 27 Jul 2026 19:15:06 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h87Zy075lz3fdW for ; Mon, 27 Jul 2026 19:15:06 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785179706; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=NpHbdFgHKhE3ib2XKN8ED0h77WMXJmiuuAwxac0xenE=; b=blGp9v+qyHvN6WVMK2YRGjmNVMBS8NPRKarnrLNAq/V54PfPK81Hu4MqmivaNXungfIdh3 tEsNQnxe+LkzfTX+KOE594uAm2Tt+B0U3xCyGrldWmFdtnF52UJHtRZVUx9/WSMIxqjQKm yJhpa4kfMF1sRN6SRAgFRvZzePL4Hkrqhm540QZwr7QgQErLjSZWKsYXR885ae3X+brSIy jfV440wov1KbJKGVsURmGJF8nDiDBDCB1TpK/ZF35npGEd/QcnYcYwstumrEOi4ColIEHN p1UA/6+9taeHuBQ4ZYJR1Q23jNy2ATYB22IF3FsdI5ygV6Ohqa4/qheLEnMKhA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1785179706; a=rsa-sha256; cv=none; b=qQTxj8OPu8AWL12CBRe3FUOYONXVi7qTmqi6tADVBpFe5P+gBU1ODrQxoZz3rLRUj2tkAI YM7lR8bTKxF++et/v/0HlAxZJ0fZMXUMTY9Ijr5iqud435k6wXBxrl7E9WkKRDGhLZlsLL 3gsiIjNpgRI47riIrM/XC9gIgBKCqHlMN9Y3LX2m5zjFBwPtil2vY54ldsh8EB7OQbsbOr 4eCd3b21CeVlCbQVtGeUOyvUhdO9iR0K2i6s56+cy0oGwfSgSMY2QAqod8mWlVWmt/AyTm 9lIkoErcpw8dKiIeupCUror9XiNG3LSGKp3qGlo+zqaw+9T9aJHaV87YtByNGg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785179706; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=NpHbdFgHKhE3ib2XKN8ED0h77WMXJmiuuAwxac0xenE=; b=qrcCY+3C+TZ5wp87vHO1b6xo7CaiQ/FCAFm2Wy15DdhDK9XN0xC06brOrs2Mm1y//MIYsU N1ZRmboV10/xSmYFJElbBsaDMTucWV/Ua5hpss2Z+opAvIrMrQxeKG7vj+IXbV9uLnAwrZ i5IY2CjwVVlh79s0ZlQM8OefJm4CKXWoivCp6qQeB3WC80nEQ814FLJ/qpPwiZN6MPPlfC /1e35Jh+jfMj+vifzWdqvOoVZgwknofy3xActaZAXBxNFj4svY9P4v2tQvwjKjPWbnClq5 tu3wsUlihqw1qAI2iyPM5gKAhbjwLLxy5gAN78D845WIp+AAxvuBned60Xh53w== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h87Zx53mcz1Klv for ; Mon, 27 Jul 2026 19:15:05 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 391ef by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 27 Jul 2026 19:15:05 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 3701716fdc45 - stable/15 - vm_page: Fix dequeue on arches with weak ordering List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 3701716fdc45c203951f44d01710da310204b629 Auto-Submitted: auto-generated Date: Mon, 27 Jul 2026 19:15:05 +0000 Message-Id: <6a67ae39.391ef.153582a0@gitrepo.freebsd.org> The branch stable/15 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=3701716fdc45c203951f44d01710da310204b629 commit 3701716fdc45c203951f44d01710da310204b629 Author: Mark Johnston AuthorDate: 2026-07-17 12:57:06 +0000 Commit: Mark Johnston CommitDate: 2026-07-27 17:34:32 +0000 vm_page: Fix dequeue on arches with weak ordering A vm_page's a.queue field records the page queue index for the page queue to which the page belongs. The PGA_ENQUEUED flag indicates whether the page is actually enqueued in that queue's TAILQ. When modifying the a.queue field, you need to hold the page queue lock for the queue corresponding to the old value, unless the old value is PQ_NONE. Suppose a managed page is freed. vm_page_free_prep() calls vm_page_dequeue_deferred(), which checks whether the page belongs to a queue; if so it schedules an asynchronous dequeue operation so that page queue lock acquisitions can be batched if possible. The dequeue operation must be completed before the page's plinks.q fields are reused. So, during page allocation, we call vm_page_dequeue() to finish the dequeue operation. Similarly, since the buddy allocator uses the plinks.q fields for its own internal linkage, vm_freelist_add() calls vm_page_dequeue(). _vm_page_pqstate_commit_dequeue() is the function which actually removes the page from its queue. It sets a.queue = PG_NONE and removes the page from its queue. However, the update to the page's atomic state is relaxed, so on systems with store reordering, it may race with a concurrent enqueue of the page into the buddy queues (probably more likely) or a page queue. Fix this: use a release store to update the page's queue state in _vm_page_pqstate_commit_dequeue(), and make sure that vm_page_dequeue() uses an acquire load when comparing m->a.queue == PQ_NONE. PR: 296767 Reported and tested by: pkubaj Reviewed by: alc, kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D58261 (cherry picked from commit d809a10218884162ed47c658233746c53a98b1aa) --- sys/vm/vm_page.c | 26 ++++++++++++++++++++++++-- sys/vm/vm_page.h | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index cd95cc620ad4..ce5748bd5f33 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -3670,6 +3670,22 @@ vm_page_pqstate_fcmpset(vm_page_t m, vm_page_astate_t *old, return (false); } +static __always_inline bool +vm_page_pqstate_fcmpset_rel(vm_page_t m, vm_page_astate_t *old, + vm_page_astate_t new) +{ + vm_page_astate_t tmp; + + tmp = *old; + do { + if (__predict_true(vm_page_astate_fcmpset_rel(m, old, new))) + return (true); + counter_u64_add(pqstate_commit_retries, 1); + } while (old->_bits == tmp._bits); + + return (false); +} + /* * Do the work of committing a queue state update that moves the page out of * its current queue. @@ -3700,7 +3716,8 @@ _vm_page_pqstate_commit_dequeue(struct vm_pagequeue *pq, vm_page_t m, next = TAILQ_NEXT(m, plinks.q); TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_dec(pq); - if (!vm_page_pqstate_fcmpset(m, old, new)) { + /* See vm_page_dequeue(). */ + if (!vm_page_pqstate_fcmpset_rel(m, old, new)) { if (next == NULL) TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); else @@ -4004,7 +4021,12 @@ vm_page_dequeue(vm_page_t m) { vm_page_astate_t new, old; - old = vm_page_astate_load(m); + /* + * Synchronize with _vm_page_pqstate_commit_dequeue(): make sure + * that the page's queue linkage field updates are visible before + * returning. + */ + old = vm_page_astate_load_acq(m); do { if (__predict_true(old.queue == PQ_NONE)) { KASSERT((old.flags & PGA_QUEUE_STATE_MASK) == 0, diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h index a091310ffd17..9913afa1992c 100644 --- a/sys/vm/vm_page.h +++ b/sys/vm/vm_page.h @@ -771,6 +771,18 @@ vm_page_astate_load(vm_page_t m) return (a); } +/* + * Load a snapshot of a page's 32-bit atomic state, with acquire semantics. + */ +static inline vm_page_astate_t +vm_page_astate_load_acq(vm_page_t m) +{ + vm_page_astate_t a; + + a._bits = atomic_load_acq_32(&m->a._bits); + return (a); +} + /* * Atomically compare and set a page's atomic state. */ @@ -788,6 +800,26 @@ vm_page_astate_fcmpset(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) return (atomic_fcmpset_32(&m->a._bits, &old->_bits, new._bits) != 0); } +/* + * Atomically compare and set a page's atomic state, with release + * semantics. + */ +static inline bool +vm_page_astate_fcmpset_rel(vm_page_t m, vm_page_astate_t *old, + vm_page_astate_t new) +{ + + KASSERT(new.queue == PQ_INACTIVE || (new.flags & PGA_REQUEUE_HEAD) == 0, + ("%s: invalid head requeue request for page %p", __func__, m)); + KASSERT((new.flags & PGA_ENQUEUED) == 0 || new.queue != PQ_NONE, + ("%s: setting PGA_ENQUEUED with PQ_NONE in page %p", __func__, m)); + KASSERT(new._bits != old->_bits, + ("%s: bits are unchanged", __func__)); + + return (atomic_fcmpset_rel_32(&m->a._bits, &old->_bits, new._bits) != + 0); +} + /* * Clear the given bits in the specified page. */