From nobody Fri Jul 17 13:18:21 2026 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h1r7x4K21z6ltC2 for ; Fri, 17 Jul 2026 13:18:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h1r7x3lw8z3XX0 for ; Fri, 17 Jul 2026 13:18:21 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784294301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dm7/r99W7u9dmSq5VBjSFozNaytlY3NTqM1DWlnC4vQ=; b=jHs+m9Hv183cf/8NqiWM2odLNr8M+JzSaqxoAYvc83MqcGMMs8rwrAZJjj5Pao45SrpmPY yRzRL6DFHog9p0YZ0MNKFzZdu/UXLLakm0BoDZe7HhLXdSZh6azK1bX4VJgLXKcpUuS6Ch sEtzj1gs3oWKX1NCO8OiSbauP7eC1IfpYq3hzZ7CejO3kPar0FuhpqZxAzVjU58Ow5nnf1 BDBdDncQqtPzhjlqh3KyUTxA5qFzit2aRfnWv3huLHbP5UaQeqlqwlpSkKhhjcleJwr7jg xz7erFSYXs1jWGjI0iEPOHdmqUiN8H5wIRGxmXYWpQ02+ZEGfrT7P4LxN91iXg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1784294301; a=rsa-sha256; cv=none; b=GmQ4qw4UtwCO3oXsf2+UbhuL7c475dBGiwx7cCh0VDz2BCorwT8YN/3ENAi3VqjKQBZcxe +SEWvcIYUHOmwlXN7END9hjfKDeoGqS3uETrbzjtxUP/g5OZZ8DRtYxKPPdDpEsr3Dnf4G FEa0AtaMAMeCyPK/vhOEzyFMfEOFTdhmcR+mP4jzKrtrZ12ZcTA6yCrL3HCcaumzb+a2S3 8pE/al+BhGhReiNDuWyr1GJeBHrOt3uJP6qtQusa8vmWiDJRtizSqwr99Xb1Tjb80AT8Yj O2pqLXRTBMmhlbtrv6oJLzym7SEOPlMwRkt4uGVQXKMTBMa0qwnZ5u4iYHrzMA== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1784294301; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=dm7/r99W7u9dmSq5VBjSFozNaytlY3NTqM1DWlnC4vQ=; b=qYKSVnvWtiO+QD4UGcf3nCGF7aZE77yGitf6XBempDNCjN5WNCdN/JZ+Ee4WbLYk7XVRAC kTdvEMJJ8E/Yl/PBcec1VyGanLK+tN3Cl94yrY1drXHaY3MgkVtj9WiJhZssu0lh17lhXx PdppMykdgkDMTgBNstv3wi+zVYKCwxW+XD0+oUDKayWeuRd67jPBqjg3YcwGLXGR99T8BP SwCOlU/xXQ8tGt7YH+P7/PWMm4pt1WUJ+QHdhuSprX4EBnnUpYCpFN77JBZy9d03dIOMvU /tbiJVag8l2NKitDGdJdMpSbfrFJIHamqH98vFsf59i7sd7nGOAesr0VufBfHg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h1r7x20jzzpbC for ; Fri, 17 Jul 2026 13:18:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3f5bb by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Fri, 17 Jul 2026 13:18:21 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Mark Johnston Subject: git: d809a1021888 - main - vm_page: Fix dequeue on arches with weak ordering List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: d809a10218884162ed47c658233746c53a98b1aa Auto-Submitted: auto-generated Date: Fri, 17 Jul 2026 13:18:21 +0000 Message-Id: <6a5a2b9d.3f5bb.f9acead@gitrepo.freebsd.org> The branch main has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=d809a10218884162ed47c658233746c53a98b1aa commit d809a10218884162ed47c658233746c53a98b1aa Author: Mark Johnston AuthorDate: 2026-07-17 12:57:06 +0000 Commit: Mark Johnston CommitDate: 2026-07-17 13:18:15 +0000 vm_page: Fix dequeue on arches with weak ordering A vm_page's a.queue field records the page queue index for the page queue to which the page belongs. The PGA_ENQUEUED flag indicates whether the page is actually enqueued in that queue's TAILQ. When modifying the a.queue field, you need to hold the page queue lock for the queue corresponding to the old value, unless the old value is PQ_NONE. Suppose a managed page is freed. vm_page_free_prep() calls vm_page_dequeue_deferred(), which checks whether the page belongs to a queue; if so it schedules an asynchronous dequeue operation so that page queue lock acquisitions can be batched if possible. The dequeue operation must be completed before the page's plinks.q fields are reused. So, during page allocation, we call vm_page_dequeue() to finish the dequeue operation. Similarly, since the buddy allocator uses the plinks.q fields for its own internal linkage, vm_freelist_add() calls vm_page_dequeue(). _vm_page_pqstate_commit_dequeue() is the function which actually removes the page from its queue. It sets a.queue = PG_NONE and removes the page from its queue. However, the update to the page's atomic state is relaxed, so on systems with store reordering, it may race with a concurrent enqueue of the page into the buddy queues (probably more likely) or a page queue. Fix this: use a release store to update the page's queue state in _vm_page_pqstate_commit_dequeue(), and make sure that vm_page_dequeue() uses an acquire load when comparing m->a.queue == PQ_NONE. PR: 296767 Reported and tested by: pkubaj Reviewed by: alc, kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D58261 --- sys/vm/vm_page.c | 26 ++++++++++++++++++++++++-- sys/vm/vm_page.h | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index bb44cd9de486..130d084815f6 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -3685,6 +3685,22 @@ vm_page_pqstate_fcmpset(vm_page_t m, vm_page_astate_t *old, return (false); } +static __always_inline bool +vm_page_pqstate_fcmpset_rel(vm_page_t m, vm_page_astate_t *old, + vm_page_astate_t new) +{ + vm_page_astate_t tmp; + + tmp = *old; + do { + if (__predict_true(vm_page_astate_fcmpset_rel(m, old, new))) + return (true); + counter_u64_add(pqstate_commit_retries, 1); + } while (old->_bits == tmp._bits); + + return (false); +} + /* * Do the work of committing a queue state update that moves the page out of * its current queue. @@ -3715,7 +3731,8 @@ _vm_page_pqstate_commit_dequeue(struct vm_pagequeue *pq, vm_page_t m, next = TAILQ_NEXT(m, plinks.q); TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_dec(pq); - if (!vm_page_pqstate_fcmpset(m, old, new)) { + /* See vm_page_dequeue(). */ + if (!vm_page_pqstate_fcmpset_rel(m, old, new)) { if (next == NULL) TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); else @@ -4019,7 +4036,12 @@ vm_page_dequeue(vm_page_t m) { vm_page_astate_t new, old; - old = vm_page_astate_load(m); + /* + * Synchronize with _vm_page_pqstate_commit_dequeue(): make sure + * that the page's queue linkage field updates are visible before + * returning. + */ + old = vm_page_astate_load_acq(m); do { if (__predict_true(old.queue == PQ_NONE)) { KASSERT((old.flags & PGA_QUEUE_STATE_MASK) == 0, diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h index e2b6caacf323..1c92ccdaf7ea 100644 --- a/sys/vm/vm_page.h +++ b/sys/vm/vm_page.h @@ -774,6 +774,18 @@ vm_page_astate_load(vm_page_t m) return (a); } +/* + * Load a snapshot of a page's 32-bit atomic state, with acquire semantics. + */ +static inline vm_page_astate_t +vm_page_astate_load_acq(vm_page_t m) +{ + vm_page_astate_t a; + + a._bits = atomic_load_acq_32(&m->a._bits); + return (a); +} + /* * Atomically compare and set a page's atomic state. */ @@ -791,6 +803,26 @@ vm_page_astate_fcmpset(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) return (atomic_fcmpset_32(&m->a._bits, &old->_bits, new._bits) != 0); } +/* + * Atomically compare and set a page's atomic state, with release + * semantics. + */ +static inline bool +vm_page_astate_fcmpset_rel(vm_page_t m, vm_page_astate_t *old, + vm_page_astate_t new) +{ + + KASSERT(new.queue == PQ_INACTIVE || (new.flags & PGA_REQUEUE_HEAD) == 0, + ("%s: invalid head requeue request for page %p", __func__, m)); + KASSERT((new.flags & PGA_ENQUEUED) == 0 || new.queue != PQ_NONE, + ("%s: setting PGA_ENQUEUED with PQ_NONE in page %p", __func__, m)); + KASSERT(new._bits != old->_bits, + ("%s: bits are unchanged", __func__)); + + return (atomic_fcmpset_rel_32(&m->a._bits, &old->_bits, new._bits) != + 0); +} + /* * Clear the given bits in the specified page. */