From nobody Mon Jul 27 22:32:44 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4h8Cz11638z6lxvC for ; Mon, 27 Jul 2026 22:32:45 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "YR1" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4h8Cz06ChPz3CQZ for ; Mon, 27 Jul 2026 22:32:44 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785191564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Sd4BMd0JOA5SFI4GK5fGZ7OcjeDU40XLsGQcB6yPIEM=; b=im+UzEw78ygnXWf5r8p8F8tr6gBgw+G3lyXCEbZ5MiaoOr1a6UqTzTa3yBT4idalZ6NIJY rbRx0MCXre0n+oQD5iZ1hiYXi3yoHoKbYB7ltswsHH8GHpXr4EmS/LhzUYsexsj3zqZbyw /jvDkkMeK49AGvYKqMeLBZWCwM/8n+WrJPzRwlVY7WN9qbzcAU41IuoZ+RxAzCs8n9SS8d kr1caVeqvWx97stFo2AEUSpVnAXDHw8qapeJ5OGhAU7v7A8JnxQjl1kDjeLQ5ZZNJvRiZH 5f3pko5pcRjErAwzDY+cJGkja36nrUdYsWtVXLFKSU3PStYUWPAfTiI4fd4YEg== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1785191564; a=rsa-sha256; cv=none; b=W4SQxB4McfLswGIkYdvpRoQG1ElrmgU2EsbRqPnDUHWA4NHykErm7zIlE4EkkGpXcVmwEc BIVevM4XWFMtP+sCJqq8GSGlVRkJKN7x043S3gJNz0fvE50Zj1ivSFSQf50NCYaDR2Ilup aS+U20TDPhURivCEWNjvNI9rIJNF/v7IpBoyOecWVueJaCjEKDRekFTePlupjWmhgno4Rw OCx6mSW+TLQAynsvJn3nnObXP6KfCP9XIujpVojlN9ODmy4CnjHoatbspfgmwEqHeW7DHX 7lXyQEsoESHB1npPRjlkqRZl/9mK2ORtUKIVr4f6BWNygB/F16bihJlVweBjzg== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1785191564; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=Sd4BMd0JOA5SFI4GK5fGZ7OcjeDU40XLsGQcB6yPIEM=; b=egzKmulS1vibVzBvasN7hnD3J6Otmc+OYm/nCE5OCSZvdoevUJPcnCHbTOxR+pPeXx0TaI aDgMoU8bRHbPA8QaaO1YB7MI3SgCWugA64X6TImhw7XzFABLKyihUUlWc5/anV9C7m3P0n 91SKzwjbb9OnviYLLE5WW9CBHyYC1Xu8t8HT8NkyXuPBx10FAIIn0UMrmOHA/9iXmXMAbV eMh/ON0gtCxQFNKDCcUlJo6V+HIBoLNWZhX00i/U6qPefnHncpnLFtRQh1lyehceICnZvD jJY7FaNwNBb3uwE1lWs+DUjDV+BrfnVKgYPawEi1RjqS9HVQM0MNlTikopaPcg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4h8Cz05BdGz1QPQ for ; Mon, 27 Jul 2026 22:32:44 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 257b3 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Mon, 27 Jul 2026 22:32:44 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: e782be2e4801 - stable/14 - vm_page: Fix dequeue on arches with weak ordering List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org List-Id: List-Post: List-Help: List-Subscribe: List-Unsubscribe: List-Owner: Precedence: list MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: e782be2e48012a95b8e9bfdc247c2e8207a825e8 Auto-Submitted: auto-generated Date: Mon, 27 Jul 2026 22:32:44 +0000 Message-Id: <6a67dc8c.257b3.97b776d@gitrepo.freebsd.org> The branch stable/14 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=e782be2e48012a95b8e9bfdc247c2e8207a825e8 commit e782be2e48012a95b8e9bfdc247c2e8207a825e8 Author: Mark Johnston AuthorDate: 2026-07-17 12:57:06 +0000 Commit: Mark Johnston CommitDate: 2026-07-27 19:35:19 +0000 vm_page: Fix dequeue on arches with weak ordering A vm_page's a.queue field records the page queue index for the page queue to which the page belongs. The PGA_ENQUEUED flag indicates whether the page is actually enqueued in that queue's TAILQ. When modifying the a.queue field, you need to hold the page queue lock for the queue corresponding to the old value, unless the old value is PQ_NONE. Suppose a managed page is freed. vm_page_free_prep() calls vm_page_dequeue_deferred(), which checks whether the page belongs to a queue; if so it schedules an asynchronous dequeue operation so that page queue lock acquisitions can be batched if possible. The dequeue operation must be completed before the page's plinks.q fields are reused. So, during page allocation, we call vm_page_dequeue() to finish the dequeue operation. Similarly, since the buddy allocator uses the plinks.q fields for its own internal linkage, vm_freelist_add() calls vm_page_dequeue(). _vm_page_pqstate_commit_dequeue() is the function which actually removes the page from its queue. It sets a.queue = PG_NONE and removes the page from its queue. However, the update to the page's atomic state is relaxed, so on systems with store reordering, it may race with a concurrent enqueue of the page into the buddy queues (probably more likely) or a page queue. Fix this: use a release store to update the page's queue state in _vm_page_pqstate_commit_dequeue(), and make sure that vm_page_dequeue() uses an acquire load when comparing m->a.queue == PQ_NONE. PR: 296767 Reported and tested by: pkubaj Reviewed by: alc, kib MFC after: 1 week Differential Revision: https://reviews.freebsd.org/D58261 (cherry picked from commit d809a10218884162ed47c658233746c53a98b1aa) --- sys/vm/vm_page.c | 26 ++++++++++++++++++++++++-- sys/vm/vm_page.h | 32 ++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+), 2 deletions(-) diff --git a/sys/vm/vm_page.c b/sys/vm/vm_page.c index 284895658651..a3b929b19091 100644 --- a/sys/vm/vm_page.c +++ b/sys/vm/vm_page.c @@ -3519,6 +3519,22 @@ vm_page_pqstate_fcmpset(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new return (false); } +static __always_inline bool +vm_page_pqstate_fcmpset_rel(vm_page_t m, vm_page_astate_t *old, + vm_page_astate_t new) +{ + vm_page_astate_t tmp; + + tmp = *old; + do { + if (__predict_true(vm_page_astate_fcmpset_rel(m, old, new))) + return (true); + counter_u64_add(pqstate_commit_retries, 1); + } while (old->_bits == tmp._bits); + + return (false); +} + /* * Do the work of committing a queue state update that moves the page out of * its current queue. @@ -3549,7 +3565,8 @@ _vm_page_pqstate_commit_dequeue(struct vm_pagequeue *pq, vm_page_t m, next = TAILQ_NEXT(m, plinks.q); TAILQ_REMOVE(&pq->pq_pl, m, plinks.q); vm_pagequeue_cnt_dec(pq); - if (!vm_page_pqstate_fcmpset(m, old, new)) { + /* See vm_page_dequeue(). */ + if (!vm_page_pqstate_fcmpset_rel(m, old, new)) { if (next == NULL) TAILQ_INSERT_TAIL(&pq->pq_pl, m, plinks.q); else @@ -3853,7 +3870,12 @@ vm_page_dequeue(vm_page_t m) { vm_page_astate_t new, old; - old = vm_page_astate_load(m); + /* + * Synchronize with _vm_page_pqstate_commit_dequeue(): make sure + * that the page's queue linkage field updates are visible before + * returning. + */ + old = vm_page_astate_load_acq(m); do { if (old.queue == PQ_NONE) { KASSERT((old.flags & PGA_QUEUE_STATE_MASK) == 0, diff --git a/sys/vm/vm_page.h b/sys/vm/vm_page.h index bcbe64bb5eb9..bd979942558a 100644 --- a/sys/vm/vm_page.h +++ b/sys/vm/vm_page.h @@ -815,6 +815,18 @@ vm_page_astate_load(vm_page_t m) return (a); } +/* + * Load a snapshot of a page's 32-bit atomic state, with acquire semantics. + */ +static inline vm_page_astate_t +vm_page_astate_load_acq(vm_page_t m) +{ + vm_page_astate_t a; + + a._bits = atomic_load_acq_32(&m->a._bits); + return (a); +} + /* * Atomically compare and set a page's atomic state. */ @@ -832,6 +844,26 @@ vm_page_astate_fcmpset(vm_page_t m, vm_page_astate_t *old, vm_page_astate_t new) return (atomic_fcmpset_32(&m->a._bits, &old->_bits, new._bits) != 0); } +/* + * Atomically compare and set a page's atomic state, with release + * semantics. + */ +static inline bool +vm_page_astate_fcmpset_rel(vm_page_t m, vm_page_astate_t *old, + vm_page_astate_t new) +{ + + KASSERT(new.queue == PQ_INACTIVE || (new.flags & PGA_REQUEUE_HEAD) == 0, + ("%s: invalid head requeue request for page %p", __func__, m)); + KASSERT((new.flags & PGA_ENQUEUED) == 0 || new.queue != PQ_NONE, + ("%s: setting PGA_ENQUEUED with PQ_NONE in page %p", __func__, m)); + KASSERT(new._bits != old->_bits, + ("%s: bits are unchanged", __func__)); + + return (atomic_fcmpset_rel_32(&m->a._bits, &old->_bits, new._bits) != + 0); +} + /* * Clear the given bits in the specified page. */