From nobody Thu Apr 23 13:49:12 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4g1crm3Dpgz6Zld1 for ; Thu, 23 Apr 2026 13:49:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4g1crm2Hcfz3XBH for ; Thu, 23 Apr 2026 13:49:12 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1776952152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=BUgfjkB4J2XzLuGv2BuUagERSLhv6d4eV+aQk0g49Ko=; b=mXM56DczBMcdEOTKqm6eDwhy+1yp6VCE9UCfk8QOoh6UglJzh+MvqenloXhlm7v35JGz3b zQ3lpFAh0gKGsQwtDjjiIAJPo05ZHd3vZYKKfXKe7NNwM8413eJuWZtipAKUr5eh400jhP Ao6SYc918M34CSEFPbt+1mNU89Bjj6DHGKFX2pVXFrLDnPcTKpZISUeJw7O/ofd8TO+xO1 G2gZ9D1KlVZUVID4VjPqN/FVxupzEtPbrfM4xlN7uBGJJ8ljUL+xYuS4b19ggQwsa5K3DH AzfcnbEcfYRrqtCsYIfnEs0D93agpZ6WNvthym14Q4I5lx+wODkfUdmgjjsXrA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1776952152; a=rsa-sha256; cv=none; b=UJp1HOxZ6Dbvc4gPmNwJNt/3/9GLXKcQgaNAhkiUV2P9guQei/U/B49hdpvxs6kT/5DMMx JI0cRnEpBejhDNq0SY5/Hv8bqB7KaC78h6O+mRTNylcwQgq6IU6UJnyoQXNcq7wdX95Hrv VWvoEgFaTPfYwlzPytAuLhHCwxb1Sq8JaCFtvZfJKF5Tbk86FXQoaLwBzCR+50ubBzDP/H +lbN9zpbZ4JjhdlR5cEtX/oatgcWGXnFUHh42cJwfDSRNcPezBLyQiwFKZBhncQiodjQuQ CtuK4yBwJPI6uxzIV9USH43ITm8XqrzATXw9Ax1gN+aKbWJOAwNrU1CtdYP+dw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1776952152; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=BUgfjkB4J2XzLuGv2BuUagERSLhv6d4eV+aQk0g49Ko=; b=YrbFO1e//THSqZT1uCQZcT1IbWZ9WT8cApB9FdFKRQBj8h2zDl1yH8nAQpZtAybek4J/yY Ul1eH1hlNy8ZzylXu26l5ZIt5Rq/0Zq1xmpUSh/nkvvSZSsiboiENVV9z485FN8p1ginNL 9qY0I595oGx/fvKDecRhgH1SoIBM972nFuCIXYjBuNjxd2a6RnHYpdQwHzjya9IsOGLVvL 1Y5b04Ug5hwfn26V5hpwCueYHau1esHunZEd5z3prw96I0z2UOrUDu0yCUvJWcwFQ0iAQ0 29AjiSU+GkWGMjWLlGTs+GfzA1SHZnJMuz9IeV/lG6AF/7bZ2FXSwEV7cYpdeg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4g1crm1rqbz42t for ; Thu, 23 Apr 2026 13:49:12 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 21cfc by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 23 Apr 2026 13:49:12 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Kyle Evans Subject: git: 280cfe2264d7 - stable/15 - amd64: fix INVLPGB range invalidation List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: kevans X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: 280cfe2264d7bf2199e5a41bdcbb9acb49d059c1 Auto-Submitted: auto-generated Date: Thu, 23 Apr 2026 13:49:12 +0000 Message-Id: <69ea2358.21cfc.11b04d9f@gitrepo.freebsd.org> The branch stable/15 has been updated by kevans: URL: https://cgit.FreeBSD.org/src/commit/?id=280cfe2264d7bf2199e5a41bdcbb9acb49d059c1 commit 280cfe2264d7bf2199e5a41bdcbb9acb49d059c1 Author: Kyle Evans AuthorDate: 2026-04-20 20:18:17 +0000 Commit: Kyle Evans CommitDate: 2026-04-23 13:48:45 +0000 amd64: fix INVLPGB range invalidation AMD64 Architecture Programmer's Manual Volume 3 says the following: > ECX[15:0] contains a count of the number of sequential pages to > invalidate in addition to the original virtual address, starting from > the virtual address specified in rAX. A count of 0 invalidates a > single page. ECX[31]=0 indicates to increment the virtual address at > the 4K boundary. ECX[31]=1 indicates to increment the virtual address > at the 2M boundary. The maximum count supported is reported in > CPUID function 8000_0008h, EDX[15:0]. ECX[31] being what we call INVLPGB_2M_CNT, signaling to increment the VA by 2M. > This instruction invalidates the TLB entry or entries, regardless of > the page size (4 Kbytes, 2 Mbytes, 4 Mbytes, or 1 Gbyte). [...] Combined with this, my interpretation of the current code is: if is aligned on a PDE boundary, we'll use INVLPGB_2M_CNT to try and invalidate PDEs with a single call, but that only works if is the start of at least 2M pages. Otherwise, if or any of the subsequent PDEs isn't actually a superpage, then we would actually only invalidate the *first* page within the PDE before skipping to the next PDE, leaving the remainder of the 4K pages in between as they were. The implication would seem to be that we would need to inspect the range that we're trying to invalidate if we're planning on using INVLPGB_2M_CNT at all, so this patch just simplifies it to a series of 4K invalidations. My gut feeling is that we likely still come out on top vs. the TLB shootdown we're avoiding. This seems to explain some issues we've seen lately with fdgrowtable() and kqueue on recent Zen4/Zen5 EPYC hardware, where we'd experience corruption that we can't explain. PR: 293382 Reviewed by: alc, kib, markj (cherry picked from commit 1b8e5c02f5c07521129e06ff8ab7c660238fd75c) --- sys/amd64/amd64/mp_machdep.c | 25 ++++++------------------- 1 file changed, 6 insertions(+), 19 deletions(-) diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c index 91f725c93158..1de6fe9227c7 100644 --- a/sys/amd64/amd64/mp_machdep.c +++ b/sys/amd64/amd64/mp_machdep.c @@ -726,25 +726,12 @@ smp_masked_invlpg_range(vm_offset_t addr1, vm_offset_t addr2, pmap_t pmap, addr2 = round_page(addr2); total = atop(addr2 - addr1); for (va = addr1; total > 0;) { - if ((va & PDRMASK) != 0 || total < NPDEPG) { - cnt = atop(NBPDR - (va & PDRMASK)); - if (cnt > total) - cnt = total; - if (cnt > invlpgb_maxcnt + 1) - cnt = invlpgb_maxcnt + 1; - invlpgb(INVLPGB_GLOB | INVLPGB_VA | va, 0, - cnt - 1); - va += ptoa(cnt); - total -= cnt; - } else { - cnt = total / NPTEPG; - if (cnt > invlpgb_maxcnt + 1) - cnt = invlpgb_maxcnt + 1; - invlpgb(INVLPGB_GLOB | INVLPGB_VA | va, 0, - INVLPGB_2M_CNT | (cnt - 1)); - va += cnt << PDRSHIFT; - total -= cnt * NPTEPG; - } + cnt = MIN(total, invlpgb_maxcnt + 1); + /* 4K increments because these may not be superpages. */ + invlpgb(INVLPGB_GLOB | INVLPGB_VA | va, 0, + cnt - 1); + va += ptoa(cnt); + total -= cnt; } tlbsync(); sched_unpin();