From nobody Thu Jan 15 14:51:09 2026 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dsQsY1D3bz6P4J5 for ; Thu, 15 Jan 2026 14:51:13 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4dsQsW43vBz3JW8 for ; Thu, 15 Jan 2026 14:51:11 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768488671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=m3FUMCFF+5NmpsXIvt2bL9IMMw/ViZ/9cUZE0u1mnQw=; b=AdG/idOe3Lp6nVwtIOEyHY4HN51OXB/WTsVKIgnOxNh/HB2E1KtS5rAPpBxneCN0fFptzp FPPx4FaKelYVlGEuXr5WzAqciLXRg6/04E50tBXgdkpmBIlFKTFy9d2cHLCRL6H+KKHjyU MtXSeODJFc4MO8jsGbtphVZWpyEOU+jqF5CTpaNmoL/uPPxalG4QZq/qU/x4LXjWEjuEXD MMJc6NtKNGWpRwSJyKzOqfi4vC9pCOeIOKfTjGqjWKKoc1y0sS9RhNquna0COSJYCNz+de +hPjsa5MSFY+/OB63AbIC/fb5zO68d6HAK3ANAO2VVcygZVCGUEWUFvjA797cA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768488671; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=m3FUMCFF+5NmpsXIvt2bL9IMMw/ViZ/9cUZE0u1mnQw=; b=AREyUuVsoOzfYu5actDhLIx51DUwlhoJ+k4QvW8lau4XQAZVHliVvZtUVAmo3b6752be/i EY5Hv8BVpVE9IAfgDps+UA6775q9feY9UHNV6cnmI/GcuwG7g4dTIFwKZYR37mZ5UjTyVA YQQddIlnnMFpU9CRmlOu0utdDZQPny9fRgATHeVWf8cRNp5ROAI6HT/rq20eHP0zKKaOQI Ak5ZAqsM84khWGFhyhIvppMzIjZXkG/FEOwj/HJAttIi4Wb6xWCQip+fvBQcFQYnJv2vse dnk3/00w1kGbX/kpVHwKu9jLAbJmuL7ZTTQzc0I8gJogOcwR7hfXmW3xSKTB/w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1768488671; a=rsa-sha256; cv=none; b=QRwSR68ocB7sTUD4JN8rzvGnlnlTHT7BPyk1ZRQLF3XFIhUABDLcwuqd/0hQlMyUm3yDW9 zXCb45JIlMWTV8yEzwH5XWCLH4WYU2UETSVyR+rtEhIhuphZko0d47yhxLi150TEMEn+cP oSIFaJgXr/1283EpJrpWfAJ4NZCsqAeoAbuxtcU+ICdkxBc8a0KldHdDECwS0mIJCaumzY nlOFLW6px7rg7MzrMIhp5a5Uhe06rcxknQ6WpGy0NQ/G9wvpJ0hA1tsHMG2expHtXnRqD2 00L6gwJbinQhCYjfPUshqWNxUKs8qGShWBqjlUWRrIa2owL/tLI6FOekE3mtJw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4dsQsT303rzYBQ for ; Thu, 15 Jan 2026 14:51:09 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 230ae by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 15 Jan 2026 14:51:09 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: 2706f7abcac7 - stable/14 - arm64: Add a multiple TLBI workaround List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-branches@freebsd.org Sender: owner-dev-commits-src-branches@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 2706f7abcac7da9bfec2895102f0631b6f7b4b7c Auto-Submitted: auto-generated Date: Thu, 15 Jan 2026 14:51:09 +0000 Message-Id: <6968fedd.230ae.6ed7ca1d@gitrepo.freebsd.org> The branch stable/14 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=2706f7abcac7da9bfec2895102f0631b6f7b4b7c commit 2706f7abcac7da9bfec2895102f0631b6f7b4b7c Author: Andrew Turner AuthorDate: 2025-09-04 17:24:56 +0000 Commit: Andrew Turner CommitDate: 2026-01-14 21:14:15 +0000 arm64: Add a multiple TLBI workaround The Arm Cortex-A55, Cortex-A76, and Cortex-A510 CPUs have errata that require multiple TLBI, DSB instructions to workaround. Add support to pmap to implement these. As it appears that all affected TLBI calls are via pmap.c this should be sufficient. As all variants of this erratum are Category-B (rare) require the user to enable it at boot time. Reviewed by: alc Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D52190 (cherry picked from commit a884f699e4bfc1be4e721d3ec4fa93915be18a86) --- sys/arm64/arm64/pmap.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index f4b2e0d4e895..517052419ed4 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -146,6 +146,7 @@ #include #include +#include #include #include #include @@ -183,6 +184,8 @@ #define PMAP_SAN_PTE_BITS (ATTR_DEFAULT | ATTR_S1_XN | \ ATTR_S1_IDX(VM_MEMATTR_WRITE_BACK) | ATTR_S1_AP(ATTR_S1_AP_RW)) +static bool __read_mostly pmap_multiple_tlbi = false; + struct pmap_large_md_page { struct rwlock pv_lock; struct md_page pv_page; @@ -1540,6 +1543,51 @@ pmap_init_pv_table(void) } } +static cpu_feat_en +pmap_multiple_tlbi_check(const struct cpu_feat *feat __unused, u_int midr) +{ + /* + * Cortex-A55 erratum 2441007 (Cat B rare) + * Present in all revisions + */ + if (CPU_IMPL(midr) == CPU_IMPL_ARM && + CPU_PART(midr) == CPU_PART_CORTEX_A55) + return (FEAT_DEFAULT_DISABLE); + + /* + * Cortex-A76 erratum 1286807 (Cat B rare) + * Present in r0p0 - r3p0 + * Fixed in r3p1 + */ + if (midr_check_var_part_range(midr, CPU_IMPL_ARM, CPU_PART_CORTEX_A76, + 0, 0, 3, 0)) + return (FEAT_DEFAULT_DISABLE); + + /* + * Cortex-A510 erratum 2441009 (Cat B rare) + * Present in r0p0 - r1p1 + * Fixed in r1p2 + */ + if (midr_check_var_part_range(midr, CPU_IMPL_ARM, CPU_PART_CORTEX_A510, + 0, 0, 1, 1)) + return (FEAT_DEFAULT_DISABLE); + + return (FEAT_ALWAYS_DISABLE); +} + +static bool +pmap_multiple_tlbi_enable(const struct cpu_feat *feat __unused, + cpu_feat_errata errata_status, u_int *errata_list __unused, + u_int errata_count __unused) +{ + pmap_multiple_tlbi = true; + return (true); +} + +CPU_FEAT(errata_multi_tlbi, "Multiple TLBI errata", + pmap_multiple_tlbi_check, NULL, pmap_multiple_tlbi_enable, NULL, + CPU_FEAT_EARLY_BOOT | CPU_FEAT_PER_CPU); + /* * Initialize the pmap module. * @@ -1654,9 +1702,17 @@ pmap_s1_invalidate_page(pmap_t pmap, vm_offset_t va, bool final_only) r = TLBI_VA(va); if (pmap == kernel_pmap) { pmap_s1_invalidate_kernel(r, final_only); + if (pmap_multiple_tlbi) { + dsb(ish); + pmap_s1_invalidate_kernel(r, final_only); + } } else { r |= ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); pmap_s1_invalidate_user(r, final_only); + if (pmap_multiple_tlbi) { + dsb(ish); + pmap_s1_invalidate_user(r, final_only); + } } dsb(ish); isb(); @@ -1698,12 +1754,24 @@ pmap_s1_invalidate_range(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, end = TLBI_VA(eva); for (r = start; r < end; r += TLBI_VA_L3_INCR) pmap_s1_invalidate_kernel(r, final_only); + + if (pmap_multiple_tlbi) { + dsb(ish); + for (r = start; r < end; r += TLBI_VA_L3_INCR) + pmap_s1_invalidate_kernel(r, final_only); + } } else { start = end = ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); start |= TLBI_VA(sva); end |= TLBI_VA(eva); for (r = start; r < end; r += TLBI_VA_L3_INCR) pmap_s1_invalidate_user(r, final_only); + + if (pmap_multiple_tlbi) { + dsb(ish); + for (r = start; r < end; r += TLBI_VA_L3_INCR) + pmap_s1_invalidate_user(r, final_only); + } } dsb(ish); isb(); @@ -1734,6 +1802,10 @@ pmap_s1_invalidate_all_kernel(void) dsb(ishst); __asm __volatile("tlbi vmalle1is"); dsb(ish); + if (pmap_multiple_tlbi) { + __asm __volatile("tlbi vmalle1is"); + dsb(ish); + } isb(); } @@ -1751,9 +1823,17 @@ pmap_s1_invalidate_all(pmap_t pmap) dsb(ishst); if (pmap == kernel_pmap) { __asm __volatile("tlbi vmalle1is"); + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vmalle1is"); + } } else { r = ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); __asm __volatile("tlbi aside1is, %0" : : "r" (r)); + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi aside1is, %0" : : "r" (r)); + } } dsb(ish); isb();