From nobody Wed Apr 8 14:01:42 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4frPr70681z6YBGh for ; Wed, 08 Apr 2026 14:01:43 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R12" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4frPr61XD9z3gRQ for ; Wed, 08 Apr 2026 14:01:42 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1775656902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3s4AxsR59KbVCtMy6Rbs+PeuLIXj/IFI4QmIlETJMaA=; b=Dtx1zjftZDdDIRC4h7oehcago0CynHNNGaDROC0Q5i1PrGnOXL6iOZfCOGbdO1gTtAhjQt O6SbYi7ewlf1RArhIuRMQCeXiLnn02E65+ic+EiU3qnu7aUWzqNS5a4Y/47UbDTmTL/ub9 1Mn27TeFj2G9+WeKojpYvr0k7QA6Hm8rC/Dnv8UwuKrfaPsPyO1tQEqncF2Al6GlDKIrD2 BE2Oh1O9tqhmiSyX+wkhj1wvxcyEQOTE8uOrHBGrkQHDWVtgnmO4IjqplIdTP4zu5YsI1C YDRQPXV44bd+kztuWkGxI3zq8OkSeUzAWCU72s1+hEMKNmz4Kwau29myBTBwSQ== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1775656902; a=rsa-sha256; cv=none; b=pZqGBvBf6eTu1B4Uvn/llCjsNFGqcPOuMPoY1Bpif+SEznL96aaQD+QiscjpiTf2yI3Wpu J9BgGsYh/6A/oGbfA7l0FHjZuzZGWOtZesE6m+8chrJ4pT3T4B88VLb5KprNNXZ8WhFCla 7cqcQXgHsMZgunYPpWRhkFmdHad9l0fh9rvi+Z4Vik8kPLaviAT6lk4AEnCv8/7A/BzMO1 Kl4W3n3v5p61hbjN5LhF3brTqTWqCmZUXhsZ97KGXT1OuS6cIIOo7+5Wi27GNyHrSMOtGl ym4AwHKQfj+dYRPRy6/xrPrfVebJsWzx2kr9qJoH+WtScOJriSm/o0UzKmUdjQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1775656902; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=3s4AxsR59KbVCtMy6Rbs+PeuLIXj/IFI4QmIlETJMaA=; b=OK+UsFciMs4p+J6XYWCnNMaIQa7m9FNGXLhKAfRcTey+Bg+ABbvRRIBzxaElCTmZw1GLQk qbCoOhQODLbmON84NolhcyI285VXNQn26XCRDHJjG68FhiUKadyFEhKjlLSQh6NvRAyjTw nnVT3A81qQWx40DQaf5K74oAHfEQkQx+8aQaCQVGGbUARn85XB7cRlkLlcr6TRKlWpmLXR wAxjX9arr3jje8XKwGbpiE72m35lH4jRX3XMCREHsqK/zboQ8dSbdjFw6pGNCZIu3UmHcg bEcHyI0++VwKWkzKClpue2aynaE6vhqaRM6HKLTM+VjencPmeX/2o+RSEg0b/g== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4frPr614QtzDW9 for ; Wed, 08 Apr 2026 14:01:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 3b6ff by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Wed, 08 Apr 2026 14:01:42 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: cf9949fa6c00 - stable/15 - arm64: Optimise the repeated TLBI workaround List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/15 X-Git-Reftype: branch X-Git-Commit: cf9949fa6c0023f98e02e2af2c886644616a25bc Auto-Submitted: auto-generated Date: Wed, 08 Apr 2026 14:01:42 +0000 Message-Id: <69d65fc6.3b6ff.bb487aa@gitrepo.freebsd.org> The branch stable/15 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=cf9949fa6c0023f98e02e2af2c886644616a25bc commit cf9949fa6c0023f98e02e2af2c886644616a25bc Author: Andrew Turner AuthorDate: 2026-03-05 14:28:01 +0000 Commit: Andrew Turner CommitDate: 2026-04-08 13:59:54 +0000 arm64: Optimise the repeated TLBI workaround It has been reported that the overhead of repeating all TLBI instructions is too large [1]. The Software Developer Errata Notices (SDEN) for the relevant Arm CPUs have been updated so a single "tlbi vale1is, xzr" followed by "dsb ish" is sufficient to work around the issues. Replace the places we repeat TLBI instructions with the new sequence. [1] https://lore.kernel.org/linux-arm-kernel/20260218164348.2022831-1-mark.rutland@arm.com/ Reviewed by: kib Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D55646 (cherry picked from commit 80b4129bef8b908eb19fe47853cb6e45e4513d76) --- sys/arm64/arm64/pmap.c | 44 ++++++++++++++------------------------------ 1 file changed, 14 insertions(+), 30 deletions(-) diff --git a/sys/arm64/arm64/pmap.c b/sys/arm64/arm64/pmap.c index 0cfc8ae80d70..44ec9f1672fe 100644 --- a/sys/arm64/arm64/pmap.c +++ b/sys/arm64/arm64/pmap.c @@ -1926,17 +1926,13 @@ pmap_s1_invalidate_page(pmap_t pmap, vm_offset_t va, bool final_only) r = TLBI_VA(va); if (pmap == kernel_pmap) { pmap_s1_invalidate_kernel(r, final_only); - if (pmap_multiple_tlbi) { - dsb(ish); - pmap_s1_invalidate_kernel(r, final_only); - } } else { r |= ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); pmap_s1_invalidate_user(r, final_only); - if (pmap_multiple_tlbi) { - dsb(ish); - pmap_s1_invalidate_user(r, final_only); - } + } + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } dsb(ish); isb(); @@ -1978,24 +1974,16 @@ pmap_s1_invalidate_strided(pmap_t pmap, vm_offset_t sva, vm_offset_t eva, end = TLBI_VA(eva); for (r = start; r < end; r += TLBI_VA(stride)) pmap_s1_invalidate_kernel(r, final_only); - - if (pmap_multiple_tlbi) { - dsb(ish); - for (r = start; r < end; r += TLBI_VA(stride)) - pmap_s1_invalidate_kernel(r, final_only); - } } else { start = end = ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); start |= TLBI_VA(sva); end |= TLBI_VA(eva); for (r = start; r < end; r += TLBI_VA(stride)) pmap_s1_invalidate_user(r, final_only); - - if (pmap_multiple_tlbi) { - dsb(ish); - for (r = start; r < end; r += TLBI_VA(stride)) - pmap_s1_invalidate_user(r, final_only); - } + } + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } dsb(ish); isb(); @@ -2036,11 +2024,11 @@ pmap_s1_invalidate_all_kernel(void) { dsb(ishst); __asm __volatile("tlbi vmalle1is"); - dsb(ish); if (pmap_multiple_tlbi) { - __asm __volatile("tlbi vmalle1is"); dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } + dsb(ish); isb(); } @@ -2058,17 +2046,13 @@ pmap_s1_invalidate_all(pmap_t pmap) dsb(ishst); if (pmap == kernel_pmap) { __asm __volatile("tlbi vmalle1is"); - if (pmap_multiple_tlbi) { - dsb(ish); - __asm __volatile("tlbi vmalle1is"); - } } else { r = ASID_TO_OPERAND(COOKIE_TO_ASID(pmap->pm_cookie)); __asm __volatile("tlbi aside1is, %0" : : "r" (r)); - if (pmap_multiple_tlbi) { - dsb(ish); - __asm __volatile("tlbi aside1is, %0" : : "r" (r)); - } + } + if (pmap_multiple_tlbi) { + dsb(ish); + __asm __volatile("tlbi vale1is, xzr" ::: "memory"); } dsb(ish); isb();