From nobody Thu Jan 15 14:50:58 2026 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4dsQsG5FvVz6P49f for ; Thu, 15 Jan 2026 14:50:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R13" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4dsQsG3QyZz3JJp for ; Thu, 15 Jan 2026 14:50:58 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768488658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=a2pRT9W6m2g6255WHVtkbL0I3jKU091oH0aNpLIzSh0=; b=iVD/+RVE+m++025vdMlM72YqtzPqMiyabra6795Ok5W6tGdTgQuA41oy1RQDoNZ556uHwJ pKv4jC5aeKjSQlk/wv0pzaPNERDYYPXCH8NRpXMFZaOibvrqpdd6x05bnyB0M2hHbduLJP AD9yoWLdk9pYh7eupIaSY5gtw16p5XU4EdWPJXHUXYzuwTRR/+sniXdkCGT0DYRok2Xkwt LcrFnPUCILJIbulhMKAkb0JGHmosZUjrVSCUku4HmukCqbNcXYlBB/ZcpMbyAu+tKy1iyH aoz59PeFACXEk1jR8gbT2ZLF2DilJgo+Gxfi7uxtNKp3STXTS59V4faAE9t9+w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1768488658; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=a2pRT9W6m2g6255WHVtkbL0I3jKU091oH0aNpLIzSh0=; b=b1wo64xVOPzDmuvozV2p1c3VDf9DbjKy//SPoGCL5dNJ/k3Q8bkyApydWwrfKM0jj/tl6r wAiZBw221w8iqN3PO0sEJFU7pqDt8s5T3LasrRJg62e+2PJjSVVox6NeuXssuZXAfD02Mg iwztIPVE6blW2dO7jD8hXxvUVL006833xxKkoh8ClgAUnMSb0ITwwwzQz0J9b2qIs0M9sd c+XcDVbChIkLMyflQanLcS63OOrnBSsAgqLkNlUEgl1qSwEq0SAXxkiwPL15jvPYVA3wQv q3ZGgMlgHT9foeQNzZ+FZgEdZOg6qM2PDRhQdskMNVvuG6G09SBd4+rtZ8Tj9Q== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1768488658; a=rsa-sha256; cv=none; b=pRrIcGecgfYuUJf5rmUz0qksBwGTlsxt2lod3WiKxV2GAwKU63i6uhM1owY8r6YST0bbaB j1Q0AGjxkNcySmzXN8xamHaxmty4u0nR1RYLw5XpRuKdkoe+D8+sP6BB+XZ+rxOKNDXpUZ jp6QSiKh2Aeq350LOj/7VIJh/o9pgGro8EbPC5ha0tMunH/6+3AjiKGmyoAuWkPEvqLjDd TwyjpYd+VWIJH2DsDJeWTErgaRv0XCiY+U40xfHAakT6EljY/0FT2k9HexkLZX1Xr+l3Rc OSsHoR1aWoPAH5YPwEa+Le8Tc4WP3veccPgmGF2UVvuEgyZuHkX0gEU6hLXzOQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) by mxrelay.nyi.freebsd.org (Postfix) with ESMTP id 4dsQsG2mZ9zYml for ; Thu, 15 Jan 2026 14:50:58 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from git (uid 1279) (envelope-from git@FreeBSD.org) id 23691 by gitrepo.freebsd.org (DragonFly Mail Agent v0.13+ on gitrepo.freebsd.org); Thu, 15 Jan 2026 14:50:58 +0000 To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andrew Turner Subject: git: 3b24eedc93b7 - stable/14 - arm64: Add a new CPU feature framework List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-all@freebsd.org Sender: owner-dev-commits-src-all@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: 3b24eedc93b77640280baf42b7ce560a32b0745b Auto-Submitted: auto-generated Date: Thu, 15 Jan 2026 14:50:58 +0000 Message-Id: <6968fed2.23691.23df8b11@gitrepo.freebsd.org> The branch stable/14 has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=3b24eedc93b77640280baf42b7ce560a32b0745b commit 3b24eedc93b77640280baf42b7ce560a32b0745b Author: Andrew Turner AuthorDate: 2025-01-24 11:42:29 +0000 Commit: Andrew Turner CommitDate: 2026-01-14 21:14:11 +0000 arm64: Add a new CPU feature framework This will be used to enable features that are discoverable on boot. It has support to check if a feature needs to be enabled, and if there is any errata associated with the feature that mean it shouldn't be enabled. It can also be used to enable errata workarounds that aren't associated with a specific feature, e.g. where we need to trap access to a register in userspace to emulate it. As some features need to be enabled early, while others can wait there are two options for when to enable a feature: * CPU_FEAT_EARLY_BOOT: The feature will be enabled early in the boot. On the boot CPU this is in initarm, while on secondary CPUs this is before signalling the CPU has started. * CPU_FEAT_AFTER_DEV: The feature will be enabled after devices have attached but before interrupts are enabled. There are also two scopes for where to enable the feature: * CPU_FEAT_PER_CPU: The feature will be checked and enabled on all CPU cores. * CPU_FEAT_SYSTEM: The feature will only be checked and enabled on a single core. It is expected the former will be used for most features, while the latter is for features that set a global variable to be checked. Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D47812 (cherry picked from commit ac4fa5838bb33f0c3ba05fce02d41164bd84a560) --- sys/arm64/arm64/cpu_feat.c | 119 +++++++++++++++++++++++++++++++++++++++++++ sys/arm64/arm64/machdep.c | 4 ++ sys/arm64/arm64/mp_machdep.c | 5 ++ sys/arm64/include/cpu_feat.h | 88 ++++++++++++++++++++++++++++++++ sys/conf/files.arm64 | 1 + 5 files changed, 217 insertions(+) diff --git a/sys/arm64/arm64/cpu_feat.c b/sys/arm64/arm64/cpu_feat.c new file mode 100644 index 000000000000..cc262394913d --- /dev/null +++ b/sys/arm64/arm64/cpu_feat.c @@ -0,0 +1,119 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2024 Arm Ltd + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include +#include +#include + +#include +#include + +/* TODO: Make this a list if we ever grow a callback other than smccc_errata */ +static cpu_feat_errata_check_fn cpu_feat_check_cb = NULL; + +void +enable_cpu_feat(uint32_t stage) +{ + struct cpu_feat **featp, *feat; + uint32_t midr; + u_int errata_count, *errata_list; + cpu_feat_errata errata_status; + + MPASS((stage & ~CPU_FEAT_STAGE_MASK) == 0); + + midr = get_midr(); + SET_FOREACH(featp, cpu_feat_set) { + feat = *featp; + + /* Run the enablement code at the correct stage of boot */ + if ((feat->feat_flags & CPU_FEAT_STAGE_MASK) != stage) + continue; + + /* If the feature is system wide run on a single CPU */ + if ((feat->feat_flags & CPU_FEAT_SCOPE_MASK)==CPU_FEAT_SYSTEM && + PCPU_GET(cpuid) != 0) + continue; + + if (feat->feat_check != NULL && !feat->feat_check(feat, midr)) + continue; + + /* + * Check if the feature has any errata that may need a + * workaround applied (or it is to install the workaround for + * known errata. + */ + errata_status = ERRATA_NONE; + errata_list = NULL; + errata_count = 0; + if (feat->feat_has_errata != NULL) { + if (feat->feat_has_errata(feat, midr, &errata_list, + &errata_count)) { + /* Assume we are affected */ + errata_status = ERRATA_AFFECTED; + } + } + + if (errata_status == ERRATA_AFFECTED && + cpu_feat_check_cb != NULL) { + for (int i = 0; i < errata_count; i++) { + cpu_feat_errata new_status; + + /* Check if affected by this erratum */ + new_status = cpu_feat_check_cb(feat, + errata_list[i]); + if (new_status != ERRATA_UNKNOWN) { + errata_status = new_status; + errata_list = &errata_list[i]; + errata_count = 1; + break; + } + } + } + + /* Shouldn't be possible */ + MPASS(errata_status != ERRATA_UNKNOWN); + + feat->feat_enable(feat, errata_status, errata_list, + errata_count); + } +} + +static void +enable_cpu_feat_after_dev(void *dummy __unused) +{ + MPASS(PCPU_GET(cpuid) == 0); + enable_cpu_feat(CPU_FEAT_AFTER_DEV); +} +SYSINIT(enable_cpu_feat_after_dev, SI_SUB_CONFIGURE, SI_ORDER_MIDDLE, + enable_cpu_feat_after_dev, NULL); + +void +cpu_feat_register_errata_check(cpu_feat_errata_check_fn cb) +{ + MPASS(cpu_feat_check_cb == NULL); + cpu_feat_check_cb = cb; +} diff --git a/sys/arm64/arm64/machdep.c b/sys/arm64/arm64/machdep.c index cdc812269542..bf7b9ea810e4 100644 --- a/sys/arm64/arm64/machdep.c +++ b/sys/arm64/arm64/machdep.c @@ -77,6 +77,7 @@ #include #include +#include #include #include #include @@ -990,6 +991,9 @@ initarm(struct arm64_bootparams *abp) panic("Invalid bus configuration: %s", kern_getenv("kern.cfg.order")); + /* Detect early CPU feature support */ + enable_cpu_feat(CPU_FEAT_EARLY_BOOT); + /* * Check if pointer authentication is available on this system, and * if so enable its use. This needs to be called before init_proc0 diff --git a/sys/arm64/arm64/mp_machdep.c b/sys/arm64/arm64/mp_machdep.c index 8dc967b0ccb3..4d111472694b 100644 --- a/sys/arm64/arm64/mp_machdep.c +++ b/sys/arm64/arm64/mp_machdep.c @@ -57,6 +57,7 @@ #include #include +#include #include #include #include @@ -222,6 +223,9 @@ init_secondary(uint64_t cpu) /* Ensure the stores in identify_cpu have completed */ atomic_thread_fence_acq_rel(); + /* Detect early CPU feature support */ + enable_cpu_feat(CPU_FEAT_EARLY_BOOT); + /* Signal the BSP and spin until it has released all APs. */ atomic_add_int(&aps_started, 1); while (!atomic_load_int(&aps_ready)) @@ -239,6 +243,7 @@ init_secondary(uint64_t cpu) pcpup->pc_curpmap = pmap0; install_cpu_errata(); + enable_cpu_feat(CPU_FEAT_AFTER_DEV); intr_pic_init_secondary(); diff --git a/sys/arm64/include/cpu_feat.h b/sys/arm64/include/cpu_feat.h new file mode 100644 index 000000000000..9fe6a9dd95d9 --- /dev/null +++ b/sys/arm64/include/cpu_feat.h @@ -0,0 +1,88 @@ +/*- + * SPDX-License-Identifier: BSD-2-Clause + * + * Copyright (c) 2024 Arm Ltd + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * 1. Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * 2. Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL + * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS + * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) + * HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT + * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY + * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef _MACHINE_CPU_FEAT_H_ +#define _MACHINE_CPU_FEAT_H_ + +#include + +typedef enum { + ERRATA_UNKNOWN, /* Unknown erratum */ + ERRATA_NONE, /* No errata for this feature on this system. */ + ERRATA_AFFECTED, /* There is errata on this system. */ + ERRATA_FW_MITIGAION, /* There is errata, and a firmware */ + /* mitigation. The mitigation may need a */ + /* kernel component. */ +} cpu_feat_errata; + +#define CPU_FEAT_STAGE_MASK 0x00000001 +#define CPU_FEAT_EARLY_BOOT 0x00000000 +#define CPU_FEAT_AFTER_DEV 0x00000001 + +#define CPU_FEAT_SCOPE_MASK 0x00000010 +#define CPU_FEAT_PER_CPU 0x00000000 +#define CPU_FEAT_SYSTEM 0x00000010 + +struct cpu_feat; + +typedef bool (cpu_feat_check)(const struct cpu_feat *, u_int); +typedef bool (cpu_feat_has_errata)(const struct cpu_feat *, u_int, + u_int **, u_int *); +typedef void (cpu_feat_enable)(const struct cpu_feat *, cpu_feat_errata, + u_int *, u_int); + +struct cpu_feat { + const char *feat_name; + cpu_feat_check *feat_check; + cpu_feat_has_errata *feat_has_errata; + cpu_feat_enable *feat_enable; + uint32_t feat_flags; +}; +SET_DECLARE(cpu_feat_set, struct cpu_feat); + +/* + * Allow drivers to mark an erratum as worked around, e.g. the Errata + * Management ABI may know the workaround isn't needed on a given system. + */ +typedef cpu_feat_errata (*cpu_feat_errata_check_fn)(const struct cpu_feat *, + u_int); +void cpu_feat_register_errata_check(cpu_feat_errata_check_fn); + +void enable_cpu_feat(uint32_t); + +/* Check if an erratum is in the list of errata */ +static inline bool +cpu_feat_has_erratum(u_int *errata_list, u_int errata_count, u_int erratum) +{ + for (u_int i = 0; i < errata_count; i++) + if (errata_list[0] == erratum) + return (true); + + return (false); +} + +#endif /* _MACHINE_CPU_FEAT_H_ */ diff --git a/sys/conf/files.arm64 b/sys/conf/files.arm64 index cf2e1d22da88..cfb4bf358f45 100644 --- a/sys/conf/files.arm64 +++ b/sys/conf/files.arm64 @@ -32,6 +32,7 @@ arm64/arm64/busdma_machdep.c standard arm64/arm64/clock.c standard arm64/arm64/copyinout.S standard arm64/arm64/cpu_errata.c standard +arm64/arm64/cpu_feat.c standard arm64/arm64/cpufunc_asm.S standard arm64/arm64/db_disasm.c optional ddb arm64/arm64/db_interface.c optional ddb