From nobody Mon Sep 30 12:23:41 2024 X-Original-To: dev-commits-src-main@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4XHKxB10xPz5YDK8; Mon, 30 Sep 2024 12:23:42 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R11" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4XHKx95cywz4Lsr; Mon, 30 Sep 2024 12:23:41 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1727699021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LA4jvNLeGJhEG8Rdt+O2UUzPloQ+sE8xSIIjzRRibGM=; b=C9NyCN1qC+qK5SZ1WEc5yHUO3KxCXVWG1zfw25y2e5Z21TbNdH72UvBFdHyEKuIUhpTqyw Yg2Mh5c5nYDBpdhjpzaIs5JzkfDsk11Drdav5/E8bi9mfvF08zoU2k8iI4n13zgKASViik e8j4VLvQ3dGkJDHPJ0mNZZ+crVfgiKMmD+xBgfIY2gLXJ1nBaIfam+DUBoDtfwWcybSNpx 37ATw7OWZjODURSWTr9Vt76OdblkF3Y0H78KB0O2M4lBLHihKiHx348zitaauJE9cKUKSK yUhiPlC417G/hII1jbKf2On8b9tGmPwIBgAA13Svzcpdx1ZpSv4tMabonIuUaw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1727699021; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=LA4jvNLeGJhEG8Rdt+O2UUzPloQ+sE8xSIIjzRRibGM=; b=kCrt6aLHoKXg1ze+vqdIxB06wgPZ58pWGnprWu0W8nLTcDaecwU6kaBgEMVwiN//5jjUYL S3pYfLz2A/gn/3sh5s1pApZBFco9cCEe8K8n0SKoYjqA5YlzXQ4Tu/tPyjz/eXIOY9Jyc1 82Wd6NuMYuNcu74iDgeuzLKhc/XFGBghcK7G5eUg8EFRasBVaIlv4Kf2hYLPgAGmoooPTD 6PLrMvUn/NIo9fq4R1qwkgKlLHX99xRFKwV4J23JM3Noomaze7h4Z6zRrAwaNz2PTX88ok Q9NJwrvipWbTJkbSOEZloYgd37pJlktyamjHDSUKD11TpsTAbZxWNGjLkWLwWQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1727699021; a=rsa-sha256; cv=none; b=WtqTbDi7dgXsxXygrOvOgwZMOhYuL+KanLxHr/bLb6dymT8zlQGp6Dg+iPDmlgreURPZPK F9gq7VzAWoFvtPjM3C/+4rZyeVJ967ZgEIVJIQ1/UlfRO2XBst4SDoPU7AqOXes4BqrREb Ifi2W494AP9Os4ff3CWWouidja4EInT26T2AIhG67F7SKjRDavII8tGlkg9YWj4REMS8vF AmLLiJ6MuHkCI2Kqn0OfU/7AmThKVWWDSZwyXtqlOcDi5nZmRRMCHi6Nw6Sxeeq2LSz8Tq 4odnRF4ZZQJUrws0VqmO2LT5oJDFLVzf3htLNs1uNWpZjVEAjI3fSvxyl0D69A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4XHKx95BxNzsZp; Mon, 30 Sep 2024 12:23:41 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.18.1/8.18.1) with ESMTP id 48UCNfHM071106; Mon, 30 Sep 2024 12:23:41 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.18.1/8.18.1/Submit) id 48UCNfH4071103; Mon, 30 Sep 2024 12:23:41 GMT (envelope-from git) Date: Mon, 30 Sep 2024 12:23:41 GMT Message-Id: <202409301223.48UCNfH4071103@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Andrew Turner Subject: git: 205c1007be6d - main - arm64: Enable SVE in userspace List-Id: Commit messages for the main branch of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-main List-Help: List-Post: List-Subscribe: List-Unsubscribe: X-BeenThere: dev-commits-src-main@freebsd.org Sender: owner-dev-commits-src-main@FreeBSD.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: andrew X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 205c1007be6d57b2eb8620f79384cb2299dde40e Auto-Submitted: auto-generated The branch main has been updated by andrew: URL: https://cgit.FreeBSD.org/src/commit/?id=205c1007be6d57b2eb8620f79384cb2299dde40e commit 205c1007be6d57b2eb8620f79384cb2299dde40e Author: Andrew Turner AuthorDate: 2024-09-27 14:06:03 +0000 Commit: Andrew Turner CommitDate: 2024-09-30 12:04:23 +0000 arm64: Enable SVE in userspace Report when SVE is present and allow it to be used by calling sve_restore_state on an SVE exception from userspace. Reviewed by: kib Sponsored by: Arm Ltd Differential Revision: https://reviews.freebsd.org/D43310 --- sys/arm64/arm64/identcpu.c | 82 ++++++++++++++++++++++++++++++++++++++-------- sys/arm64/arm64/trap.c | 6 ++-- sys/arm64/include/armreg.h | 1 + 3 files changed, 74 insertions(+), 15 deletions(-) diff --git a/sys/arm64/arm64/identcpu.c b/sys/arm64/arm64/identcpu.c index eead0051d315..3bf4f9a80c90 100644 --- a/sys/arm64/arm64/identcpu.c +++ b/sys/arm64/arm64/identcpu.c @@ -1383,13 +1383,10 @@ static const struct mrs_field_value id_aa64pfr0_sve[] = { MRS_FIELD_VALUE_END, }; -#if 0 -/* Enable when we add SVE support */ static const struct mrs_field_hwcap id_aa64pfr0_sve_caps[] = { MRS_HWCAP(1, HWCAP_SVE, ID_AA64PFR0_SVE_IMPL), MRS_HWCAP_END }; -#endif static const struct mrs_field_value id_aa64pfr0_ras[] = { MRS_FIELD_VALUE(ID_AA64PFR0_RAS_NONE, ""), @@ -1463,7 +1460,8 @@ static const struct mrs_field id_aa64pfr0_fields[] = { MRS_FIELD(ID_AA64PFR0, AMU, false, MRS_EXACT, id_aa64pfr0_amu), MRS_FIELD(ID_AA64PFR0, MPAM, false, MRS_EXACT, id_aa64pfr0_mpam), MRS_FIELD(ID_AA64PFR0, SEL2, false, MRS_EXACT, id_aa64pfr0_sel2), - MRS_FIELD(ID_AA64PFR0, SVE, false, MRS_EXACT, id_aa64pfr0_sve), + MRS_FIELD_HWCAP_SPLIT(ID_AA64PFR0, SVE, false, MRS_LOWER, MRS_EXACT, + id_aa64pfr0_sve, id_aa64pfr0_sve_caps), MRS_FIELD(ID_AA64PFR0, RAS, false, MRS_EXACT, id_aa64pfr0_ras), MRS_FIELD(ID_AA64PFR0, GIC, false, MRS_EXACT, id_aa64pfr0_gic), MRS_FIELD_HWCAP(ID_AA64PFR0, AdvSIMD, true, MRS_LOWER, @@ -1578,59 +1576,117 @@ static const struct mrs_field_value id_aa64zfr0_f64mm[] = { MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_f64mm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEF64MM, ID_AA64ZFR0_F64MM_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_f32mm[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, F32MM, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_f32mm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEF32MM, ID_AA64ZFR0_F32MM_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_i8mm[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, I8MM, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_i8mm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEI8MM, ID_AA64ZFR0_I8MM_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_sm4[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, SM4, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_sm4_caps[] = { + MRS_HWCAP(2, HWCAP2_SVESM4, ID_AA64ZFR0_SM4_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_sha3[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, SHA3, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_sha3_caps[] = { + MRS_HWCAP(2, HWCAP2_SVESHA3, ID_AA64ZFR0_SHA3_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_bf16[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, BF16, NONE, BASE), MRS_FIELD_VALUE(ID_AA64ZFR0_BF16_EBF, "BF16+EBF"), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_bf16_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEBF16, ID_AA64ZFR0_BF16_BASE), + MRS_HWCAP(2, HWCAP2_SVE_EBF16, ID_AA64ZFR0_BF16_EBF), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_bitperm[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, BitPerm, NONE, IMPL), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_bitperm_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEBITPERM, ID_AA64ZFR0_BitPerm_IMPL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_aes[] = { MRS_FIELD_VALUE_NONE_IMPL(ID_AA64ZFR0, AES, NONE, BASE), MRS_FIELD_VALUE(ID_AA64ZFR0_AES_PMULL, "AES+PMULL"), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_aes_caps[] = { + MRS_HWCAP(2, HWCAP2_SVEAES, ID_AA64ZFR0_AES_BASE), + MRS_HWCAP(2, HWCAP2_SVEPMULL, ID_AA64ZFR0_AES_PMULL), + MRS_HWCAP_END, +}; + static const struct mrs_field_value id_aa64zfr0_svever[] = { MRS_FIELD_VALUE(ID_AA64ZFR0_SVEver_SVE1, "SVE1"), MRS_FIELD_VALUE(ID_AA64ZFR0_SVEver_SVE2, "SVE2"), + MRS_FIELD_VALUE(ID_AA64ZFR0_SVEver_SVE2P1, "SVE2P1"), MRS_FIELD_VALUE_END, }; +static const struct mrs_field_hwcap id_aa64zfr0_svever_caps[] = { + MRS_HWCAP(2, HWCAP2_SVE2, ID_AA64ZFR0_SVEver_SVE2), + MRS_HWCAP(2, HWCAP2_SVE2P1, ID_AA64ZFR0_SVEver_SVE2P1), + MRS_HWCAP_END, +}; + static const struct mrs_field id_aa64zfr0_fields[] = { - MRS_FIELD(ID_AA64ZFR0, F64MM, false, MRS_EXACT, id_aa64zfr0_f64mm), - MRS_FIELD(ID_AA64ZFR0, F32MM, false, MRS_EXACT, id_aa64zfr0_f32mm), - MRS_FIELD(ID_AA64ZFR0, I8MM, false, MRS_EXACT, id_aa64zfr0_i8mm), - MRS_FIELD(ID_AA64ZFR0, SM4, false, MRS_EXACT, id_aa64zfr0_sm4), - MRS_FIELD(ID_AA64ZFR0, SHA3, false, MRS_EXACT, id_aa64zfr0_sha3), - MRS_FIELD(ID_AA64ZFR0, BF16, false, MRS_EXACT, id_aa64zfr0_bf16), - MRS_FIELD(ID_AA64ZFR0, BitPerm, false, MRS_EXACT, id_aa64zfr0_bitperm), - MRS_FIELD(ID_AA64ZFR0, AES, false, MRS_EXACT, id_aa64zfr0_aes), - MRS_FIELD(ID_AA64ZFR0, SVEver, false, MRS_EXACT, id_aa64zfr0_svever), + MRS_FIELD_HWCAP(ID_AA64ZFR0, F64MM, false, MRS_LOWER, + id_aa64zfr0_f64mm, id_aa64zfr0_f64mm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, F32MM, false, MRS_LOWER, + id_aa64zfr0_f32mm, id_aa64zfr0_f32mm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, I8MM, false, MRS_LOWER, id_aa64zfr0_i8mm, + id_aa64zfr0_i8mm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, SM4, false, MRS_LOWER, id_aa64zfr0_sm4, + id_aa64zfr0_sm4_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, SHA3, false, MRS_LOWER, id_aa64zfr0_sha3, + id_aa64zfr0_sha3_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, BF16, false, MRS_LOWER, id_aa64zfr0_bf16, + id_aa64zfr0_bf16_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, BitPerm, false, MRS_LOWER, + id_aa64zfr0_bitperm, id_aa64zfr0_bitperm_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, AES, false, MRS_LOWER, id_aa64zfr0_aes, + id_aa64zfr0_aes_caps), + MRS_FIELD_HWCAP(ID_AA64ZFR0, SVEver, false, MRS_LOWER, + id_aa64zfr0_svever, id_aa64zfr0_svever_caps), MRS_FIELD_END, }; diff --git a/sys/arm64/arm64/trap.c b/sys/arm64/arm64/trap.c index 35defef90837..fa0b46111a34 100644 --- a/sys/arm64/arm64/trap.c +++ b/sys/arm64/arm64/trap.c @@ -639,8 +639,10 @@ do_el0_sync(struct thread *td, struct trapframe *frame) #endif break; case EXCP_SVE: - call_trapsignal(td, SIGILL, ILL_ILLTRP, (void *)frame->tf_elr, - exception); + /* Returns true if this thread can use SVE */ + if (!sve_restore_state(td)) + call_trapsignal(td, SIGILL, ILL_ILLTRP, + (void *)frame->tf_elr, exception); userret(td, frame); break; case EXCP_SVC32: diff --git a/sys/arm64/include/armreg.h b/sys/arm64/include/armreg.h index 667353831f48..08a046762a58 100644 --- a/sys/arm64/include/armreg.h +++ b/sys/arm64/include/armreg.h @@ -1583,6 +1583,7 @@ #define ID_AA64ZFR0_SVEver_VAL(x) ((x) & ID_AA64ZFR0_SVEver_MASK #define ID_AA64ZFR0_SVEver_SVE1 (UL(0x0) << ID_AA64ZFR0_SVEver_SHIFT) #define ID_AA64ZFR0_SVEver_SVE2 (UL(0x1) << ID_AA64ZFR0_SVEver_SHIFT) +#define ID_AA64ZFR0_SVEver_SVE2P1 (UL(0x2) << ID_AA64ZFR0_SVEver_SHIFT) #define ID_AA64ZFR0_AES_SHIFT 4 #define ID_AA64ZFR0_AES_MASK (UL(0xf) << ID_AA64ZFR0_AES_SHIFT) #define ID_AA64ZFR0_AES_VAL(x) ((x) & ID_AA64ZFR0_AES_MASK