From nobody Mon Aug 28 23:24:57 2023 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4RZRVK3WS2z4s457; Mon, 28 Aug 2023 23:24:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4RZRVK2rYTz4Z33; Mon, 28 Aug 2023 23:24:57 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1693265097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=s6xV2n5gMUnlu1Msd9MKb6P7RTu08TlNgacaO8UCmGA=; b=aXy9yDSzGrpBYtmFmMbxO1xY2W3HgtGxQStejwncvWHZj5U+2cE6htlP4jLQYm7DNF/RwH P8pGP/9k/nDoTXi5YOfnxLkCRZtxQ7aCk8A/GtI6BxTf57Q9ZQDoTnEQ0nK4neYKiiNZ7n qCm9XkQFfsrAvRzKSXuAmPWjre4FKnUq7vAAbquyKZFz7peqlLfF14mtwwCxcFrjVbqeA5 jn8q7JlJ/yIyglN+uwR7g8/UZv+6AvIfQZurz1cL8o16qz1rfm3L89ZNHWNZKlWRpOap39 wq8ZhxISmQJqTC5aqxGJvzpoXmBgLvC+muwaaZP4tONXFstuc0JxI9ZDXPP+Sw== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1693265097; a=rsa-sha256; cv=none; b=bF6EARVnx5bZwqyd2+Bau92ghI0hzbgSl+MYGQYgjaGnN0X9cNjGVW3dU/45B/cCFBDbn2 hlKeeIK6r2fqFLf1+uk1fPcqCtlpdFqzL+Bu84QYbjDbP/DSwMTs7AFUfY6Q9vlJfdnj1Y LW2xrBT9vsuzQ7VBJOYyqm7Z9Jv8G/1lo+hBAcVumUI2FkupL94phvBmVyz0QXzaa4rR6r fE803vDPpEEq2+3sSWdR2gNb2zlnq/LP2Yb4TNO4S74T8PNx4yXqhaje4y5Bk15rvMIWyW W2ivLL10VSL+Ifdc7dvckJa/YixkGAaQ8SV4YMcZxnKduePgP98nvU/PI+JV8Q== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1693265097; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=s6xV2n5gMUnlu1Msd9MKb6P7RTu08TlNgacaO8UCmGA=; b=uYPLb8cUvX8X7ac0YZ2GiQ8XZ3gJTfA0hf/vAKi945Mc3GOfWEsK/Xkq303It+TGMp+FgG 2zvzkQHxDM+O5ptEH9EF52WCpTWLC2YzgmUw7c5Yo+WiMxvv6mH6BHKO/8w++5Vi5BUAGf Z+KgBgdWTVuR4w5UeiXXP2Ih/fug08tU9GhB0gp8mo/lGukp5gryE6KeKu70n3HG7nk/8q o5Q2wBWAIrrqVwTFJcwxhbwEmkRE+LIKMJku3mtbakWUtRajYyxOgQryFSgp9SHvux0bXS 9ziIRrKmUEFnbsRmiAju1/56MBtuUksZ4ao45yh/tYu60lNDXJdvXG1Tejbztg== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4RZRVK1xQszD8m; Mon, 28 Aug 2023 23:24:57 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 37SNOvW2060533; Mon, 28 Aug 2023 23:24:57 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 37SNOvjp060530; Mon, 28 Aug 2023 23:24:57 GMT (envelope-from git) Date: Mon, 28 Aug 2023 23:24:57 GMT Message-Id: <202308282324.37SNOvjp060530@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 6485286f536f - main - armv8_crypto: Switch to using FPU_KERN_NOCTX List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 6485286f536f1afb36d6d39e75fceabd114b082b Auto-Submitted: auto-generated The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=6485286f536f1afb36d6d39e75fceabd114b082b commit 6485286f536f1afb36d6d39e75fceabd114b082b Author: John Baldwin AuthorDate: 2023-08-28 23:22:33 +0000 Commit: John Baldwin CommitDate: 2023-08-28 23:22:33 +0000 armv8_crypto: Switch to using FPU_KERN_NOCTX Reviewed by: markj Differential Revision: https://reviews.freebsd.org/D41578 --- sys/crypto/armv8/armv8_crypto.c | 85 +++++------------------------------------ 1 file changed, 9 insertions(+), 76 deletions(-) diff --git a/sys/crypto/armv8/armv8_crypto.c b/sys/crypto/armv8/armv8_crypto.c index a1a9d4dfcdcd..a8d88c31edc0 100644 --- a/sys/crypto/armv8/armv8_crypto.c +++ b/sys/crypto/armv8/armv8_crypto.c @@ -48,7 +48,6 @@ #include #include #include -#include #include #include #include @@ -69,22 +68,6 @@ struct armv8_crypto_softc { bool has_pmul; }; -static struct mtx *ctx_mtx; -static struct fpu_kern_ctx **ctx_vfp; - -#define AQUIRE_CTX(i, ctx) \ - do { \ - (i) = PCPU_GET(cpuid); \ - mtx_lock(&ctx_mtx[(i)]); \ - (ctx) = ctx_vfp[(i)]; \ - } while (0) -#define RELEASE_CTX(i, ctx) \ - do { \ - mtx_unlock(&ctx_mtx[(i)]); \ - (i) = -1; \ - (ctx) = NULL; \ - } while (0) - static int armv8_crypto_cipher_process(struct armv8_crypto_session *, struct cryptop *); @@ -134,7 +117,6 @@ armv8_crypto_attach(device_t dev) { struct armv8_crypto_softc *sc; uint64_t reg; - int i; sc = device_get_softc(dev); sc->dieing = 0; @@ -153,16 +135,6 @@ armv8_crypto_attach(device_t dev) rw_init(&sc->lock, "armv8crypto"); - ctx_mtx = malloc(sizeof(*ctx_mtx) * (mp_maxid + 1), M_ARMV8_CRYPTO, - M_WAITOK|M_ZERO); - ctx_vfp = malloc(sizeof(*ctx_vfp) * (mp_maxid + 1), M_ARMV8_CRYPTO, - M_WAITOK|M_ZERO); - - CPU_FOREACH(i) { - ctx_vfp[i] = fpu_kern_alloc_ctx(0); - mtx_init(&ctx_mtx[i], "armv8cryptoctx", NULL, MTX_DEF|MTX_NEW); - } - return (0); } @@ -170,7 +142,6 @@ static int armv8_crypto_detach(device_t dev) { struct armv8_crypto_softc *sc; - int i; sc = device_get_softc(dev); @@ -181,18 +152,6 @@ armv8_crypto_detach(device_t dev) rw_destroy(&sc->lock); - CPU_FOREACH(i) { - if (ctx_vfp[i] != NULL) { - mtx_destroy(&ctx_mtx[i]); - fpu_kern_free_ctx(ctx_vfp[i]); - } - ctx_vfp[i] = NULL; - } - free(ctx_mtx, M_ARMV8_CRYPTO); - ctx_mtx = NULL; - free(ctx_vfp, M_ARMV8_CRYPTO); - ctx_vfp = NULL; - return (0); } @@ -271,8 +230,6 @@ armv8_crypto_cipher_setup(struct armv8_crypto_session *ses, const struct crypto_session_params *csp, const uint8_t *key, int keylen) { __uint128_val_t H; - struct fpu_kern_ctx *ctx; - int kt, i; if (csp->csp_cipher_alg == CRYPTO_AES_XTS) keylen /= 2; @@ -286,12 +243,7 @@ armv8_crypto_cipher_setup(struct armv8_crypto_session *ses, return (EINVAL); } - kt = is_fpu_kern_thread(0); - if (!kt) { - AQUIRE_CTX(i, ctx); - fpu_kern_enter(curthread, ctx, - FPU_KERN_NORMAL | FPU_KERN_KTHR); - } + fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); aes_v8_set_encrypt_key(key, keylen * 8, &ses->enc_schedule); @@ -312,10 +264,7 @@ armv8_crypto_cipher_setup(struct armv8_crypto_session *ses, gcm_init_v8(ses->Htable, H.u); } - if (!kt) { - fpu_kern_leave(curthread, ctx); - RELEASE_CTX(i, ctx); - } + fpu_kern_leave(curthread, NULL); return (0); } @@ -378,12 +327,10 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, { struct crypto_buffer_cursor fromc, toc; const struct crypto_session_params *csp; - struct fpu_kern_ctx *ctx; uint8_t *authbuf; uint8_t iv[AES_BLOCK_LEN], tag[GMAC_DIGEST_LEN]; - int authallocated, i; + int authallocated; int encflag; - int kt; int error; csp = crypto_get_params(crp->crp_session); @@ -391,7 +338,6 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, authallocated = 0; authbuf = NULL; - kt = 1; if (csp->csp_cipher_alg == CRYPTO_AES_NIST_GCM_16) { if (crp->crp_aad != NULL) @@ -399,10 +345,8 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, else authbuf = armv8_crypto_cipher_alloc(crp, crp->crp_aad_start, crp->crp_aad_length, &authallocated); - if (authbuf == NULL) { - error = ENOMEM; - goto out; - } + if (authbuf == NULL) + return (ENOMEM); } crypto_cursor_init(&fromc, &crp->crp_buf); crypto_cursor_advance(&fromc, crp->crp_payload_start); @@ -413,12 +357,7 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, crypto_cursor_copy(&fromc, &toc); } - kt = is_fpu_kern_thread(0); - if (!kt) { - AQUIRE_CTX(i, ctx); - fpu_kern_enter(curthread, ctx, - FPU_KERN_NORMAL | FPU_KERN_KTHR); - } + fpu_kern_enter(curthread, NULL, FPU_KERN_NORMAL | FPU_KERN_NOCTX); if (crp->crp_cipher_key != NULL) { armv8_crypto_cipher_setup(ses, csp, crp->crp_cipher_key, @@ -427,11 +366,12 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, crypto_read_iv(crp, iv); + error = 0; switch (csp->csp_cipher_alg) { case CRYPTO_AES_CBC: if ((crp->crp_payload_length % AES_BLOCK_LEN) != 0) { error = EINVAL; - goto out; + break; } if (encflag) armv8_aes_encrypt_cbc(&ses->enc_schedule, @@ -464,18 +404,11 @@ armv8_crypto_cipher_process(struct armv8_crypto_session *ses, error = armv8_aes_decrypt_gcm(&ses->enc_schedule, crp->crp_payload_length, &fromc, &toc, crp->crp_aad_length, authbuf, tag, iv, ses->Htable); - if (error != 0) - goto out; } break; } - error = 0; -out: - if (!kt) { - fpu_kern_leave(curthread, ctx); - RELEASE_CTX(i, ctx); - } + fpu_kern_leave(curthread, NULL); if (authallocated) zfree(authbuf, M_ARMV8_CRYPTO);