From owner-dev-commits-src-all@freebsd.org Thu Feb 11 15:25:55 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 60C9F52BBF4; Thu, 11 Feb 2021 15:25:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dc0nv2FzBz3vp2; Thu, 11 Feb 2021 15:25:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 403A0132D6; Thu, 11 Feb 2021 15:25:55 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 11BFPttG010971; Thu, 11 Feb 2021 15:25:55 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 11BFPtIN010970; Thu, 11 Feb 2021 15:25:55 GMT (envelope-from git) Date: Thu, 11 Feb 2021 15:25:55 GMT Message-Id: <202102111525.11BFPtIN010970@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Mark Johnston Subject: git: 860e0c7fb848 - stable/13 - armv8crypto: Fix some edge cases in the AES-GCM implementation MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: markj X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 860e0c7fb84863580521142825330aa941dee313 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 11 Feb 2021 15:25:55 -0000 The branch stable/13 has been updated by markj: URL: https://cgit.FreeBSD.org/src/commit/?id=860e0c7fb84863580521142825330aa941dee313 commit 860e0c7fb84863580521142825330aa941dee313 Author: Mark Johnston AuthorDate: 2021-02-08 14:19:07 +0000 Commit: Mark Johnston CommitDate: 2021-02-11 15:25:39 +0000 armv8crypto: Fix some edge cases in the AES-GCM implementation - We were only hashing up to the first 16 bytes of the AAD. - When computing the digest during decryption, handle the case where len == trailer, i.e., len < AES_BLOCK_LEN, properly. While here: - trailer is always smaller than AES_BLOCK_LEN, so remove a pair of unnecessary modulus operations. - Replace some byte-by-byte loops with memcpy() and memset() calls. In particular, zero the full block before copying a partial block into it since we do that elsewhere and it means that the memset() length is known at compile time. Reviewed by: jhb Sponsored by: Ampere Computing Submitted by: Klara, Inc. Differential Revision: https://reviews.freebsd.org/D28501 (cherry picked from commit 0dc7076037a87100060309f7179ef6a01f32f99e) --- sys/crypto/armv8/armv8_crypto_wrap.c | 47 ++++++++++++++++++++++-------------- 1 file changed, 29 insertions(+), 18 deletions(-) diff --git a/sys/crypto/armv8/armv8_crypto_wrap.c b/sys/crypto/armv8/armv8_crypto_wrap.c index ea93f1b9a176..2f880258bf46 100644 --- a/sys/crypto/armv8/armv8_crypto_wrap.c +++ b/sys/crypto/armv8/armv8_crypto_wrap.c @@ -263,9 +263,16 @@ armv8_aes_encrypt_gcm(AES_key_t *aes_key, size_t len, aes_counter[AES_BLOCK_LEN - 1] = 2; memset(Xi.c, 0, sizeof(Xi.c)); - memset(block, 0, sizeof(block)); - memcpy(block, authdata, min(authdatalen, sizeof(block))); - gcm_ghash_v8(Xi.u, Htable, block, AES_BLOCK_LEN); + trailer = authdatalen % AES_BLOCK_LEN; + if (authdatalen - trailer > 0) { + gcm_ghash_v8(Xi.u, Htable, authdata, authdatalen - trailer); + authdata += authdatalen - trailer; + } + if (trailer > 0 || authdatalen == 0) { + memset(block, 0, sizeof(block)); + memcpy(block, authdata, trailer); + gcm_ghash_v8(Xi.u, Htable, block, AES_BLOCK_LEN); + } from64 = (const uint64_t*)from; to64 = (uint64_t*)to; @@ -288,13 +295,11 @@ armv8_aes_encrypt_gcm(AES_key_t *aes_key, size_t len, if (trailer) { aes_v8_encrypt(aes_counter, EKi.c, aes_key); AES_INC_COUNTER(aes_counter); + memset(block, 0, sizeof(block)); for (i = 0; i < trailer; i++) { - block[i] = to[i] = from[i] ^ EKi.c[i % AES_BLOCK_LEN]; + block[i] = to[i] = from[i] ^ EKi.c[i]; } - for (; i < AES_BLOCK_LEN; i++) - block[i] = 0; - gcm_ghash_v8(Xi.u, Htable, block, AES_BLOCK_LEN); } @@ -343,17 +348,23 @@ armv8_aes_decrypt_gcm(AES_key_t *aes_key, size_t len, aes_v8_encrypt(aes_counter, EK0.c, aes_key); memset(Xi.c, 0, sizeof(Xi.c)); - memset(block, 0, sizeof(block)); - memcpy(block, authdata, min(authdatalen, sizeof(block))); - gcm_ghash_v8(Xi.u, Htable, block, AES_BLOCK_LEN); - trailer = len % AES_BLOCK_LEN; - gcm_ghash_v8(Xi.u, Htable, from, len - trailer); + trailer = authdatalen % AES_BLOCK_LEN; + if (authdatalen - trailer > 0) { + gcm_ghash_v8(Xi.u, Htable, authdata, authdatalen - trailer); + authdata += authdatalen - trailer; + } + if (trailer > 0 || authdatalen == 0) { + memset(block, 0, sizeof(block)); + memcpy(block, authdata, trailer); + gcm_ghash_v8(Xi.u, Htable, block, AES_BLOCK_LEN); + } - if (trailer) { - for (i = 0; i < trailer; i++) - block[i] = from[len - trailer + i]; - for (; i < AES_BLOCK_LEN; i++) - block[i] = 0; + trailer = len % AES_BLOCK_LEN; + if (len - trailer > 0) + gcm_ghash_v8(Xi.u, Htable, from, len - trailer); + if (trailer > 0) { + memset(block, 0, sizeof(block)); + memcpy(block, from + len - trailer, trailer); gcm_ghash_v8(Xi.u, Htable, block, AES_BLOCK_LEN); } @@ -392,7 +403,7 @@ armv8_aes_decrypt_gcm(AES_key_t *aes_key, size_t len, aes_v8_encrypt(aes_counter, EKi.c, aes_key); AES_INC_COUNTER(aes_counter); for (i = 0; i < trailer; i++) - to[i] = from[i] ^ EKi.c[i % AES_BLOCK_LEN]; + to[i] = from[i] ^ EKi.c[i]; } out: