From owner-svn-src-head@freebsd.org Mon Jun 22 23:41:34 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 8B5D0340181; Mon, 22 Jun 2020 23:41:34 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49rQsp3Bdsz4ZbQ; Mon, 22 Jun 2020 23:41:34 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 68D0D166A2; Mon, 22 Jun 2020 23:41:34 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05MNfYnZ032509; Mon, 22 Jun 2020 23:41:34 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05MNfY57032508; Mon, 22 Jun 2020 23:41:34 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <202006222341.05MNfY57032508@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Mon, 22 Jun 2020 23:41:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r362519 - head/sys/dev/cxgbe/crypto X-SVN-Group: head X-SVN-Commit-Author: jhb X-SVN-Commit-Paths: head/sys/dev/cxgbe/crypto X-SVN-Commit-Revision: 362519 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 22 Jun 2020 23:41:34 -0000 Author: jhb Date: Mon Jun 22 23:41:33 2020 New Revision: 362519 URL: https://svnweb.freebsd.org/changeset/base/362519 Log: Add support for requests with separate AAD to ccr(4). Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D25290 Modified: head/sys/dev/cxgbe/crypto/t4_crypto.c Modified: head/sys/dev/cxgbe/crypto/t4_crypto.c ============================================================================== --- head/sys/dev/cxgbe/crypto/t4_crypto.c Mon Jun 22 23:22:13 2020 (r362518) +++ head/sys/dev/cxgbe/crypto/t4_crypto.c Mon Jun 22 23:41:33 2020 (r362519) @@ -387,7 +387,6 @@ ccr_write_ulptx_sgl(struct ccr_softc *sc, void *dst, i usgl->sge[i / 2].addr[i & 1] = htobe64(ss->ss_paddr); ss++; } - } static bool @@ -919,8 +918,13 @@ ccr_eta(struct ccr_softc *sc, struct ccr_session *s, s imm_len = 0; sglist_reset(sc->sg_ulptx); if (crp->crp_aad_length != 0) { - error = sglist_append_sglist(sc->sg_ulptx, sc->sg_input, - crp->crp_aad_start, crp->crp_aad_length); + if (crp->crp_aad != NULL) + error = sglist_append(sc->sg_ulptx, + crp->crp_aad, crp->crp_aad_length); + else + error = sglist_append_sglist(sc->sg_ulptx, + sc->sg_input, crp->crp_aad_start, + crp->crp_aad_length); if (error) return (error); } @@ -938,11 +942,7 @@ ccr_eta(struct ccr_softc *sc, struct ccr_session *s, s sgl_len = ccr_ulptx_sgl_len(sgl_nsegs); } - /* - * Any auth-only data before the cipher region is marked as AAD. - * Auth-data that overlaps with the cipher region is placed in - * the auth section. - */ + /* Any AAD comes after the IV. */ if (crp->crp_aad_length != 0) { aad_start = iv_len + 1; aad_stop = aad_start + crp->crp_aad_length - 1; @@ -1054,8 +1054,11 @@ ccr_eta(struct ccr_softc *sc, struct ccr_session *s, s dst += iv_len; if (imm_len != 0) { if (crp->crp_aad_length != 0) { - crypto_copydata(crp, crp->crp_aad_start, - crp->crp_aad_length, dst); + if (crp->crp_aad != NULL) + memcpy(dst, crp->crp_aad, crp->crp_aad_length); + else + crypto_copydata(crp, crp->crp_aad_start, + crp->crp_aad_length, dst); dst += crp->crp_aad_length; } crypto_copydata(crp, crp->crp_payload_start, @@ -1224,8 +1227,13 @@ ccr_gcm(struct ccr_softc *sc, struct ccr_session *s, s imm_len = 0; sglist_reset(sc->sg_ulptx); if (crp->crp_aad_length != 0) { - error = sglist_append_sglist(sc->sg_ulptx, sc->sg_input, - crp->crp_aad_start, crp->crp_aad_length); + if (crp->crp_aad != NULL) + error = sglist_append(sc->sg_ulptx, + crp->crp_aad, crp->crp_aad_length); + else + error = sglist_append_sglist(sc->sg_ulptx, + sc->sg_input, crp->crp_aad_start, + crp->crp_aad_length); if (error) return (error); } @@ -1337,8 +1345,11 @@ ccr_gcm(struct ccr_softc *sc, struct ccr_session *s, s dst += iv_len; if (imm_len != 0) { if (crp->crp_aad_length != 0) { - crypto_copydata(crp, crp->crp_aad_start, - crp->crp_aad_length, dst); + if (crp->crp_aad != NULL) + memcpy(dst, crp->crp_aad, crp->crp_aad_length); + else + crypto_copydata(crp, crp->crp_aad_start, + crp->crp_aad_length, dst); dst += crp->crp_aad_length; } crypto_copydata(crp, crp->crp_payload_start, @@ -1438,11 +1449,24 @@ ccr_gcm_soft(struct ccr_session *s, struct cryptop *cr axf->Reinit(auth_ctx, iv, sizeof(iv)); /* MAC the AAD. */ - for (i = 0; i < crp->crp_aad_length; i += sizeof(block)) { - len = imin(crp->crp_aad_length - i, sizeof(block)); - crypto_copydata(crp, crp->crp_aad_start + i, len, block); - bzero(block + len, sizeof(block) - len); - axf->Update(auth_ctx, block, sizeof(block)); + if (crp->crp_aad != NULL) { + len = rounddown(crp->crp_aad_length, sizeof(block)); + if (len != 0) + axf->Update(auth_ctx, crp->crp_aad, len); + if (crp->crp_aad_length != len) { + memset(block, 0, sizeof(block)); + memcpy(block, (char *)crp->crp_aad + len, + crp->crp_aad_length - len); + axf->Update(auth_ctx, block, sizeof(block)); + } + } else { + for (i = 0; i < crp->crp_aad_length; i += sizeof(block)) { + len = imin(crp->crp_aad_length - i, sizeof(block)); + crypto_copydata(crp, crp->crp_aad_start + i, len, + block); + bzero(block + len, sizeof(block) - len); + axf->Update(auth_ctx, block, sizeof(block)); + } } exf->reinit(kschedule, iv); @@ -1679,8 +1703,13 @@ ccr_ccm(struct ccr_softc *sc, struct ccr_session *s, s sglist_reset(sc->sg_ulptx); if (crp->crp_aad_length != 0) { - error = sglist_append_sglist(sc->sg_ulptx, sc->sg_input, - crp->crp_aad_start, crp->crp_aad_length); + if (crp->crp_aad != NULL) + error = sglist_append(sc->sg_ulptx, + crp->crp_aad, crp->crp_aad_length); + else + error = sglist_append_sglist(sc->sg_ulptx, + sc->sg_input, crp->crp_aad_start, + crp->crp_aad_length); if (error) return (error); } @@ -1788,8 +1817,11 @@ ccr_ccm(struct ccr_softc *sc, struct ccr_session *s, s if (sgl_nsegs == 0) { dst += b0_len; if (crp->crp_aad_length != 0) { - crypto_copydata(crp, crp->crp_aad_start, - crp->crp_aad_length, dst); + if (crp->crp_aad != NULL) + memcpy(dst, crp->crp_aad, crp->crp_aad_length); + else + crypto_copydata(crp, crp->crp_aad_start, + crp->crp_aad_length, dst); dst += crp->crp_aad_length; } crypto_copydata(crp, crp->crp_payload_start, @@ -1905,12 +1937,14 @@ ccr_ccm_soft(struct ccr_session *s, struct cryptop *cr axf->Reinit(auth_ctx, iv, sizeof(iv)); /* MAC the AAD. */ - for (i = 0; i < crp->crp_aad_length; i += sizeof(block)) { - len = imin(crp->crp_aad_length - i, sizeof(block)); - crypto_copydata(crp, crp->crp_aad_start + i, len, block); - bzero(block + len, sizeof(block) - len); - axf->Update(auth_ctx, block, sizeof(block)); - } + if (crp->crp_aad != NULL) + error = axf->Update(auth_ctx, crp->crp_aad, + crp->crp_aad_length); + else + error = crypto_apply(crp, crp->crp_aad_start, + crp->crp_aad_length, axf->Update, auth_ctx); + if (error) + goto out; exf->reinit(kschedule, iv); @@ -2339,7 +2373,8 @@ ccr_probesession(device_t dev, const struct crypto_ses { unsigned int cipher_mode; - if ((csp->csp_flags & ~(CSP_F_SEPARATE_OUTPUT)) != 0) + if ((csp->csp_flags & ~(CSP_F_SEPARATE_OUTPUT | CSP_F_SEPARATE_AAD)) != + 0) return (EINVAL); switch (csp->csp_mode) { case CSP_MODE_DIGEST: