From owner-dev-commits-src-all@freebsd.org Fri Mar 12 19:00:21 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4BA525AA68C; Fri, 12 Mar 2021 19:00:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Dxw9x1fkCz4kqt; Fri, 12 Mar 2021 19:00:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 2B8E61CACD; Fri, 12 Mar 2021 19:00:21 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 12CJ0LSJ068538; Fri, 12 Mar 2021 19:00:21 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 12CJ0LIo068537; Fri, 12 Mar 2021 19:00:21 GMT (envelope-from git) Date: Fri, 12 Mar 2021 19:00:21 GMT Message-Id: <202103121900.12CJ0LIo068537@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: John Baldwin Subject: git: 8f885fd1f381 - main - ccr: Set the RX channel ID correctly in work requests. MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: jhb X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 8f885fd1f38159a06db82d680fa259f358e9f872 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 12 Mar 2021 19:00:21 -0000 The branch main has been updated by jhb: URL: https://cgit.FreeBSD.org/src/commit/?id=8f885fd1f38159a06db82d680fa259f358e9f872 commit 8f885fd1f38159a06db82d680fa259f358e9f872 Author: John Baldwin AuthorDate: 2021-03-12 18:35:05 +0000 Commit: John Baldwin CommitDate: 2021-03-12 18:59:35 +0000 ccr: Set the RX channel ID correctly in work requests. These fixes are only relevant for requests on the second port. In some cases, the crypto completion data, completion message, and receive descriptor could be written in the wrong order. - Add a separate rx_channel_id that is a copy of the port's rx_c_chan and use it when an RX channel ID is required in crypto requests instead of using the tx_channel_id. - Set the correct rx_channel_id in the CPL_RX_PHYS_ADDR used to write the crypto result. - Set the FID to the first rx queue ID on the adapter rather than the queue ID of the first rx queue for the port. - While here, use tx_chan to set the tx_channel_id though this is identical to the previous value. Reviewed by: np Reported by: Chelsio QA Sponsored by: Chelsio Communications Differential Revision: https://reviews.freebsd.org/D29175 --- sys/dev/cxgbe/crypto/t4_crypto.c | 33 ++++++++++++++++++++++----------- 1 file changed, 22 insertions(+), 11 deletions(-) diff --git a/sys/dev/cxgbe/crypto/t4_crypto.c b/sys/dev/cxgbe/crypto/t4_crypto.c index 8861e710cfd2..46ea9d778fe3 100644 --- a/sys/dev/cxgbe/crypto/t4_crypto.c +++ b/sys/dev/cxgbe/crypto/t4_crypto.c @@ -165,6 +165,7 @@ struct ccr_session_blkcipher { struct ccr_port { struct sge_wrq *txq; struct sge_rxq *rxq; + int rx_channel_id; int tx_channel_id; u_int active_sessions; }; @@ -207,6 +208,7 @@ struct ccr_softc { bool detaching; struct ccr_port ports[MAX_NPORTS]; u_int port_mask; + int first_rxq_id; /* * Pre-allocate a dummy output buffer for the IV and AAD for @@ -333,6 +335,7 @@ ccr_write_phys_dsgl(struct ccr_session *s, void *dst, int nsegs) cpl->rss_hdr_int.opcode = CPL_RX_PHYS_ADDR; cpl->rss_hdr_int.qid = htobe16(s->port->rxq->iq.abs_id); cpl->rss_hdr_int.hash_val = 0; + cpl->rss_hdr_int.channel = s->port->rx_channel_id; sgl = (struct phys_sge_pairs *)(cpl + 1); j = 0; for (i = 0; i < sg->sg_nseg; i++) { @@ -423,12 +426,12 @@ ccr_populate_wreq(struct ccr_softc *sc, struct ccr_session *s, V_FW_CRYPTO_LOOKASIDE_WR_LEN16(wr_len / 16)); crwr->wreq.session_id = 0; crwr->wreq.rx_chid_to_rx_q_id = htobe32( - V_FW_CRYPTO_LOOKASIDE_WR_RX_CHID(s->port->tx_channel_id) | + V_FW_CRYPTO_LOOKASIDE_WR_RX_CHID(s->port->rx_channel_id) | V_FW_CRYPTO_LOOKASIDE_WR_LCB(0) | V_FW_CRYPTO_LOOKASIDE_WR_PHASH(0) | V_FW_CRYPTO_LOOKASIDE_WR_IV(IV_NOP) | V_FW_CRYPTO_LOOKASIDE_WR_FQIDX(0) | - V_FW_CRYPTO_LOOKASIDE_WR_TX_CH(0) | + V_FW_CRYPTO_LOOKASIDE_WR_TX_CH(0) | /* unused in firmware */ V_FW_CRYPTO_LOOKASIDE_WR_RX_Q_ID(s->port->rxq->iq.abs_id)); crwr->wreq.key_addr = 0; crwr->wreq.pld_size_hash_size = htobe32( @@ -440,7 +443,7 @@ ccr_populate_wreq(struct ccr_softc *sc, struct ccr_session *s, V_ULP_TXPKT_DATAMODIFY(0) | V_ULP_TXPKT_CHANNELID(s->port->tx_channel_id) | V_ULP_TXPKT_DEST(0) | - V_ULP_TXPKT_FID(s->port->rxq->iq.abs_id) | V_ULP_TXPKT_RO(1)); + V_ULP_TXPKT_FID(sc->first_rxq_id) | V_ULP_TXPKT_RO(1)); crwr->ulptx.len = htobe32( ((wr_len - sizeof(struct fw_crypto_lookaside_wr)) / 16)); @@ -525,7 +528,7 @@ ccr_hash(struct ccr_softc *sc, struct ccr_session *s, struct cryptop *crp) crwr->sec_cpl.op_ivinsrtofst = htobe32( V_CPL_TX_SEC_PDU_OPCODE(CPL_TX_SEC_PDU) | - V_CPL_TX_SEC_PDU_RXCHID(s->port->tx_channel_id) | + V_CPL_TX_SEC_PDU_RXCHID(s->port->rx_channel_id) | V_CPL_TX_SEC_PDU_ACKFOLLOWS(0) | V_CPL_TX_SEC_PDU_ULPTXLPBK(1) | V_CPL_TX_SEC_PDU_CPLLEN(2) | V_CPL_TX_SEC_PDU_PLACEHOLDER(0) | V_CPL_TX_SEC_PDU_IVINSRTOFST(0)); @@ -685,7 +688,7 @@ ccr_blkcipher(struct ccr_softc *sc, struct ccr_session *s, struct cryptop *crp) crwr->sec_cpl.op_ivinsrtofst = htobe32( V_CPL_TX_SEC_PDU_OPCODE(CPL_TX_SEC_PDU) | - V_CPL_TX_SEC_PDU_RXCHID(s->port->tx_channel_id) | + V_CPL_TX_SEC_PDU_RXCHID(s->port->rx_channel_id) | V_CPL_TX_SEC_PDU_ACKFOLLOWS(0) | V_CPL_TX_SEC_PDU_ULPTXLPBK(1) | V_CPL_TX_SEC_PDU_CPLLEN(2) | V_CPL_TX_SEC_PDU_PLACEHOLDER(0) | V_CPL_TX_SEC_PDU_IVINSRTOFST(1)); @@ -986,7 +989,7 @@ ccr_eta(struct ccr_softc *sc, struct ccr_session *s, struct cryptop *crp) crwr->sec_cpl.op_ivinsrtofst = htobe32( V_CPL_TX_SEC_PDU_OPCODE(CPL_TX_SEC_PDU) | - V_CPL_TX_SEC_PDU_RXCHID(s->port->tx_channel_id) | + V_CPL_TX_SEC_PDU_RXCHID(s->port->rx_channel_id) | V_CPL_TX_SEC_PDU_ACKFOLLOWS(0) | V_CPL_TX_SEC_PDU_ULPTXLPBK(1) | V_CPL_TX_SEC_PDU_CPLLEN(2) | V_CPL_TX_SEC_PDU_PLACEHOLDER(0) | V_CPL_TX_SEC_PDU_IVINSRTOFST(1)); @@ -1293,7 +1296,7 @@ ccr_gcm(struct ccr_softc *sc, struct ccr_session *s, struct cryptop *crp) crwr->sec_cpl.op_ivinsrtofst = htobe32( V_CPL_TX_SEC_PDU_OPCODE(CPL_TX_SEC_PDU) | - V_CPL_TX_SEC_PDU_RXCHID(s->port->tx_channel_id) | + V_CPL_TX_SEC_PDU_RXCHID(s->port->rx_channel_id) | V_CPL_TX_SEC_PDU_ACKFOLLOWS(0) | V_CPL_TX_SEC_PDU_ULPTXLPBK(1) | V_CPL_TX_SEC_PDU_CPLLEN(2) | V_CPL_TX_SEC_PDU_PLACEHOLDER(0) | V_CPL_TX_SEC_PDU_IVINSRTOFST(1)); @@ -1768,7 +1771,7 @@ ccr_ccm(struct ccr_softc *sc, struct ccr_session *s, struct cryptop *crp) crwr->sec_cpl.op_ivinsrtofst = htobe32( V_CPL_TX_SEC_PDU_OPCODE(CPL_TX_SEC_PDU) | - V_CPL_TX_SEC_PDU_RXCHID(s->port->tx_channel_id) | + V_CPL_TX_SEC_PDU_RXCHID(s->port->rx_channel_id) | V_CPL_TX_SEC_PDU_ACKFOLLOWS(0) | V_CPL_TX_SEC_PDU_ULPTXLPBK(1) | V_CPL_TX_SEC_PDU_CPLLEN(2) | V_CPL_TX_SEC_PDU_PLACEHOLDER(0) | V_CPL_TX_SEC_PDU_IVINSRTOFST(1)); @@ -2131,11 +2134,13 @@ ccr_sysctls(struct ccr_softc *sc) static void ccr_init_port(struct ccr_softc *sc, int port) { + struct port_info *pi; + pi = sc->adapter->port[port]; sc->ports[port].txq = &sc->adapter->sge.ctrlq[port]; - sc->ports[port].rxq = - &sc->adapter->sge.rxq[sc->adapter->port[port]->vi->first_rxq]; - sc->ports[port].tx_channel_id = port; + sc->ports[port].rxq = &sc->adapter->sge.rxq[pi->vi->first_rxq]; + sc->ports[port].rx_channel_id = pi->rx_c_chan; + sc->ports[port].tx_channel_id = pi->tx_chan; _Static_assert(sizeof(sc->port_mask) * NBBY >= MAX_NPORTS - 1, "Too many ports to fit in port_mask"); sc->port_mask |= 1u << port; @@ -2163,6 +2168,12 @@ ccr_attach(device_t dev) sc->cid = cid; sc->adapter->ccr_softc = sc; + /* + * The FID must be the first RXQ for port 0 regardless of + * which port is used to service the request. + */ + sc->first_rxq_id = sc->adapter->sge.rxq[0].iq.abs_id; + mtx_init(&sc->lock, "ccr", NULL, MTX_DEF); sc->iv_aad_buf = malloc(MAX_AAD_LEN, M_CCR, M_WAITOK); sc->sg_iv_aad = sglist_build(sc->iv_aad_buf, MAX_AAD_LEN, M_WAITOK);