From nobody Thu Jan 11 05:34:29 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T9YJP3mclz56VfH; Thu, 11 Jan 2024 05:34:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T9YJP1c3Mz47pF; Thu, 11 Jan 2024 05:34:29 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704951269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FVlkx0hXvihMja/WYDThAl5vS6ut7kxMs9R9r4zAuRo=; b=H7XgtTreu8zFnJ5EAjIbed7reiEGRSZzPTu0w36KiXmF0sYTm8pNFKcyEZMfC9QazPFmiV Mqs+e4F8tD5kSudQZfA5LCGwc/hSmvKIw/Ul9Kryo1vhWABtqzwa5UUUxfbYZyF1zK3EHK L3gsWK7LHzBVTFQSDo8ddiMBZC17TBgGMDcf4uQltB7nut0UEQMx6fxSayYXjqRfd24YT9 lvzRAmqVX1H8VPqrsZIJYhvxCxbe3UKG816dvZy5waBYK3PSBR16Ux6HR0EDWVcbhWRryT wUdt5vlR137T0pxVVbc1HQQjiULcMuSAjA6av2gk+uZLkw1tHMNU6jrr2jetyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704951269; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=FVlkx0hXvihMja/WYDThAl5vS6ut7kxMs9R9r4zAuRo=; b=QYEbGKoMstWktNlmrjpWgwLVsmMYCuqZRbt4dlrLhs0sN2mcVX+2YfuMZqQ2mcyRMUuIfu D7jgl1JR69WyhbAtfCejsbDthLLzlbi61nP324xIQkHgbl5MdlmU31wc+14luL477jw0if Rm8jDdRGkdpqKK90AvC/XgYMN5krIFHWjXSjDBZLMJ+6GoUPHF4O9ITvLnjMXAJ1Mfl1oI oShB0+o4be2czf7ZEp9QC2tXjiymDIcX143zIrcWhDAjxhpIWBksdjiD8XmLMt5WXd0UIR M7jm8L5+1oxBNp66o8XX2fQYOSDBEEIhIJHiU3naNBKAlPHF/OUyc9qaTXigzQ== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704951269; a=rsa-sha256; cv=none; b=oIPFhh/dz79P+wJEt5YKjJ9D1b5UZubkvu5vFU7tnEOh/KRKxp0whpRWNipSx++l8AUwDa hblFU9D+w04tRs/PhDoZUDeONKHzBWptdni9s8CTr6TIiyaCN730J/l/iFXlL3MgvdBVTa gsPSVmLejpnRo5xIm/EXw9OvTpPgCNfJNSD2ZMotwWtZdlbDTsteENdqtBUry4gcAzf4nX ffCX4pqjra9kAzmt0npuApOuTLgkV0cVoVVt9j+rYXzFtZuf1fUjv7/W6tGIlYPiipj3mz qsNA9/9jqbiwEGbPnJ9rS4Q7/mmNwJrSIqN+dKtF/qT0KFa18y68fISuTXiQxQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4T9YJP0jFxz15S8; Thu, 11 Jan 2024 05:34:29 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 40B5YT48009892; Thu, 11 Jan 2024 05:34:29 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 40B5YTtV009889; Thu, 11 Jan 2024 05:34:29 GMT (envelope-from git) Date: Thu, 11 Jan 2024 05:34:29 GMT Message-Id: <202401110534.40B5YTtV009889@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Navdeep Parhar Subject: git: 8b7e008183c9 - stable/13 - cxgbe/iw_cxgbe: debug routines to dump STAG (steering tag) entries. List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: np X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: 8b7e008183c94f8763c18390a9dc95c75b5f55b3 Auto-Submitted: auto-generated The branch stable/13 has been updated by np: URL: https://cgit.FreeBSD.org/src/commit/?id=8b7e008183c94f8763c18390a9dc95c75b5f55b3 commit 8b7e008183c94f8763c18390a9dc95c75b5f55b3 Author: Navdeep Parhar AuthorDate: 2023-04-06 17:52:15 +0000 Commit: Navdeep Parhar CommitDate: 2024-01-11 05:06:58 +0000 cxgbe/iw_cxgbe: debug routines to dump STAG (steering tag) entries. t4_dump_stag to dump hw state for a known STAG. t4_dump_all_stag to dump hw state for all valid STAGs. This routine walks the entire STAG region looking for valid entries and this can take a while for some configurations. Sponsored by: Chelsio Communications (cherry picked from commit 7adf138ba9b90db5ed0f1e8cd68e787fb63fa760) --- sys/dev/cxgbe/iw_cxgbe/ev.c | 82 ++++++++++++++++++++++++++++++++++++--- sys/dev/cxgbe/iw_cxgbe/iw_cxgbe.h | 2 + 2 files changed, 79 insertions(+), 5 deletions(-) diff --git a/sys/dev/cxgbe/iw_cxgbe/ev.c b/sys/dev/cxgbe/iw_cxgbe/ev.c index f163fb27ed44..7185bda1e2ed 100644 --- a/sys/dev/cxgbe/iw_cxgbe/ev.c +++ b/sys/dev/cxgbe/iw_cxgbe/ev.c @@ -39,6 +39,82 @@ #include "iw_cxgbe.h" +static void print_tpte(struct adapter *sc, const u32 stag, + const struct fw_ri_tpte *tpte) +{ + const __be64 *p = (const void *)tpte; + + CH_ERR(sc, "stag idx 0x%x valid %d key 0x%x state %d pdid %d " + "perm 0x%x ps %d len 0x%016llx va 0x%016llx\n", + stag & 0xffffff00, + G_FW_RI_TPTE_VALID(ntohl(tpte->valid_to_pdid)), + G_FW_RI_TPTE_STAGKEY(ntohl(tpte->valid_to_pdid)), + G_FW_RI_TPTE_STAGSTATE(ntohl(tpte->valid_to_pdid)), + G_FW_RI_TPTE_PDID(ntohl(tpte->valid_to_pdid)), + G_FW_RI_TPTE_PERM(ntohl(tpte->locread_to_qpid)), + G_FW_RI_TPTE_PS(ntohl(tpte->locread_to_qpid)), + (long long)(((u64)ntohl(tpte->len_hi) << 32) | ntohl(tpte->len_lo)), + (long long)(((u64)ntohl(tpte->va_hi) << 32) | ntohl(tpte->va_lo_fbo))); + CH_ERR(sc, "stag idx 0x%x: %016llx %016llx %016llx %016llx\n", + stag & 0xffffff00, + (long long)be64_to_cpu(p[0]), (long long)be64_to_cpu(p[1]), + (long long)be64_to_cpu(p[2]), (long long)be64_to_cpu(p[3])); +} + +void t4_dump_stag(struct adapter *sc, const u32 stag) +{ + struct fw_ri_tpte tpte __aligned(sizeof(__be64)) = {0}; + const u32 offset = sc->vres.stag.start + ((stag >> 8) * 32); + + if (offset > sc->vres.stag.start + sc->vres.stag.size - 32) { + CH_ERR(sc, "stag 0x%x is invalid for current configuration.\n", + stag); + return; + } + read_via_memwin(sc, 0, offset, (u32 *)&tpte, 32); + print_tpte(sc, stag, &tpte); +} + +void t4_dump_all_stag(struct adapter *sc) +{ + struct fw_ri_tpte tpte __aligned(sizeof(__be64)) = {0}; + const u32 first = sc->vres.stag.start; + const u32 last = first + sc->vres.stag.size - 32; + u32 offset, i; + + for (i = 0, offset = first; offset <= last; i++, offset += 32) { + tpte.valid_to_pdid = 0; + read_via_memwin(sc, 0, offset, (u32 *)&tpte, 4); + if (tpte.valid_to_pdid != 0) { + read_via_memwin(sc, 0, offset, (u32 *)&tpte, 32); + print_tpte(sc, i << 8, &tpte); + } + } +} + +static void dump_err_cqe(struct c4iw_dev *dev, struct t4_cqe *err_cqe) +{ + struct adapter *sc = dev->rdev.adap; + __be64 *p = (void *)err_cqe; + + CH_ERR(sc, "AE qpid 0x%x opcode %d status 0x%x " + "type %d wrid.hi 0x%x wrid.lo 0x%x\n", + CQE_QPID(err_cqe), CQE_OPCODE(err_cqe), + CQE_STATUS(err_cqe), CQE_TYPE(err_cqe), + CQE_WRID_HI(err_cqe), CQE_WRID_LOW(err_cqe)); + CH_ERR(sc, "%016llx %016llx %016llx %016llx\n", + (long long)be64_to_cpu(p[0]), (long long)be64_to_cpu(p[1]), + (long long)be64_to_cpu(p[2]), (long long)be64_to_cpu(p[3])); + + /* + * Ingress WRITE and READ_RESP errors provide + * the offending stag, so parse and log it. + */ + if (RQ_TYPE(err_cqe) && (CQE_OPCODE(err_cqe) == FW_RI_RDMA_WRITE || + CQE_OPCODE(err_cqe) == FW_RI_READ_RESP)) + t4_dump_stag(sc, CQE_WRID_STAG(err_cqe)); +} + static void post_qp_event(struct c4iw_dev *dev, struct c4iw_cq *chp, struct c4iw_qp *qhp, struct t4_cqe *err_cqe, @@ -56,11 +132,7 @@ static void post_qp_event(struct c4iw_dev *dev, struct c4iw_cq *chp, return; } - printf("AE qpid 0x%x opcode %d status 0x%x " - "type %d wrid.hi 0x%x wrid.lo 0x%x\n", - CQE_QPID(err_cqe), CQE_OPCODE(err_cqe), - CQE_STATUS(err_cqe), CQE_TYPE(err_cqe), - CQE_WRID_HI(err_cqe), CQE_WRID_LOW(err_cqe)); + dump_err_cqe(dev, err_cqe); if (qhp->attr.state == C4IW_QP_STATE_RTS) { attrs.next_state = C4IW_QP_STATE_TERMINATE; diff --git a/sys/dev/cxgbe/iw_cxgbe/iw_cxgbe.h b/sys/dev/cxgbe/iw_cxgbe/iw_cxgbe.h index 89b05fd0a2e3..2e6abf6171c0 100644 --- a/sys/dev/cxgbe/iw_cxgbe/iw_cxgbe.h +++ b/sys/dev/cxgbe/iw_cxgbe/iw_cxgbe.h @@ -984,4 +984,6 @@ u32 c4iw_get_qpid(struct c4iw_rdev *rdev, struct c4iw_dev_ucontext *uctx); void c4iw_put_qpid(struct c4iw_rdev *rdev, u32 qid, struct c4iw_dev_ucontext *uctx); void c4iw_ev_dispatch(struct c4iw_dev *dev, struct t4_cqe *err_cqe); +void t4_dump_stag(struct adapter *sc, const u32 stag); +void t4_dump_all_stag(struct adapter *sc); #endif