From nobody Mon Sep 18 10:38:27 2023 X-Original-To: dev-commits-src-branches@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4Rq1VD0WZMz4syPh; Mon, 18 Sep 2023 10:38:28 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Rq1VC3hMCz3Xdk; Mon, 18 Sep 2023 10:38:27 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695033507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hV/yXuEzAwdD+wXVZVJ1lZrfm595LMDnhnT7szjdRC0=; b=tgopWm6J3dzC0+08G8NyNSbDE9xqbw2iPKeBQ/z12ZUx8kmepz5C72w+s2miWuhcBN4BYu vxDg+15lmU0+83YWRSbSEH0UrpsgdyJcmlkyZSgpde7asrPVdE20Epajdz1UkWV6lx1qZr kBdo21+6DbNkXB97tPcse2YaOvPAjlQSYj2DJKCsVtQ0KwlqeQDcKHIDLx8tAowIIoKGz+ xqaDk2IYx4Mq34g1DNwb2PYXC5IgB1XDfDf5TG4KLgrnmvuiFoQWpjx1ASHTVjCoEAikiq i1c50w0vcFzs+sH81MOVSlMtGHvQjQtP2DM/tGWsmc5RSgNyfR02tJD1i6ZvXA== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1695033507; a=rsa-sha256; cv=none; b=DI20mGp6aPtKZtv3+4cVlX6oOYGbLc6Z+i+HmcP4bmhZQEhohqeb26HI1gKTm4bCyfdDQ3 47MNZLtqXNHhUFDu9XBkI9RrCHPJmmFCNds694GMn8TeEy/9VY+jgp+EMdv1sWAxH9RtQE WDWGY6Ez3InCoBd6dvY1kdo/XZWwfsH/oIi9yG9SjXjAE0dk7I2YlDf6f11BtAMxWwdpCd 9ae1iedeE0D8eob5NQGoCHAQpioRIKIwmry0H0x/8qTydreXGXx4gLVfHSBpb9Fv8QXKH4 4fUF57EckCseCuF9Oq667CdW560V0fniXdi8F1l0uNDEiVXbizKXP6riyRQ32g== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1695033507; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=hV/yXuEzAwdD+wXVZVJ1lZrfm595LMDnhnT7szjdRC0=; b=giSbiG26tQXQZHvwvPPjOOXypZZA3HEXC56Iaoarl+5e9aoSXcfdHvfkPkUJYOMMd6qR2h 10/nwQkCtQERb0gb+e34WVzEJfgqWKW+7jDR9lWHONO1F3q9OyKAEPbWZcPJ9BvAcD44gU CZoLi1oXFcDJIr8FgZ/P5+9oiaK322NPbDejRXu+j56DmSbhCNH90cNNbo/dC5Ln6DTTAV 4frx1+JcCnW7LFuRQOt8dif740NdZxiYTQ2RpNNP4z9K08DVK6OE+TlXqIfLOuYNpu5CF5 JsCB67G3IwQL01rLK4auKc1GGJ6uSf9nEYekRqh4OFVY10DYTHoCtqITH9RkXw== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4Rq1VC2bgFzCll; Mon, 18 Sep 2023 10:38:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 38IAcRfU013942; Mon, 18 Sep 2023 10:38:27 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 38IAcRw0013939; Mon, 18 Sep 2023 10:38:27 GMT (envelope-from git) Date: Mon, 18 Sep 2023 10:38:27 GMT Message-Id: <202309181038.38IAcRw0013939@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Wei Hu Subject: git: a72a0af8194e - stable/14 - mana: add lro and tso stat counters List-Id: Commits to the stable branches of the FreeBSD src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-branches List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-branches@freebsd.org X-BeenThere: dev-commits-src-branches@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: whu X-Git-Repository: src X-Git-Refname: refs/heads/stable/14 X-Git-Reftype: branch X-Git-Commit: a72a0af8194effa5f7e7a88d09673ed798819e88 Auto-Submitted: auto-generated The branch stable/14 has been updated by whu: URL: https://cgit.FreeBSD.org/src/commit/?id=a72a0af8194effa5f7e7a88d09673ed798819e88 commit a72a0af8194effa5f7e7a88d09673ed798819e88 Author: Wei Hu AuthorDate: 2023-09-14 11:56:20 +0000 Commit: Wei Hu CommitDate: 2023-09-18 10:27:46 +0000 mana: add lro and tso stat counters Add a few stat counters for tso and lro. MFC after: 3 days Sponsored by: Microsoft (cherry picked from commit b167e449c8db01f082691503fb5c1255ad5750eb) --- sys/dev/mana/mana.h | 5 ++ sys/dev/mana/mana_en.c | 12 ++++ sys/dev/mana/mana_sysctl.c | 136 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 153 insertions(+) diff --git a/sys/dev/mana/mana.h b/sys/dev/mana/mana.h index 9a22ea87baa3..ab394f3203ad 100644 --- a/sys/dev/mana/mana.h +++ b/sys/dev/mana/mana.h @@ -170,6 +170,9 @@ struct mana_txq { struct mtx txq_mtx; char txq_mtx_name[16]; + uint64_t tso_pkts; + uint64_t tso_bytes; + struct task enqueue_task; struct taskqueue *enqueue_tq; @@ -423,6 +426,8 @@ struct mana_rxq { uint32_t buf_index; + uint64_t lro_tried; + uint64_t lro_failed; struct mana_stats stats; /* MUST BE THE LAST MEMBER: diff --git a/sys/dev/mana/mana_en.c b/sys/dev/mana/mana_en.c index 064b28fa94a4..812a42e51dfc 100644 --- a/sys/dev/mana/mana_en.c +++ b/sys/dev/mana/mana_en.c @@ -501,6 +501,7 @@ mana_xmit(struct mana_txq *txq) struct gdma_queue *gdma_sq; struct mana_cq *cq; int err, len; + bool is_tso; gdma_sq = txq->gdma_sq; cq = &apc->tx_qp[txq->idx].tx_cq; @@ -578,7 +579,10 @@ mana_xmit(struct mana_txq *txq) pkg.wqe_req.flags = 0; pkg.wqe_req.client_data_unit = 0; + is_tso = false; if (mbuf->m_pkthdr.csum_flags & CSUM_TSO) { + is_tso = true; + if (MANA_L3_PROTO(mbuf) == ETHERTYPE_IP) pkg.tx_oob.s_oob.is_outer_ipv4 = 1; else @@ -641,6 +645,11 @@ mana_xmit(struct mana_txq *txq) packets++; bytes += len; + + if (is_tso) { + txq->tso_pkts++; + txq->tso_bytes += len; + } } counter_enter(); @@ -1697,9 +1706,12 @@ mana_rx_mbuf(struct mbuf *mbuf, struct mana_rxcomp_oob *cqe, do_if_input = true; if ((if_getcapenable(ndev) & IFCAP_LRO) && do_lro) { + rxq->lro_tried++; if (rxq->lro.lro_cnt != 0 && tcp_lro_rx(&rxq->lro, mbuf, 0) == 0) do_if_input = false; + else + rxq->lro_failed++; } if (do_if_input) { if_input(ndev, mbuf); diff --git a/sys/dev/mana/mana_sysctl.c b/sys/dev/mana/mana_sysctl.c index 79a4867a0abf..d03ebe44ab31 100644 --- a/sys/dev/mana/mana_sysctl.c +++ b/sys/dev/mana/mana_sysctl.c @@ -46,6 +46,96 @@ SYSCTL_INT(_hw_mana, OID_AUTO, log_level, CTLFLAG_RWTUN, SYSCTL_CONST_STRING(_hw_mana, OID_AUTO, driver_version, CTLFLAG_RD, DRV_MODULE_VERSION, "MANA driver version"); +static int +mana_sysctl_rx_stat_agg_u64(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, i, err; + struct mana_rxq *rxq; + uint64_t stat; + + stat = 0; + for (i = 0; i < apc->num_queues; i++) { + rxq = apc->rxqs[i]; + stat += *((uint64_t *)((uint8_t *)rxq + offset)); + } + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + + for (i = 0; i < apc->num_queues; i++) { + rxq = apc->rxqs[i]; + *((uint64_t *)((uint8_t *)rxq + offset)) = 0; + } + return 0; +} + +static int +mana_sysctl_rx_stat_u16(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, err; + struct mana_rxq *rxq; + uint64_t stat; + uint16_t val; + + rxq = apc->rxqs[0]; + val = *((uint16_t *)((uint8_t *)rxq + offset)); + stat = val; + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + else + return 0; +} + +static int +mana_sysctl_rx_stat_u32(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, err; + struct mana_rxq *rxq; + uint64_t stat; + uint32_t val; + + rxq = apc->rxqs[0]; + val = *((uint32_t *)((uint8_t *)rxq + offset)); + stat = val; + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + else + return 0; +} + +static int +mana_sysctl_tx_stat_agg_u64(SYSCTL_HANDLER_ARGS) +{ + struct mana_port_context *apc = arg1; + int offset = arg2, i, err; + struct mana_txq *txq; + uint64_t stat; + + stat = 0; + for (i = 0; i < apc->num_queues; i++) { + txq = &apc->tx_qp[i].txq; + stat += *((uint64_t *)((uint8_t *)txq + offset)); + } + + err = sysctl_handle_64(oidp, &stat, 0, req); + if (err || req->newptr == NULL) + return err; + + for (i = 0; i < apc->num_queues; i++) { + txq = &apc->tx_qp[i].txq; + *((uint64_t *)((uint8_t *)txq + offset)) = 0; + } + return 0; +} + void mana_sysctl_add_port(struct mana_port_context *apc) { @@ -99,6 +189,52 @@ mana_sysctl_add_port(struct mana_port_context *apc) CTLFLAG_RD, &port_stats->rx_drops, "Receive packet drops"); SYSCTL_ADD_COUNTER_U64(ctx, stats_list, OID_AUTO, "tx_drops", CTLFLAG_RD, &port_stats->tx_drops, "Transmit packet drops"); + + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_queued", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_queued), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO queued"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_flushed", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_flushed), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO flushed"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_bad_csum", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_MPSAFE | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_bad_csum), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO bad checksum"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_tried", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro_tried), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO tried"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "rx_lro_failed", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro_failed), + mana_sysctl_rx_stat_agg_u64, "LU", "LRO failed"); + + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "lro_ackcnt_lim", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_ackcnt_lim), + mana_sysctl_rx_stat_u16, + "LU", "Max # of ACKs to be aggregated by LRO"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "lro_length_lim", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_length_lim), + mana_sysctl_rx_stat_u32, + "LU", "Max len of aggregated data in byte by LRO"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "lro_cnt", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_rxq, lro.lro_cnt), + mana_sysctl_rx_stat_u32, + "LU", "Max # or LRO packet count"); + + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "tx_tso_packets", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_txq, tso_pkts), + mana_sysctl_tx_stat_agg_u64, "LU", "TSO packets"); + SYSCTL_ADD_PROC(ctx, stats_list, OID_AUTO, "tx_tso_bytes", + CTLTYPE_U64 | CTLFLAG_RD | CTLFLAG_STATS, apc, + __offsetof(struct mana_txq, tso_bytes), + mana_sysctl_tx_stat_agg_u64, "LU", "TSO bytes"); } void