From nobody Sun Jan 14 22:08:01 2024 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4TCqCP4Mgzz57G5d; Sun, 14 Jan 2024 22:08:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4TCqCP2nGJz4K7p; Sun, 14 Jan 2024 22:08:01 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1705270081; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yF/D6jH9qepvFX0gdtie+rbtHm68+ozgPsUvDG3eZxA=; b=skrMELLhZtMas0Hd+rW01ky12Hze2Or6Q3iOMJSrBAEIS+2if9BtE1aMNdhYREsN2RlPB1 dXWxVUQcDbjNMTrPtQcQUirDVWe/ijJy708nzLfoE6Gtx4uPBI1BrUKsbxcIPmhzDLKTtF aQGih88tXQ5ryLspHAAe+aH94EyTdTSqYmlZdGuMBD4rdrxJo522fhbB2PHuq/TcTHYazf J7U60qA3W7H4UmC5VTWBAMGvJLMZx3OWZj3WvbrLqqRZTgfhZGVVYfPNKLKl3KvQhTf78g W3hMU0r8aX2j4FqUh2Y+EZfji6PXFUXt1p0xBAxMGWPeuePpfYnBVw8PdI2cdw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1705270081; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=yF/D6jH9qepvFX0gdtie+rbtHm68+ozgPsUvDG3eZxA=; b=FO6915uyGR3mqKNRfL8V/LDOZsvziBZ5fHx9mJxzTkhSg+tyTWuH7PUM8EW9a6K6L2Gv9B CKJCrwqSF/HZk2Ag+Kq8idydg+RuPnf3mwAGavjwOibjffocUBoIUnIfgzdSIVSwiSe7OM A4eMLqzUT7vVsH8B3r0izHx3Ka1YrnoHC2Vxrn9R1BhQImSRYOH8+3VfCZI/fGC58qkkRH yH8mCJCNuz1MJRk7LNm2j5yQDWCxiB/n6VFup+3TUv9idvJi7NwyEKvCjlYU7av+OHqM/a jT5IvWBFA4e1W4QsWDnhFllGo3wIsa3j2lFxNb+NXCz3CnOxGk88Bg/vay3mrw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1705270081; a=rsa-sha256; cv=none; b=GQ7SHSN5o42UtcYBEpdRaYaou0u9HyLqwSLL7hhGYlLSlLpKKnXNfI0YZExkzTQDjzHDu9 DmMfKKMdXOlTkNq/Sfa77zmLPjYQFvcoBhyXHR1JtlyJjb2/l5wA5R4P1haXQNbK4fFBqU +LFFe0PpMg7WphNfJRGZqogZKjUoz97MEQtvJbrX0Qj7/9HjrhkAXfZt9KTuvaCt1yWVj1 q//vPIGGe5MIBTTfLP5IuYqDP8XZmvbBewHNEDWpRg8lrtFv+7R/nfwyyy5tBkQyuek+8q WrGh5BIksPh8htNWWhl2clGeiGyzj/BcIcwPYlTw/vX2M3tXIgBWw8t33yJkVQ== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 4TCqCP1RHLzXB9; Sun, 14 Jan 2024 22:08:01 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.17.1/8.17.1) with ESMTP id 40EM81WI060348; Sun, 14 Jan 2024 22:08:01 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.17.1/8.17.1/Submit) id 40EM81E2060345; Sun, 14 Jan 2024 22:08:01 GMT (envelope-from git) Date: Sun, 14 Jan 2024 22:08:01 GMT Message-Id: <202401142208.40EM81E2060345@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Arthur Kiyanovski Subject: git: be6292777476 - stable/13 - ena: Support srd metrics with sysctl List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: akiyano X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: be62927774761ff99dda241e6b3e3e0d8b91fb05 Auto-Submitted: auto-generated The branch stable/13 has been updated by akiyano: URL: https://cgit.FreeBSD.org/src/commit/?id=be62927774761ff99dda241e6b3e3e0d8b91fb05 commit be62927774761ff99dda241e6b3e3e0d8b91fb05 Author: Osama Abboud AuthorDate: 2023-09-12 11:17:45 +0000 Commit: Arthur Kiyanovski CommitDate: 2024-01-14 07:43:23 +0000 ena: Support srd metrics with sysctl This commit introduces SRD metrics through sysctl. The metrics can be queried using the following sysctl node: sysctl dev.ena..ena_srd_info Approved by: cperciva (mentor) MFC after: 2 weeks Sponsored by: Amazon, Inc. (cherry picked from commit 36d42c862f4a5643f6e2395e8d7b7e5c4580499a) --- sys/dev/ena/ena.c | 10 +++++++ sys/dev/ena/ena.h | 1 + sys/dev/ena/ena_sysctl.c | 76 +++++++++++++++++++++++++++++++++++++++++++++++- 3 files changed, 86 insertions(+), 1 deletion(-) diff --git a/sys/dev/ena/ena.c b/sys/dev/ena/ena.c index 72db1a9d99cb..a61635aa8c80 100644 --- a/sys/dev/ena/ena.c +++ b/sys/dev/ena/ena.c @@ -166,6 +166,7 @@ static int ena_enable_msix_and_set_admin_interrupts(struct ena_adapter *); static void ena_update_on_link_change(void *, struct ena_admin_aenq_entry *); static void unimplemented_aenq_handler(void *, struct ena_admin_aenq_entry *); static int ena_copy_eni_metrics(struct ena_adapter *); +static int ena_copy_srd_metrics(struct ena_adapter *); static int ena_copy_customer_metrics(struct ena_adapter *); static void ena_timer_service(void *); @@ -3307,6 +3308,12 @@ ena_copy_eni_metrics(struct ena_adapter *adapter) return (rc); } +static int +ena_copy_srd_metrics(struct ena_adapter *adapter) +{ + return ena_com_get_ena_srd_info(adapter->ena_dev, &adapter->ena_srd_info); +} + static int ena_copy_customer_metrics(struct ena_adapter *adapter) { @@ -3569,6 +3576,9 @@ ena_metrics_task(void *arg, int pending) else if (ena_com_get_cap(adapter->ena_dev, ENA_ADMIN_ENI_STATS)) (void)ena_copy_eni_metrics(adapter); + if (ena_com_get_cap(adapter->ena_dev, ENA_ADMIN_ENA_SRD_INFO)) + (void)ena_copy_srd_metrics(adapter); + ENA_LOCK_UNLOCK(); } diff --git a/sys/dev/ena/ena.h b/sys/dev/ena/ena.h index 18988d719a18..5647fe973502 100644 --- a/sys/dev/ena/ena.h +++ b/sys/dev/ena/ena.h @@ -487,6 +487,7 @@ struct ena_adapter { struct ena_stats_dev dev_stats; struct ena_hw_stats hw_stats; struct ena_admin_eni_stats eni_metrics; + struct ena_admin_ena_srd_info ena_srd_info; uint64_t *customer_metrics_array; enum ena_regs_reset_reason_types reset_reason; diff --git a/sys/dev/ena/ena_sysctl.c b/sys/dev/ena/ena_sysctl.c index edae5c3c23fa..fd5545dbe5fb 100644 --- a/sys/dev/ena/ena_sysctl.c +++ b/sys/dev/ena/ena_sysctl.c @@ -38,6 +38,7 @@ static void ena_sysctl_add_wd(struct ena_adapter *); static void ena_sysctl_add_stats(struct ena_adapter *); static void ena_sysctl_add_eni_metrics(struct ena_adapter *); static void ena_sysctl_add_customer_metrics(struct ena_adapter *); +static void ena_sysctl_add_srd_info(struct ena_adapter *); static void ena_sysctl_add_tuneables(struct ena_adapter *); static void ena_sysctl_add_irq_affinity(struct ena_adapter *); /* Kernel option RSS prevents manipulation of key hash and indirection table. */ @@ -59,18 +60,46 @@ static int ena_sysctl_rss_indir_table(SYSCTL_HANDLER_ARGS); #define ENA_METRICS_MAX_SAMPLE_INTERVAL 3600 #define ENA_HASH_KEY_MSG_SIZE (ENA_HASH_KEY_SIZE * 2 + 1) -#define SYSCTL_GSTRING_LEN 64 +#define SYSCTL_GSTRING_LEN 128 #define ENA_METRIC_ENI_ENTRY(stat, desc) { \ .name = #stat, \ .description = #desc, \ } +#define ENA_STAT_ENTRY(stat, desc, stat_type) { \ + .name = #stat, \ + .description = #desc, \ + .stat_offset = offsetof(struct ena_admin_##stat_type, stat) / sizeof(u64), \ +} + +#define ENA_STAT_ENA_SRD_ENTRY(stat, desc) \ + ENA_STAT_ENTRY(stat, desc, ena_srd_stats) + struct ena_hw_metrics { char name[SYSCTL_GSTRING_LEN]; char description[SYSCTL_GSTRING_LEN]; }; +struct ena_srd_metrics { + char name[SYSCTL_GSTRING_LEN]; + char description[SYSCTL_GSTRING_LEN]; + int stat_offset; +}; + +static const struct ena_srd_metrics ena_srd_stats_strings[] = { + ENA_STAT_ENA_SRD_ENTRY( + ena_srd_tx_pkts, Number of packets transmitted over ENA SRD), + ENA_STAT_ENA_SRD_ENTRY( + ena_srd_eligible_tx_pkts, Number of packets transmitted or could + have been transmitted over ENA SRD), + ENA_STAT_ENA_SRD_ENTRY( + ena_srd_rx_pkts, Number of packets received over ENA SRD), + ENA_STAT_ENA_SRD_ENTRY( + ena_srd_resource_utilization, Percentage of the ENA SRD resources + that are in use), +}; + static const struct ena_hw_metrics ena_hw_stats_strings[] = { ENA_METRIC_ENI_ENTRY( bw_in_allowance_exceeded, Inbound BW allowance exceeded), @@ -91,6 +120,7 @@ static const struct ena_hw_metrics ena_hw_stats_strings[] = { #endif #define ENA_CUSTOMER_METRICS_ARRAY_SIZE ARRAY_SIZE(ena_hw_stats_strings) +#define ENA_SRD_METRICS_ARRAY_SIZE ARRAY_SIZE(ena_srd_stats_strings) static SYSCTL_NODE(_hw, OID_AUTO, ena, CTLFLAG_RD | CTLFLAG_MPSAFE, 0, "ENA driver parameters"); @@ -153,6 +183,9 @@ ena_sysctl_add_nodes(struct ena_adapter *adapter) else if (ena_com_get_cap(dev, ENA_ADMIN_ENI_STATS)) ena_sysctl_add_eni_metrics(adapter); + if (ena_com_get_cap(adapter->ena_dev, ENA_ADMIN_ENA_SRD_INFO)) + ena_sysctl_add_srd_info(adapter); + ena_sysctl_add_wd(adapter); ena_sysctl_add_stats(adapter); ena_sysctl_add_tuneables(adapter); @@ -379,6 +412,47 @@ ena_sysctl_add_stats(struct ena_adapter *adapter) &admin_stats->no_completion, 0, "Commands not completed"); } +static void +ena_sysctl_add_srd_info(struct ena_adapter *adapter) +{ + device_t dev; + + struct sysctl_oid *ena_srd_info; + struct sysctl_oid_list *srd_list; + + struct sysctl_ctx_list *ctx; + struct sysctl_oid *tree; + struct sysctl_oid_list *child; + + struct ena_admin_ena_srd_stats *srd_stats_ptr; + struct ena_srd_metrics cur_stat_strings; + + int i; + + dev = adapter->pdev; + + ctx = device_get_sysctl_ctx(dev); + tree = device_get_sysctl_tree(dev); + child = SYSCTL_CHILDREN(tree); + + ena_srd_info = SYSCTL_ADD_NODE(ctx, child, OID_AUTO, "ena_srd_info", + CTLFLAG_RD | CTLFLAG_MPSAFE, NULL, "ENA's SRD information"); + srd_list = SYSCTL_CHILDREN(ena_srd_info); + + SYSCTL_ADD_U64(ctx, srd_list, OID_AUTO, "ena_srd_mode", + CTLFLAG_RD, &adapter->ena_srd_info.flags, 0, + "Describes which ENA-express features are enabled"); + + srd_stats_ptr = &adapter->ena_srd_info.ena_srd_stats; + + for (i = 0 ; i < ENA_SRD_METRICS_ARRAY_SIZE; i++) { + cur_stat_strings = ena_srd_stats_strings[i]; + SYSCTL_ADD_U64(ctx, srd_list, OID_AUTO, cur_stat_strings.name, + CTLFLAG_RD, (u64 *)srd_stats_ptr + cur_stat_strings.stat_offset, + 0, cur_stat_strings.description); + } +} + static void ena_sysctl_add_customer_metrics(struct ena_adapter *adapter) {