From owner-dev-commits-src-all@freebsd.org Wed Sep 29 03:24:25 2021 Return-Path: Delivered-To: dev-commits-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 4CBB7669DCF; Wed, 29 Sep 2021 03:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4HK1vF0Tmfz3qq6; Wed, 29 Sep 2021 03:24:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id B690610CCF; Wed, 29 Sep 2021 03:24:24 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 18T3OOmS075584; Wed, 29 Sep 2021 03:24:24 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 18T3OOEm075583; Wed, 29 Sep 2021 03:24:24 GMT (envelope-from git) Date: Wed, 29 Sep 2021 03:24:24 GMT Message-Id: <202109290324.18T3OOEm075583@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: Warner Losh Subject: git: 587aa25525e5 - main - nvme: count number of ignored interrupts MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 587aa25525e54ea775298c402acd7a647f9838fb Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commit messages for all branches of the src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Sep 2021 03:24:25 -0000 The branch main has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=587aa25525e54ea775298c402acd7a647f9838fb commit 587aa25525e54ea775298c402acd7a647f9838fb Author: Warner Losh AuthorDate: 2021-09-29 03:13:11 +0000 Commit: Warner Losh CommitDate: 2021-09-29 03:18:00 +0000 nvme: count number of ignored interrupts Count the number of times we're asked to process completions, but that we ignore because the state of the qpair isn't in RECOVERY_NONE. Sponsored by: Netflix Reviewed by: mav, chuck Differential Revision: https://reviews.freebsd.org/D32212 --- sys/dev/nvme/nvme_private.h | 1 + sys/dev/nvme/nvme_qpair.c | 5 ++++- sys/dev/nvme/nvme_sysctl.c | 24 ++++++++++++++++++++++++ 3 files changed, 29 insertions(+), 1 deletion(-) diff --git a/sys/dev/nvme/nvme_private.h b/sys/dev/nvme/nvme_private.h index 02cecaf03a97..b00130910a82 100644 --- a/sys/dev/nvme/nvme_private.h +++ b/sys/dev/nvme/nvme_private.h @@ -195,6 +195,7 @@ struct nvme_qpair { int64_t num_intr_handler_calls; int64_t num_retries; int64_t num_failures; + int64_t num_ignored; struct nvme_command *cmd; struct nvme_completion *cpl; diff --git a/sys/dev/nvme/nvme_qpair.c b/sys/dev/nvme/nvme_qpair.c index 827054efd48e..788322092f88 100644 --- a/sys/dev/nvme/nvme_qpair.c +++ b/sys/dev/nvme/nvme_qpair.c @@ -544,8 +544,10 @@ nvme_qpair_process_completions(struct nvme_qpair *qpair) * as soon as initialization is complete and we start sending commands * to the device. */ - if (qpair->recovery_state != RECOVERY_NONE) + if (qpair->recovery_state != RECOVERY_NONE) { + qpair->num_ignored++; return (false); + } /* * Sanity check initialization. After we reset the hardware, the phase @@ -746,6 +748,7 @@ nvme_qpair_construct(struct nvme_qpair *qpair, qpair->num_intr_handler_calls = 0; qpair->num_retries = 0; qpair->num_failures = 0; + qpair->num_ignored = 0; qpair->cmd = (struct nvme_command *)queuemem; qpair->cpl = (struct nvme_completion *)(queuemem + cmdsz); prpmem = (uint8_t *)(queuemem + cmdsz + cplsz); diff --git a/sys/dev/nvme/nvme_sysctl.c b/sys/dev/nvme/nvme_sysctl.c index 58db38dc373d..cfcc814f44d2 100644 --- a/sys/dev/nvme/nvme_sysctl.c +++ b/sys/dev/nvme/nvme_sysctl.c @@ -164,6 +164,7 @@ nvme_qpair_reset_stats(struct nvme_qpair *qpair) qpair->num_intr_handler_calls = 1; qpair->num_retries = 0; qpair->num_failures = 0; + qpair->num_ignored = 0; } static int @@ -226,6 +227,21 @@ nvme_sysctl_num_failures(SYSCTL_HANDLER_ARGS) return (sysctl_handle_64(oidp, &num_failures, 0, req)); } +static int +nvme_sysctl_num_ignored(SYSCTL_HANDLER_ARGS) +{ + struct nvme_controller *ctrlr = arg1; + int64_t num_ignored = 0; + int i; + + num_ignored = ctrlr->adminq.num_ignored; + + for (i = 0; i < ctrlr->num_io_queues; i++) + num_ignored += ctrlr->ioq[i].num_ignored; + + return (sysctl_handle_64(oidp, &num_ignored, 0, req)); +} + static int nvme_sysctl_reset_stats(SYSCTL_HANDLER_ARGS) { @@ -281,6 +297,9 @@ nvme_sysctl_initialize_queue(struct nvme_qpair *qpair, SYSCTL_ADD_QUAD(ctrlr_ctx, que_list, OID_AUTO, "num_failures", CTLFLAG_RD, &qpair->num_failures, "Number of commands ending in failure after all retries"); + SYSCTL_ADD_QUAD(ctrlr_ctx, que_list, OID_AUTO, "num_ignored", + CTLFLAG_RD, &qpair->num_ignored, + "Number of interrupts posted, but were administratively ignored"); SYSCTL_ADD_PROC(ctrlr_ctx, que_list, OID_AUTO, "dump_debug", CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_MPSAFE, @@ -344,6 +363,11 @@ nvme_sysctl_initialize_ctrlr(struct nvme_controller *ctrlr) ctrlr, 0, nvme_sysctl_num_failures, "IU", "Number of commands ending in failure after all retries"); + SYSCTL_ADD_PROC(ctrlr_ctx, ctrlr_list, OID_AUTO, + "num_ignored", CTLTYPE_S64 | CTLFLAG_RD | CTLFLAG_MPSAFE, + ctrlr, 0, nvme_sysctl_num_ignored, "IU", + "Number of interrupts ignored administratively"); + SYSCTL_ADD_PROC(ctrlr_ctx, ctrlr_list, OID_AUTO, "reset_stats", CTLTYPE_UINT | CTLFLAG_RW | CTLFLAG_MPSAFE, ctrlr, 0, nvme_sysctl_reset_stats, "IU", "Reset statistics to zero");