From owner-svn-src-head@freebsd.org Mon Jun 29 00:32:44 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C0211359572; Mon, 29 Jun 2020 00:32:43 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49w7k15gXcz43dq; Mon, 29 Jun 2020 00:32:41 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id F0B8E1EC6C; Mon, 29 Jun 2020 00:32:01 +0000 (UTC) (envelope-from chuck@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 05T0W1Sp049873; Mon, 29 Jun 2020 00:32:01 GMT (envelope-from chuck@FreeBSD.org) Received: (from chuck@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 05T0W1bm049872; Mon, 29 Jun 2020 00:32:01 GMT (envelope-from chuck@FreeBSD.org) Message-Id: <202006290032.05T0W1bm049872@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: chuck set sender to chuck@FreeBSD.org using -f From: Chuck Tuffli Date: Mon, 29 Jun 2020 00:32:01 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r362759 - head/usr.sbin/bhyve X-SVN-Group: head X-SVN-Commit-Author: chuck X-SVN-Commit-Paths: head/usr.sbin/bhyve X-SVN-Commit-Revision: 362759 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2020 00:32:44 -0000 Author: chuck Date: Mon Jun 29 00:32:01 2020 New Revision: 362759 URL: https://svnweb.freebsd.org/changeset/base/362759 Log: bhyve: implement NVMe SMART data I/O statistics SMART data in NVMe includes statistics for number of read and write commands issued as well as the number of "data units" read and written. NVMe defines "data unit" as thousands of 512 byte blocks (e.g. 1 data unit is 1-1,000 512 byte blocks, 3 data units are 2,001-3,000 512 byte blocks). This patch implements counters for: - Data Units Read - Data Units Written - Host Read Commands - Host Write Commands and exposes the values when the guest reads the SMART/Health Log Page. Fixes UNH Test 1.3.8 Tested by: Jason Tubnor MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D24894 Modified: head/usr.sbin/bhyve/pci_nvme.c Modified: head/usr.sbin/bhyve/pci_nvme.c ============================================================================== --- head/usr.sbin/bhyve/pci_nvme.c Mon Jun 29 00:31:58 2020 (r362758) +++ head/usr.sbin/bhyve/pci_nvme.c Mon Jun 29 00:32:01 2020 (r362759) @@ -218,6 +218,7 @@ struct pci_nvme_ioreq { uint64_t prev_gpaddr; size_t prev_size; + size_t bytes; struct blockif_req io_req; @@ -287,6 +288,14 @@ struct pci_nvme_softc { struct nvme_feature_obj feat[NVME_FID_MAX]; enum nvme_dsm_type dataset_management; + + /* Accounting for SMART data */ + __uint128_t read_data_units; + __uint128_t write_data_units; + __uint128_t read_commands; + __uint128_t write_commands; + uint32_t read_dunits_remainder; + uint32_t write_dunits_remainder; }; @@ -576,6 +585,10 @@ pci_nvme_init_logpages(struct pci_nvme_softc *sc) memset(&sc->err_log, 0, sizeof(sc->err_log)); memset(&sc->health_log, 0, sizeof(sc->health_log)); memset(&sc->fw_log, 0, sizeof(sc->fw_log)); + + /* Set read/write remainder to round up according to spec */ + sc->read_dunits_remainder = 999; + sc->write_dunits_remainder = 999; } static void @@ -961,7 +974,17 @@ nvme_opc_get_log_page(struct pci_nvme_softc* sc, struc NVME_COPY_TO_PRP); break; case NVME_LOG_HEALTH_INFORMATION: - /* TODO: present some smart info */ + pthread_mutex_lock(&sc->mtx); + memcpy(&sc->health_log.data_units_read, &sc->read_data_units, + sizeof(sc->health_log.data_units_read)); + memcpy(&sc->health_log.data_units_written, &sc->write_data_units, + sizeof(sc->health_log.data_units_written)); + memcpy(&sc->health_log.host_read_commands, &sc->read_commands, + sizeof(sc->health_log.host_read_commands)); + memcpy(&sc->health_log.host_write_commands, &sc->write_commands, + sizeof(sc->health_log.host_write_commands)); + pthread_mutex_unlock(&sc->mtx); + nvme_prp_memcpy(sc->nsc_pi->pi_vmctx, command->prp1, command->prp2, (uint8_t *)&sc->health_log, MIN(logsize, sizeof(sc->health_log)), @@ -1465,6 +1488,47 @@ pci_nvme_handle_admin_cmd(struct pci_nvme_softc* sc, u pthread_mutex_unlock(&sq->mtx); } +/* + * Update the Write and Read statistics reported in SMART data + * + * NVMe defines "data unit" as thousand's of 512 byte blocks and is rounded up. + * E.g. 1 data unit is 1 - 1,000 512 byte blocks. 3 data units are 2,001 - 3,000 + * 512 byte blocks. Rounding up is acheived by initializing the remainder to 999. + */ +static void +pci_nvme_stats_write_read_update(struct pci_nvme_softc *sc, uint8_t opc, + size_t bytes, uint16_t status) +{ + + pthread_mutex_lock(&sc->mtx); + switch (opc) { + case NVME_OPC_WRITE: + sc->write_commands++; + if (status != NVME_SC_SUCCESS) + break; + sc->write_dunits_remainder += (bytes / 512); + while (sc->write_dunits_remainder >= 1000) { + sc->write_data_units++; + sc->write_dunits_remainder -= 1000; + } + break; + case NVME_OPC_READ: + sc->read_commands++; + if (status != NVME_SC_SUCCESS) + break; + sc->read_dunits_remainder += (bytes / 512); + while (sc->read_dunits_remainder >= 1000) { + sc->read_data_units++; + sc->read_dunits_remainder -= 1000; + } + break; + default: + DPRINTF("%s: Invalid OPC 0x%02x for stats", __func__, opc); + break; + } + pthread_mutex_unlock(&sc->mtx); +} + static int pci_nvme_append_iov_req(struct pci_nvme_softc *sc, struct pci_nvme_ioreq *req, uint64_t gpaddr, size_t size, int do_write, uint64_t lba) @@ -1605,6 +1669,8 @@ pci_nvme_io_done(struct blockif_req *br, int err) pci_nvme_status_genc(&status, code); pci_nvme_set_completion(req->sc, sq, req->sqid, req->cid, 0, status); + pci_nvme_stats_write_read_update(req->sc, req->opc, + req->bytes, status); pci_nvme_release_ioreq(req->sc, req); } @@ -1779,6 +1845,7 @@ nvme_opc_write_read(struct pci_nvme_softc *sc, goto out; } + req->bytes = bytes; req->io_req.br_offset = lba; /* PRP bits 1:0 must be zero */ @@ -1796,6 +1863,9 @@ nvme_opc_write_read(struct pci_nvme_softc *sc, pending = true; } out: + if (!pending) + pci_nvme_stats_write_read_update(sc, cmd->opc, bytes, *status); + return (pending); }