From owner-svn-src-all@freebsd.org Thu Sep 5 23:12:57 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CA016D7204; Thu, 5 Sep 2019 23:12:57 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Pc1553zsz4W6J; Thu, 5 Sep 2019 23:12:57 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 92B71BA25; Thu, 5 Sep 2019 23:12:57 +0000 (UTC) (envelope-from imp@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x85NCvDM035063; Thu, 5 Sep 2019 23:12:57 GMT (envelope-from imp@FreeBSD.org) Received: (from imp@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x85NCvDo035060; Thu, 5 Sep 2019 23:12:57 GMT (envelope-from imp@FreeBSD.org) Message-Id: <201909052312.x85NCvDo035060@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: imp set sender to imp@FreeBSD.org using -f From: Warner Losh Date: Thu, 5 Sep 2019 23:12:57 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-12@freebsd.org Subject: svn commit: r351910 - stable/12/sys/dev/nvme X-SVN-Group: stable-12 X-SVN-Commit-Author: imp X-SVN-Commit-Paths: stable/12/sys/dev/nvme X-SVN-Commit-Revision: 351910 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 05 Sep 2019 23:12:57 -0000 Author: imp Date: Thu Sep 5 23:12:56 2019 New Revision: 351910 URL: https://svnweb.freebsd.org/changeset/base/351910 Log: MFC r351705: In all the places that we use the polled for completion interface, except crash dump support code, move the while loop into an inline function. These aren't done in the fast path, so if the compiler choses to not inline, any performance hit is tiny. Modified: stable/12/sys/dev/nvme/nvme_ctrlr.c stable/12/sys/dev/nvme/nvme_ns.c stable/12/sys/dev/nvme/nvme_private.h Directory Properties: stable/12/ (props changed) Modified: stable/12/sys/dev/nvme/nvme_ctrlr.c ============================================================================== --- stable/12/sys/dev/nvme/nvme_ctrlr.c Thu Sep 5 23:12:06 2019 (r351909) +++ stable/12/sys/dev/nvme/nvme_ctrlr.c Thu Sep 5 23:12:56 2019 (r351910) @@ -394,8 +394,7 @@ nvme_ctrlr_identify(struct nvme_controller *ctrlr) status.done = 0; nvme_ctrlr_cmd_identify_controller(ctrlr, &ctrlr->cdata, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_identify_controller failed!\n"); return (ENXIO); @@ -424,8 +423,7 @@ nvme_ctrlr_set_num_qpairs(struct nvme_controller *ctrl status.done = 0; nvme_ctrlr_cmd_set_num_queues(ctrlr, ctrlr->num_io_queues, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_ctrlr_set_num_qpairs failed!\n"); return (ENXIO); @@ -463,8 +461,7 @@ nvme_ctrlr_create_qpairs(struct nvme_controller *ctrlr status.done = 0; nvme_ctrlr_cmd_create_io_cq(ctrlr, qpair, qpair->vector, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_create_io_cq failed!\n"); return (ENXIO); @@ -473,8 +470,7 @@ nvme_ctrlr_create_qpairs(struct nvme_controller *ctrlr status.done = 0; nvme_ctrlr_cmd_create_io_sq(qpair->ctrlr, qpair, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_create_io_sq failed!\n"); return (ENXIO); @@ -496,8 +492,7 @@ nvme_ctrlr_destroy_qpairs(struct nvme_controller *ctrl status.done = 0; nvme_ctrlr_cmd_delete_io_sq(ctrlr, qpair, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_destroy_io_sq failed!\n"); return (ENXIO); @@ -506,8 +501,7 @@ nvme_ctrlr_destroy_qpairs(struct nvme_controller *ctrl status.done = 0; nvme_ctrlr_cmd_delete_io_cq(ctrlr, qpair, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_destroy_io_cq failed!\n"); return (ENXIO); @@ -789,8 +783,7 @@ nvme_ctrlr_configure_aer(struct nvme_controller *ctrlr status.done = 0; nvme_ctrlr_cmd_get_feature(ctrlr, NVME_FEAT_TEMPERATURE_THRESHOLD, 0, NULL, 0, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl) || (status.cpl.cdw0 & 0xFFFF) == 0xFFFF || (status.cpl.cdw0 & 0xFFFF) == 0x0000) { Modified: stable/12/sys/dev/nvme/nvme_ns.c ============================================================================== --- stable/12/sys/dev/nvme/nvme_ns.c Thu Sep 5 23:12:06 2019 (r351909) +++ stable/12/sys/dev/nvme/nvme_ns.c Thu Sep 5 23:12:56 2019 (r351910) @@ -530,8 +530,7 @@ nvme_ns_construct(struct nvme_namespace *ns, uint32_t status.done = 0; nvme_ctrlr_cmd_identify_namespace(ctrlr, id, &ns->data, nvme_completion_poll_cb, &status); - while (!atomic_load_acq_int(&status.done)) - pause("nvme", 1); + nvme_completion_poll(&status); if (nvme_completion_is_error(&status.cpl)) { nvme_printf(ctrlr, "nvme_identify_namespace failed\n"); return (ENXIO); Modified: stable/12/sys/dev/nvme/nvme_private.h ============================================================================== --- stable/12/sys/dev/nvme/nvme_private.h Thu Sep 5 23:12:06 2019 (r351909) +++ stable/12/sys/dev/nvme/nvme_private.h Thu Sep 5 23:12:56 2019 (r351910) @@ -446,6 +446,14 @@ int nvme_attach(device_t dev); int nvme_shutdown(device_t dev); int nvme_detach(device_t dev); +static __inline +void +nvme_completion_poll(struct nvme_completion_poll_status *status) +{ + while (!atomic_load_acq_int(&status->done)) + pause("nvme", 1); +} + static __inline void nvme_single_map(void *arg, bus_dma_segment_t *seg, int nseg, int error) {