From owner-svn-src-all@freebsd.org Thu Jun 14 17:06:20 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 4EB221014897; Thu, 14 Jun 2018 17:06:20 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id EF49574EA9; Thu, 14 Jun 2018 17:06:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id CCB5033F6; Thu, 14 Jun 2018 17:06:19 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w5EH6J9l087479; Thu, 14 Jun 2018 17:06:19 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w5EH6JSj087478; Thu, 14 Jun 2018 17:06:19 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201806141706.w5EH6JSj087478@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Thu, 14 Jun 2018 17:06:19 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r335153 - stable/11/sys/dev/nvme X-SVN-Group: stable-11 X-SVN-Commit-Author: mav X-SVN-Commit-Paths: stable/11/sys/dev/nvme X-SVN-Commit-Revision: 335153 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 14 Jun 2018 17:06:20 -0000 Author: mav Date: Thu Jun 14 17:06:19 2018 New Revision: 335153 URL: https://svnweb.freebsd.org/changeset/base/335153 Log: MFC r333130: Improve nvme(4) attach/detach sequences. This change allows clean device detach on attach failures and driver unload, while previous code tried to talk to already shut down controller, or even accessed resources failed to allocate. Modified: stable/11/sys/dev/nvme/nvme_ctrlr.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/nvme/nvme_ctrlr.c ============================================================================== --- stable/11/sys/dev/nvme/nvme_ctrlr.c Thu Jun 14 17:02:58 2018 (r335152) +++ stable/11/sys/dev/nvme/nvme_ctrlr.c Thu Jun 14 17:06:19 2018 (r335153) @@ -1151,6 +1151,7 @@ nvme_ctrlr_setup_interrupts(struct nvme_controller *ct int nvme_ctrlr_construct(struct nvme_controller *ctrlr, device_t dev) { + struct make_dev_args md_args; union cap_lo_register cap_lo; union cap_hi_register cap_hi; int status, timeout_period; @@ -1196,14 +1197,6 @@ nvme_ctrlr_construct(struct nvme_controller *ctrlr, de if (nvme_ctrlr_construct_admin_qpair(ctrlr) != 0) return (ENXIO); - ctrlr->cdev = make_dev(&nvme_ctrlr_cdevsw, device_get_unit(dev), - UID_ROOT, GID_WHEEL, 0600, "nvme%d", device_get_unit(dev)); - - if (ctrlr->cdev == NULL) - return (ENXIO); - - ctrlr->cdev->si_drv1 = (void *)ctrlr; - ctrlr->taskqueue = taskqueue_create("nvme_taskq", M_WAITOK, taskqueue_thread_enqueue, &ctrlr->taskqueue); taskqueue_start_threads(&ctrlr->taskqueue, 1, PI_DISK, "nvme taskq"); @@ -1212,11 +1205,22 @@ nvme_ctrlr_construct(struct nvme_controller *ctrlr, de ctrlr->is_initialized = 0; ctrlr->notification_sent = 0; TASK_INIT(&ctrlr->reset_task, 0, nvme_ctrlr_reset_task, ctrlr); - TASK_INIT(&ctrlr->fail_req_task, 0, nvme_ctrlr_fail_req_task, ctrlr); STAILQ_INIT(&ctrlr->fail_req); ctrlr->is_failed = FALSE; + make_dev_args_init(&md_args); + md_args.mda_devsw = &nvme_ctrlr_cdevsw; + md_args.mda_uid = UID_ROOT; + md_args.mda_gid = GID_WHEEL; + md_args.mda_mode = 0600; + md_args.mda_unit = device_get_unit(dev); + md_args.mda_si_drv1 = (void *)ctrlr; + status = make_dev_s(&md_args, &ctrlr->cdev, "nvme%d", + device_get_unit(dev)); + if (status != 0) + return (ENXIO); + return (0); } @@ -1225,18 +1229,9 @@ nvme_ctrlr_destruct(struct nvme_controller *ctrlr, dev { int i; - /* - * Notify the controller of a shutdown, even though this is due to - * a driver unload, not a system shutdown (this path is not invoked - * during shutdown). This ensures the controller receives a - * shutdown notification in case the system is shutdown before - * reloading the driver. - */ - nvme_ctrlr_shutdown(ctrlr); + if (ctrlr->resource == NULL) + goto nores; - nvme_ctrlr_disable(ctrlr); - taskqueue_free(ctrlr->taskqueue); - for (i = 0; i < NVME_MAX_NAMESPACES; i++) nvme_ns_destruct(&ctrlr->ns[i]); @@ -1247,21 +1242,24 @@ nvme_ctrlr_destruct(struct nvme_controller *ctrlr, dev nvme_ctrlr_destroy_qpair(ctrlr, &ctrlr->ioq[i]); nvme_io_qpair_destroy(&ctrlr->ioq[i]); } - free(ctrlr->ioq, M_NVME); nvme_admin_qpair_destroy(&ctrlr->adminq); - if (ctrlr->resource != NULL) { - bus_release_resource(dev, SYS_RES_MEMORY, - ctrlr->resource_id, ctrlr->resource); - } + /* + * Notify the controller of a shutdown, even though this is due to + * a driver unload, not a system shutdown (this path is not invoked + * during shutdown). This ensures the controller receives a + * shutdown notification in case the system is shutdown before + * reloading the driver. + */ + nvme_ctrlr_shutdown(ctrlr); - if (ctrlr->bar4_resource != NULL) { - bus_release_resource(dev, SYS_RES_MEMORY, - ctrlr->bar4_resource_id, ctrlr->bar4_resource); - } + nvme_ctrlr_disable(ctrlr); + if (ctrlr->taskqueue) + taskqueue_free(ctrlr->taskqueue); + if (ctrlr->tag) bus_teardown_intr(ctrlr->dev, ctrlr->res, ctrlr->tag); @@ -1271,6 +1269,17 @@ nvme_ctrlr_destruct(struct nvme_controller *ctrlr, dev if (ctrlr->msix_enabled) pci_release_msi(dev); + + if (ctrlr->bar4_resource != NULL) { + bus_release_resource(dev, SYS_RES_MEMORY, + ctrlr->bar4_resource_id, ctrlr->bar4_resource); + } + + bus_release_resource(dev, SYS_RES_MEMORY, + ctrlr->resource_id, ctrlr->resource); + +nores: + mtx_destroy(&ctrlr->lock); } void