From owner-dev-commits-src-branches@freebsd.org Sat Jul 31 00:21:36 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id DFA8F65149D; Sat, 31 Jul 2021 00:21:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Gc4gz0TYVz4Vtd; Sat, 31 Jul 2021 00:21:35 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A7E366CA1; Sat, 31 Jul 2021 00:21:34 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 16V0LY2Q052169; Sat, 31 Jul 2021 00:21:34 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 16V0LYrt052168; Sat, 31 Jul 2021 00:21:34 GMT (envelope-from git) Date: Sat, 31 Jul 2021 00:21:34 GMT Message-Id: <202107310021.16V0LYrt052168@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Warner Losh Subject: git: 5b59c5186902 - stable/12 - nvme: use config_intrhook_drain to avoid removable card races MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: imp X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 5b59c5186902f8ac402b89796567fa630abb10c2 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 31 Jul 2021 00:21:36 -0000 The branch stable/12 has been updated by imp: URL: https://cgit.FreeBSD.org/src/commit/?id=5b59c5186902f8ac402b89796567fa630abb10c2 commit 5b59c5186902f8ac402b89796567fa630abb10c2 Author: Warner Losh AuthorDate: 2021-03-11 15:42:44 +0000 Commit: Warner Losh CommitDate: 2021-07-31 00:02:53 +0000 nvme: use config_intrhook_drain to avoid removable card races nvme drives are configured early in boot. However, a number of the configuration steps takes which take a while, so we defer those to a config intrhook that runs before the root filesystem is mounted. At the same time, the PCI hot plug wakes up and tests the status of the card. It may decide that the card has gone away and deletes the child. As part of that process nvme_detach is called. If this call happens after the config_intrhook starts to run, but before it is finished, there's a race where we can tear down the device's soft state while the config_intrhook is still using it. Use the new config_intrhook_drain to disestablish the hook. Either it will be removed w/o running, or the routine will wait for it to finish. This closes the race and allows safe hotplug at any time, even very early in boot. Sponsored by: Netflix, Inc Reviewed by: jhb, mav Differential Revision: https://reviews.freebsd.org/D29006 (cherry picked from commit 8423f5d4c127f18e7500bc455bc7b6b1691385ef) --- sys/dev/nvme/nvme.c | 5 +---- sys/dev/nvme/nvme_ctrlr.c | 2 -- 2 files changed, 1 insertion(+), 6 deletions(-) diff --git a/sys/dev/nvme/nvme.c b/sys/dev/nvme/nvme.c index 8e5bc11b35d4..17ee7b19177d 100644 --- a/sys/dev/nvme/nvme.c +++ b/sys/dev/nvme/nvme.c @@ -144,10 +144,7 @@ nvme_detach(device_t dev) { struct nvme_controller *ctrlr = DEVICE2SOFTC(dev); - if (ctrlr->config_hook.ich_arg != NULL) { - config_intrhook_disestablish(&ctrlr->config_hook); - ctrlr->config_hook.ich_arg = NULL; - } + config_intrhook_drain(&ctrlr->config_hook); nvme_ctrlr_destruct(ctrlr, dev); return (0); diff --git a/sys/dev/nvme/nvme_ctrlr.c b/sys/dev/nvme/nvme_ctrlr.c index d4d472878b82..007e0758023a 100644 --- a/sys/dev/nvme/nvme_ctrlr.c +++ b/sys/dev/nvme/nvme_ctrlr.c @@ -1135,7 +1135,6 @@ nvme_ctrlr_start_config_hook(void *arg) fail: nvme_ctrlr_fail(ctrlr); config_intrhook_disestablish(&ctrlr->config_hook); - ctrlr->config_hook.ich_arg = NULL; return; } @@ -1154,7 +1153,6 @@ fail: nvme_sysctl_initialize_ctrlr(ctrlr); config_intrhook_disestablish(&ctrlr->config_hook); - ctrlr->config_hook.ich_arg = NULL; ctrlr->is_initialized = 1; nvme_notify_new_controller(ctrlr);