From owner-dev-commits-src-branches@freebsd.org Fri May 21 10:28:39 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id B40DC65685E; Fri, 21 May 2021 10:28:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmjWC4dJmz3tVJ; Fri, 21 May 2021 10:28:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 88567754B; Fri, 21 May 2021 10:28:39 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 14LASdN9051231; Fri, 21 May 2021 10:28:39 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 14LASdQA051230; Fri, 21 May 2021 10:28:39 GMT (envelope-from git) Date: Fri, 21 May 2021 10:28:39 GMT Message-Id: <202105211028.14LASdQA051230@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andriy Gapon Subject: git: d7d4592ada54 - stable/13 - PCI hot-plug: use dedicated taskqueue for device attach / detach MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: avg X-Git-Repository: src X-Git-Refname: refs/heads/stable/13 X-Git-Reftype: branch X-Git-Commit: d7d4592ada54a4babcb9336d518929168539b515 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 10:28:39 -0000 The branch stable/13 has been updated by avg: URL: https://cgit.FreeBSD.org/src/commit/?id=d7d4592ada54a4babcb9336d518929168539b515 commit d7d4592ada54a4babcb9336d518929168539b515 Author: Andriy Gapon AuthorDate: 2021-05-06 18:49:37 +0000 Commit: Andriy Gapon CommitDate: 2021-05-21 10:28:06 +0000 PCI hot-plug: use dedicated taskqueue for device attach / detach Attaching and detaching devices can be heavy-weight and detaching can sleep waiting for events. For that reason using the system-wide single-threaded taskqueue_thread is not really appropriate. There is even a possibility for a deadlock if taskqueue_thread is used for detaching. In fact, there is an easy to reproduce deadlock involving nvme, pass and a sudden removal of an NVMe device. A pass peripheral would not release a reference on an nvme sim until pass_shutdown_kqueue() is executed via taskqueue_thread. But the taskqueue's thread is blocked in nvme_detach() -> ... -> cam_sim_free() because of the outstanding reference. Sponsored by: CyberSecure Reviewed by: mav, imp (cherry picked from commit 12588ce02dd835b332952d9fece5881d943554a9) --- sys/dev/pci/pci_pci.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/dev/pci/pci_pci.c b/sys/dev/pci/pci_pci.c index 3a11d59f51fd..d6fbb06a61ac 100644 --- a/sys/dev/pci/pci_pci.c +++ b/sys/dev/pci/pci_pci.c @@ -925,6 +925,8 @@ SYSCTL_INT(_hw_pci, OID_AUTO, enable_pcie_hp, CTLFLAG_RDTUN, &pci_enable_pcie_hp, 0, "Enable support for native PCI-express HotPlug."); +TASKQUEUE_DEFINE_THREAD(pci_hp); + static void pcib_probe_hotplug(struct pcib_softc *sc) { @@ -1154,7 +1156,7 @@ pcib_pcie_hotplug_update(struct pcib_softc *sc, uint16_t val, uint16_t mask, */ if (schedule_task && (pcib_hotplug_present(sc) != 0) != (sc->child != NULL)) - taskqueue_enqueue(taskqueue_thread, &sc->pcie_hp_task); + taskqueue_enqueue(taskqueue_pci_hp, &sc->pcie_hp_task); } static void @@ -1449,7 +1451,7 @@ pcib_detach_hotplug(struct pcib_softc *sc) error = pcib_release_pcie_irq(sc); if (error) return (error); - taskqueue_drain(taskqueue_thread, &sc->pcie_hp_task); + taskqueue_drain(taskqueue_pci_hp, &sc->pcie_hp_task); callout_drain(&sc->pcie_ab_timer); callout_drain(&sc->pcie_cc_timer); callout_drain(&sc->pcie_dll_timer);