From owner-dev-commits-src-branches@freebsd.org Fri May 21 10:31:27 2021 Return-Path: Delivered-To: dev-commits-src-branches@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 9D4776567E9; Fri, 21 May 2021 10:31:27 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4FmjZR0v9fz3vMG; Fri, 21 May 2021 10:31:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9F4047A2C; Fri, 21 May 2021 10:31:26 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 14LAVQZ0063149; Fri, 21 May 2021 10:31:26 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 14LAVQYo063148; Fri, 21 May 2021 10:31:26 GMT (envelope-from git) Date: Fri, 21 May 2021 10:31:26 GMT Message-Id: <202105211031.14LAVQYo063148@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-branches@FreeBSD.org From: Andriy Gapon Subject: git: 2def2de60a6d - stable/12 - PCI hot-plug: use dedicated taskqueue for device attach / detach MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: avg X-Git-Repository: src X-Git-Refname: refs/heads/stable/12 X-Git-Reftype: branch X-Git-Commit: 2def2de60a6d0b979964bb67c5ff0e24d098e4a9 Auto-Submitted: auto-generated X-BeenThere: dev-commits-src-branches@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Commits to the stable branches of the FreeBSD src repository List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 21 May 2021 10:31:27 -0000 The branch stable/12 has been updated by avg: URL: https://cgit.FreeBSD.org/src/commit/?id=2def2de60a6d0b979964bb67c5ff0e24d098e4a9 commit 2def2de60a6d0b979964bb67c5ff0e24d098e4a9 Author: Andriy Gapon AuthorDate: 2021-05-06 18:49:37 +0000 Commit: Andriy Gapon CommitDate: 2021-05-21 10:31:04 +0000 PCI hot-plug: use dedicated taskqueue for device attach / detach Attaching and detaching devices can be heavy-weight and detaching can sleep waiting for events. For that reason using the system-wide single-threaded taskqueue_thread is not really appropriate. There is even a possibility for a deadlock if taskqueue_thread is used for detaching. In fact, there is an easy to reproduce deadlock involving nvme, pass and a sudden removal of an NVMe device. A pass peripheral would not release a reference on an nvme sim until pass_shutdown_kqueue() is executed via taskqueue_thread. But the taskqueue's thread is blocked in nvme_detach() -> ... -> cam_sim_free() because of the outstanding reference. Sponsored by: CyberSecure Reviewed by: mav, imp (cherry picked from commit 12588ce02dd835b332952d9fece5881d943554a9) --- sys/dev/pci/pci_pci.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/sys/dev/pci/pci_pci.c b/sys/dev/pci/pci_pci.c index ff3dcdc84c47..4b7560f499ef 100644 --- a/sys/dev/pci/pci_pci.c +++ b/sys/dev/pci/pci_pci.c @@ -922,6 +922,8 @@ SYSCTL_INT(_hw_pci, OID_AUTO, enable_pcie_hp, CTLFLAG_RDTUN, &pci_enable_pcie_hp, 0, "Enable support for native PCI-express HotPlug."); +TASKQUEUE_DEFINE_THREAD(pci_hp); + static void pcib_probe_hotplug(struct pcib_softc *sc) { @@ -1151,7 +1153,7 @@ pcib_pcie_hotplug_update(struct pcib_softc *sc, uint16_t val, uint16_t mask, */ if (schedule_task && (pcib_hotplug_present(sc) != 0) != (sc->child != NULL)) - taskqueue_enqueue(taskqueue_thread, &sc->pcie_hp_task); + taskqueue_enqueue(taskqueue_pci_hp, &sc->pcie_hp_task); } static void @@ -1443,7 +1445,7 @@ pcib_detach_hotplug(struct pcib_softc *sc) error = pcib_release_pcie_irq(sc); if (error) return (error); - taskqueue_drain(taskqueue_thread, &sc->pcie_hp_task); + taskqueue_drain(taskqueue_pci_hp, &sc->pcie_hp_task); callout_drain(&sc->pcie_ab_timer); callout_drain(&sc->pcie_cc_timer); callout_drain(&sc->pcie_dll_timer);