From owner-svn-src-all@freebsd.org Fri Dec 18 03:46:50 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id CA8E24ADE53; Fri, 18 Dec 2020 03:46:50 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Cxvtf5M2Fz3hXJ; Fri, 18 Dec 2020 03:46:50 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id AAC0E241EC; Fri, 18 Dec 2020 03:46:50 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 0BI3kodv075515; Fri, 18 Dec 2020 03:46:50 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 0BI3ko8o075514; Fri, 18 Dec 2020 03:46:50 GMT (envelope-from kib@FreeBSD.org) Message-Id: <202012180346.0BI3ko8o075514@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Fri, 18 Dec 2020 03:46:50 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r368749 - head/sys/dev/pci X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/dev/pci X-SVN-Commit-Revision: 368749 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 18 Dec 2020 03:46:50 -0000 Author: kib Date: Fri Dec 18 03:46:50 2020 New Revision: 368749 URL: https://svnweb.freebsd.org/changeset/base/368749 Log: pci_iov: When pci_iov_detach(9) is called, destroy VF children instead of bailing out with EBUSY if there are any. If driver module is unloaded, or just device is forcibly detached from the driver, there is no way for driver to correctly unload otherwise. Esp. if there are resources dedicated to the VFs which prevent turning down other resources. Reviewed by: jhb Sponsored by: Mellanox Technologies / NVidia Networking MFC after: 1 week Differential revision: https://reviews.freebsd.org/D27615 Modified: head/sys/dev/pci/pci_iov.c Modified: head/sys/dev/pci/pci_iov.c ============================================================================== --- head/sys/dev/pci/pci_iov.c Fri Dec 18 02:41:12 2020 (r368748) +++ head/sys/dev/pci/pci_iov.c Fri Dec 18 03:46:50 2020 (r368749) @@ -95,6 +95,7 @@ static void pci_iov_build_pf_schema(nvlist_t *schema, nvlist_t **driver_schema); static void pci_iov_build_vf_schema(nvlist_t *schema, nvlist_t **driver_schema); +static int pci_iov_delete_iov_children(struct pci_devinfo *dinfo); static nvlist_t *pci_iov_get_pf_subsystem_schema(void); static nvlist_t *pci_iov_get_vf_subsystem_schema(void); @@ -191,6 +192,7 @@ pci_iov_detach_method(device_t bus, device_t dev) { struct pci_devinfo *dinfo; struct pcicfg_iov *iov; + int error; mtx_lock(&Giant); dinfo = device_get_ivars(dev); @@ -201,11 +203,17 @@ pci_iov_detach_method(device_t bus, device_t dev) return (0); } - if (iov->iov_num_vfs != 0 || iov->iov_flags & IOV_BUSY) { + if ((iov->iov_flags & IOV_BUSY) != 0) { mtx_unlock(&Giant); return (EBUSY); } + error = pci_iov_delete_iov_children(dinfo); + if (error != 0) { + mtx_unlock(&Giant); + return (error); + } + dinfo->cfg.iov = NULL; if (iov->iov_cdev) { @@ -823,31 +831,20 @@ pci_iov_is_child_vf(struct pcicfg_iov *pf, device_t ch } static int -pci_iov_delete(struct cdev *cdev) +pci_iov_delete_iov_children(struct pci_devinfo *dinfo) { device_t bus, dev, vf, *devlist; - struct pci_devinfo *dinfo; struct pcicfg_iov *iov; int i, error, devcount; uint32_t iov_ctl; - mtx_lock(&Giant); - dinfo = cdev->si_drv1; + mtx_assert(&Giant, MA_OWNED); + iov = dinfo->cfg.iov; dev = dinfo->cfg.dev; bus = device_get_parent(dev); devlist = NULL; - if (iov->iov_flags & IOV_BUSY) { - mtx_unlock(&Giant); - return (EBUSY); - } - - if (iov->iov_num_vfs == 0) { - mtx_unlock(&Giant); - return (ECHILD); - } - iov->iov_flags |= IOV_BUSY; error = device_get_children(bus, &devlist, &devcount); @@ -905,6 +902,32 @@ pci_iov_delete(struct cdev *cdev) out: free(devlist, M_TEMP); iov->iov_flags &= ~IOV_BUSY; + return (error); +} + +static int +pci_iov_delete(struct cdev *cdev) +{ + struct pci_devinfo *dinfo; + struct pcicfg_iov *iov; + int error; + + mtx_lock(&Giant); + dinfo = cdev->si_drv1; + iov = dinfo->cfg.iov; + + if ((iov->iov_flags & IOV_BUSY) != 0) { + error = EBUSY; + goto out; + } + if (iov->iov_num_vfs == 0) { + error = ECHILD; + goto out; + } + + error = pci_iov_delete_iov_children(dinfo); + +out: mtx_unlock(&Giant); return (error); }