From owner-svn-src-head@freebsd.org Sun Jul 30 06:54:00 2017 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 18B58DB08C3; Sun, 30 Jul 2017 06:54:00 +0000 (UTC) (envelope-from scottl@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D965067CFE; Sun, 30 Jul 2017 06:53:59 +0000 (UTC) (envelope-from scottl@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v6U6rwGo099102; Sun, 30 Jul 2017 06:53:58 GMT (envelope-from scottl@FreeBSD.org) Received: (from scottl@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v6U6rwLN099096; Sun, 30 Jul 2017 06:53:58 GMT (envelope-from scottl@FreeBSD.org) Message-Id: <201707300653.v6U6rwLN099096@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: scottl set sender to scottl@FreeBSD.org using -f From: Scott Long Date: Sun, 30 Jul 2017 06:53:58 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r321714 - in head/sys/dev: mpr mps X-SVN-Group: head X-SVN-Commit-Author: scottl X-SVN-Commit-Paths: in head/sys/dev: mpr mps X-SVN-Commit-Revision: 321714 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 30 Jul 2017 06:54:00 -0000 Author: scottl Date: Sun Jul 30 06:53:58 2017 New Revision: 321714 URL: https://svnweb.freebsd.org/changeset/base/321714 Log: Split the interrupt setup code into two parts: allocation and configuration. Do the allocation before requesting the IOCFacts message. This triggers the LSI firmware to recognize the multiqueue should be enabled if available. Multiqueue isn't used by the driver yet, but this also fixes a problem with the cached IOCFacts not matching latter checks, leading to potential problems with error recovery. As a side-effect, fetch the driver tunables as early as possible. Reviewed by: slm Obtained from: Netflix Differential Revision: D9243 Modified: head/sys/dev/mpr/mpr.c head/sys/dev/mpr/mpr_pci.c head/sys/dev/mpr/mprvar.h head/sys/dev/mps/mps.c head/sys/dev/mps/mps_pci.c head/sys/dev/mps/mpsvar.h Modified: head/sys/dev/mpr/mpr.c ============================================================================== --- head/sys/dev/mpr/mpr.c Sun Jul 30 06:27:32 2017 (r321713) +++ head/sys/dev/mpr/mpr.c Sun Jul 30 06:53:58 2017 (r321714) @@ -1482,7 +1482,7 @@ mpr_init_queues(struct mpr_softc *sc) * Next are the global settings, if they exist. Highest are the per-unit * settings, if they exist. */ -static void +void mpr_get_tunables(struct mpr_softc *sc) { char tmpstr[80]; @@ -1657,8 +1657,6 @@ int mpr_attach(struct mpr_softc *sc) { int error; - - mpr_get_tunables(sc); MPR_FUNCTRACE(sc); Modified: head/sys/dev/mpr/mpr_pci.c ============================================================================== --- head/sys/dev/mpr/mpr_pci.c Sun Jul 30 06:27:32 2017 (r321713) +++ head/sys/dev/mpr/mpr_pci.c Sun Jul 30 06:53:58 2017 (r321714) @@ -69,6 +69,7 @@ static int mpr_pci_resume(device_t); static void mpr_pci_free(struct mpr_softc *); static int mpr_alloc_msix(struct mpr_softc *sc, int msgs); static int mpr_alloc_msi(struct mpr_softc *sc, int msgs); +static int mpr_pci_alloc_interrupts(struct mpr_softc *sc); static device_method_t mpr_methods[] = { DEVMETHOD(device_probe, mpr_pci_probe), @@ -191,6 +192,8 @@ mpr_pci_attach(device_t dev) m = mpr_find_ident(dev); sc->mpr_flags = m->flags; + mpr_get_tunables(sc); + /* Twiddle basic PCI config bits for a sanity check */ pci_enable_busmaster(dev); @@ -240,28 +243,51 @@ mpr_pci_attach(device_t dev) return (ENOMEM); } - if ((error = mpr_attach(sc)) != 0) + if (((error = mpr_pci_alloc_interrupts(sc)) != 0) || + ((error = mpr_attach(sc)) != 0)) mpr_pci_free(sc); return (error); } +/* + * Allocate, but don't assign interrupts early. Doing it before requesting + * the IOCFacts message informs the firmware that we want to do MSI-X + * multiqueue. We might not use all of the available messages, but there's + * no reason to re-alloc if we don't. + */ int -mpr_pci_setup_interrupts(struct mpr_softc *sc) +mpr_pci_alloc_interrupts(struct mpr_softc *sc) { device_t dev; - int i, error, msgs; + int error, msgs; dev = sc->mpr_dev; - error = ENXIO; + error = 0; + if ((sc->disable_msix == 0) && ((msgs = pci_msix_count(dev)) >= MPR_MSI_COUNT)) error = mpr_alloc_msix(sc, MPR_MSI_COUNT); if ((error != 0) && (sc->disable_msi == 0) && ((msgs = pci_msi_count(dev)) >= MPR_MSI_COUNT)) error = mpr_alloc_msi(sc, MPR_MSI_COUNT); + else + msgs = 0; - if (error != 0) { + sc->msi_msgs = msgs; + return (error); +} + +int +mpr_pci_setup_interrupts(struct mpr_softc *sc) +{ + device_t dev; + int i, error; + + dev = sc->mpr_dev; + error = ENXIO; + + if (sc->msi_msgs == 0) { sc->mpr_flags |= MPR_FLAGS_INTX; sc->mpr_irq_rid[0] = 0; sc->mpr_irq[0] = bus_alloc_resource_any(dev, SYS_RES_IRQ, Modified: head/sys/dev/mpr/mprvar.h ============================================================================== --- head/sys/dev/mpr/mprvar.h Sun Jul 30 06:27:32 2017 (r321713) +++ head/sys/dev/mpr/mprvar.h Sun Jul 30 06:53:58 2017 (r321714) @@ -278,6 +278,7 @@ struct mpr_softc { u_int mpr_debug; u_int disable_msix; u_int disable_msi; + int msi_msgs; u_int atomic_desc_capable; int tm_cmds_active; int io_cmds_active; @@ -702,6 +703,7 @@ mpr_unmask_intr(struct mpr_softc *sc) int mpr_pci_setup_interrupts(struct mpr_softc *sc); int mpr_pci_restore(struct mpr_softc *sc); +void mpr_get_tunables(struct mpr_softc *sc); int mpr_attach(struct mpr_softc *sc); int mpr_free(struct mpr_softc *sc); void mpr_intr(void *); Modified: head/sys/dev/mps/mps.c ============================================================================== --- head/sys/dev/mps/mps.c Sun Jul 30 06:27:32 2017 (r321713) +++ head/sys/dev/mps/mps.c Sun Jul 30 06:53:58 2017 (r321714) @@ -1341,7 +1341,7 @@ mps_init_queues(struct mps_softc *sc) * Next are the global settings, if they exist. Highest are the per-unit * settings, if they exist. */ -static void +void mps_get_tunables(struct mps_softc *sc) { char tmpstr[80]; @@ -1512,8 +1512,6 @@ int mps_attach(struct mps_softc *sc) { int error; - - mps_get_tunables(sc); MPS_FUNCTRACE(sc); Modified: head/sys/dev/mps/mps_pci.c ============================================================================== --- head/sys/dev/mps/mps_pci.c Sun Jul 30 06:27:32 2017 (r321713) +++ head/sys/dev/mps/mps_pci.c Sun Jul 30 06:53:58 2017 (r321714) @@ -68,6 +68,7 @@ static int mps_pci_resume(device_t); static void mps_pci_free(struct mps_softc *); static int mps_alloc_msix(struct mps_softc *sc, int msgs); static int mps_alloc_msi(struct mps_softc *sc, int msgs); +static int mps_pci_alloc_interrupts(struct mps_softc *sc); static device_method_t mps_methods[] = { DEVMETHOD(device_probe, mps_pci_probe), @@ -191,6 +192,8 @@ mps_pci_attach(device_t dev) m = mps_find_ident(dev); sc->mps_flags = m->flags; + mps_get_tunables(sc); + /* Twiddle basic PCI config bits for a sanity check */ pci_enable_busmaster(dev); @@ -221,28 +224,51 @@ mps_pci_attach(device_t dev) return (ENOMEM); } - if ((error = mps_attach(sc)) != 0) + if (((error = mps_pci_alloc_interrupts(sc)) != 0) || + ((error = mps_attach(sc)) != 0)) mps_pci_free(sc); return (error); } -int -mps_pci_setup_interrupts(struct mps_softc *sc) +/* + * Allocate, but don't assign interrupts early. Doing it before requesting + * the IOCFacts message informs the firmware that we want to do MSI-X + * multiqueue. We might not use all of the available messages, but there's + * no reason to re-alloc if we don't. + */ +static int +mps_pci_alloc_interrupts(struct mps_softc *sc) { device_t dev; - int i, error, msgs; + int error, msgs; dev = sc->mps_dev; - error = ENXIO; + error = 0; + if ((sc->disable_msix == 0) && ((msgs = pci_msix_count(dev)) >= MPS_MSI_COUNT)) error = mps_alloc_msix(sc, MPS_MSI_COUNT); if ((error != 0) && (sc->disable_msi == 0) && ((msgs = pci_msi_count(dev)) >= MPS_MSI_COUNT)) error = mps_alloc_msi(sc, MPS_MSI_COUNT); + else + msgs = 0; - if (error != 0) { + sc->msi_msgs = msgs; + return (error); +} + +int +mps_pci_setup_interrupts(struct mps_softc *sc) +{ + device_t dev; + int i, error; + + dev = sc->mps_dev; + error = ENXIO; + + if (sc->msi_msgs == 0) { sc->mps_flags |= MPS_FLAGS_INTX; sc->mps_irq_rid[0] = 0; sc->mps_irq[0] = bus_alloc_resource_any(dev, SYS_RES_IRQ, Modified: head/sys/dev/mps/mpsvar.h ============================================================================== --- head/sys/dev/mps/mpsvar.h Sun Jul 30 06:27:32 2017 (r321713) +++ head/sys/dev/mps/mpsvar.h Sun Jul 30 06:53:58 2017 (r321714) @@ -274,6 +274,7 @@ struct mps_softc { u_int mps_debug; u_int disable_msix; u_int disable_msi; + u_int msi_msgs; int tm_cmds_active; int io_cmds_active; int io_cmds_highwater; @@ -671,6 +672,7 @@ mps_unmask_intr(struct mps_softc *sc) int mps_pci_setup_interrupts(struct mps_softc *sc); int mps_pci_restore(struct mps_softc *sc); +void mps_get_tunables(struct mps_softc *sc); int mps_attach(struct mps_softc *sc); int mps_free(struct mps_softc *sc); void mps_intr(void *);