From owner-svn-src-all@freebsd.org Tue Feb 6 16:02:26 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 34609EE40A9; Tue, 6 Feb 2018 16:02:26 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id DA7CB70E93; Tue, 6 Feb 2018 16:02:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id D56D61A184; Tue, 6 Feb 2018 16:02:25 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w16G2PaL051431; Tue, 6 Feb 2018 16:02:25 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w16G2PRV051425; Tue, 6 Feb 2018 16:02:25 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201802061602.w16G2PRV051425@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 6 Feb 2018 16:02:25 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r328937 - in head/sys/dev: mpr mps X-SVN-Group: head X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in head/sys/dev: mpr mps X-SVN-Commit-Revision: 328937 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Feb 2018 16:02:26 -0000 Author: mav Date: Tue Feb 6 16:02:25 2018 New Revision: 328937 URL: https://svnweb.freebsd.org/changeset/base/328937 Log: Fix queue length reporting in mps(4) and mpr(4). Both drivers were found to report CAM bigger queue depth then they really can handle. It made them later under high load with many disks return some of submitted requests back with CAM_REQUEUE_REQ status for later resubmission. Reviewed by: scottl MFC after: 1 week Sponsored by: iXsystems, Inc. Differential Revision: https://reviews.freebsd.org/D14215 Modified: head/sys/dev/mpr/mpr.c head/sys/dev/mpr/mpr_sas.c head/sys/dev/mpr/mprvar.h head/sys/dev/mps/mps.c head/sys/dev/mps/mps_sas.c head/sys/dev/mps/mpsvar.h Modified: head/sys/dev/mpr/mpr.c ============================================================================== --- head/sys/dev/mpr/mpr.c Tue Feb 6 15:58:22 2018 (r328936) +++ head/sys/dev/mpr/mpr.c Tue Feb 6 16:02:25 2018 (r328937) @@ -397,6 +397,7 @@ mpr_resize_queues(struct mpr_softc *sc) reqcr = MIN(reqcr, sc->facts->RequestCredit); sc->num_reqs = prireqcr + reqcr; + sc->num_prireqs = prireqcr; sc->num_replies = MIN(sc->max_replyframes + sc->max_evtframes, sc->facts->MaxReplyDescriptorPostQueueDepth) - 1; @@ -1507,7 +1508,7 @@ mpr_alloc_requests(struct mpr_softc *sc) /* XXX Is a failure here a critical problem? */ if (bus_dmamap_create(sc->buffer_dmat, 0, &cm->cm_dmamap) == 0) { - if (i <= sc->facts->HighPriorityCredit) + if (i <= sc->num_prireqs) mpr_free_high_priority_command(sc, cm); else mpr_free_command(sc, cm); Modified: head/sys/dev/mpr/mpr_sas.c ============================================================================== --- head/sys/dev/mpr/mpr_sas.c Tue Feb 6 15:58:22 2018 (r328936) +++ head/sys/dev/mpr/mpr_sas.c Tue Feb 6 16:02:25 2018 (r328937) @@ -728,7 +728,7 @@ mpr_attach_sas(struct mpr_softc *sc) { struct mprsas_softc *sassc; cam_status status; - int unit, error = 0; + int unit, error = 0, reqs; MPR_FUNCTRACE(sc); mpr_dprint(sc, MPR_INIT, "%s entered\n", __func__); @@ -758,7 +758,8 @@ mpr_attach_sas(struct mpr_softc *sc) sc->sassc = sassc; sassc->sc = sc; - if ((sassc->devq = cam_simq_alloc(sc->num_reqs)) == NULL) { + reqs = sc->num_reqs - sc->num_prireqs - 1; + if ((sassc->devq = cam_simq_alloc(reqs)) == NULL) { mpr_dprint(sc, MPR_INIT|MPR_ERROR, "Cannot allocate SIMQ\n"); error = ENOMEM; goto out; @@ -766,7 +767,7 @@ mpr_attach_sas(struct mpr_softc *sc) unit = device_get_unit(sc->mpr_dev); sassc->sim = cam_sim_alloc(mprsas_action, mprsas_poll, "mpr", sassc, - unit, &sc->mpr_mtx, sc->num_reqs, sc->num_reqs, sassc->devq); + unit, &sc->mpr_mtx, reqs, reqs, sassc->devq); if (sassc->sim == NULL) { mpr_dprint(sc, MPR_INIT|MPR_ERROR, "Cannot allocate SIM\n"); error = EINVAL; Modified: head/sys/dev/mpr/mprvar.h ============================================================================== --- head/sys/dev/mpr/mprvar.h Tue Feb 6 15:58:22 2018 (r328936) +++ head/sys/dev/mpr/mprvar.h Tue Feb 6 16:02:25 2018 (r328937) @@ -357,6 +357,7 @@ struct mpr_softc { MPI2_IOC_FACTS_REPLY *facts; int num_reqs; + int num_prireqs; int num_replies; int fqdepth; /* Free queue */ int pqdepth; /* Post queue */ Modified: head/sys/dev/mps/mps.c ============================================================================== --- head/sys/dev/mps/mps.c Tue Feb 6 15:58:22 2018 (r328936) +++ head/sys/dev/mps/mps.c Tue Feb 6 16:02:25 2018 (r328937) @@ -394,6 +394,7 @@ mps_resize_queues(struct mps_softc *sc) reqcr = MIN(reqcr, sc->facts->RequestCredit); sc->num_reqs = prireqcr + reqcr; + sc->num_prireqs = prireqcr; sc->num_replies = MIN(sc->max_replyframes + sc->max_evtframes, sc->facts->MaxReplyDescriptorPostQueueDepth) - 1; @@ -1453,7 +1454,7 @@ mps_alloc_requests(struct mps_softc *sc) /* XXX Is a failure here a critical problem? */ if (bus_dmamap_create(sc->buffer_dmat, 0, &cm->cm_dmamap) == 0) - if (i <= sc->facts->HighPriorityCredit) + if (i <= sc->num_prireqs) mps_free_high_priority_command(sc, cm); else mps_free_command(sc, cm); Modified: head/sys/dev/mps/mps_sas.c ============================================================================== --- head/sys/dev/mps/mps_sas.c Tue Feb 6 15:58:22 2018 (r328936) +++ head/sys/dev/mps/mps_sas.c Tue Feb 6 16:02:25 2018 (r328937) @@ -718,7 +718,7 @@ mps_attach_sas(struct mps_softc *sc) { struct mpssas_softc *sassc; cam_status status; - int unit, error = 0; + int unit, error = 0, reqs; MPS_FUNCTRACE(sc); mps_dprint(sc, MPS_INIT, "%s entered\n", __func__); @@ -748,7 +748,8 @@ mps_attach_sas(struct mps_softc *sc) sc->sassc = sassc; sassc->sc = sc; - if ((sassc->devq = cam_simq_alloc(sc->num_reqs)) == NULL) { + reqs = sc->num_reqs - sc->num_prireqs - 1; + if ((sassc->devq = cam_simq_alloc(reqs)) == NULL) { mps_dprint(sc, MPS_ERROR, "Cannot allocate SIMQ\n"); error = ENOMEM; goto out; @@ -756,7 +757,7 @@ mps_attach_sas(struct mps_softc *sc) unit = device_get_unit(sc->mps_dev); sassc->sim = cam_sim_alloc(mpssas_action, mpssas_poll, "mps", sassc, - unit, &sc->mps_mtx, sc->num_reqs, sc->num_reqs, sassc->devq); + unit, &sc->mps_mtx, reqs, reqs, sassc->devq); if (sassc->sim == NULL) { mps_dprint(sc, MPS_INIT|MPS_ERROR, "Cannot allocate SIM\n"); error = EINVAL; Modified: head/sys/dev/mps/mpsvar.h ============================================================================== --- head/sys/dev/mps/mpsvar.h Tue Feb 6 15:58:22 2018 (r328936) +++ head/sys/dev/mps/mpsvar.h Tue Feb 6 16:02:25 2018 (r328937) @@ -345,6 +345,7 @@ struct mps_softc { MPI2_IOC_FACTS_REPLY *facts; int num_reqs; + int num_prireqs; int num_replies; int fqdepth; /* Free queue */ int pqdepth; /* Post queue */