From owner-svn-src-all@freebsd.org Tue Feb 13 02:11:41 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14A24F12B62; Tue, 13 Feb 2018 02:11:41 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id BBD1A7C793; Tue, 13 Feb 2018 02:11:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 9E2281A677; Tue, 13 Feb 2018 02:11:40 +0000 (UTC) (envelope-from mav@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w1D2BeZi031290; Tue, 13 Feb 2018 02:11:40 GMT (envelope-from mav@FreeBSD.org) Received: (from mav@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w1D2BdXX031284; Tue, 13 Feb 2018 02:11:39 GMT (envelope-from mav@FreeBSD.org) Message-Id: <201802130211.w1D2BdXX031284@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mav set sender to mav@FreeBSD.org using -f From: Alexander Motin Date: Tue, 13 Feb 2018 02:11:39 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r329189 - in stable/11/sys/dev: mpr mps X-SVN-Group: stable-11 X-SVN-Commit-Author: mav X-SVN-Commit-Paths: in stable/11/sys/dev: mpr mps X-SVN-Commit-Revision: 329189 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Feb 2018 02:11:41 -0000 Author: mav Date: Tue Feb 13 02:11:39 2018 New Revision: 329189 URL: https://svnweb.freebsd.org/changeset/base/329189 Log: MFC r328937: Fix queue length reporting in mps(4) and mpr(4). Both drivers were found to report CAM bigger queue depth then they really can handle. It made them later under high load with many disks return some of submitted requests back with CAM_REQUEUE_REQ status for later resubmission. Modified: stable/11/sys/dev/mpr/mpr.c stable/11/sys/dev/mpr/mpr_sas.c stable/11/sys/dev/mpr/mprvar.h stable/11/sys/dev/mps/mps.c stable/11/sys/dev/mps/mps_sas.c stable/11/sys/dev/mps/mpsvar.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/dev/mpr/mpr.c ============================================================================== --- stable/11/sys/dev/mpr/mpr.c Mon Feb 12 23:53:38 2018 (r329188) +++ stable/11/sys/dev/mpr/mpr.c Tue Feb 13 02:11:39 2018 (r329189) @@ -503,7 +503,10 @@ mpr_iocfacts_allocate(struct mpr_softc *sc, uint8_t at * Size the queues. Since the reply queues always need one free * entry, we'll just deduct one reply message here. */ - sc->num_reqs = MIN(MPR_REQ_FRAMES, sc->facts->RequestCredit); + sc->num_prireqs = MIN(MPR_PRI_REQ_FRAMES, + sc->facts->HighPriorityCredit); + sc->num_reqs = MIN(MPR_REQ_FRAMES, sc->facts->RequestCredit) + + sc->num_prireqs; sc->num_replies = MIN(MPR_REPLY_FRAMES + MPR_EVT_REPLY_FRAMES, sc->facts->MaxReplyDescriptorPostQueueDepth) - 1; @@ -1362,7 +1365,7 @@ mpr_alloc_requests(struct mpr_softc *sc) /* XXX Is a failure here a critical problem? */ if (bus_dmamap_create(sc->buffer_dmat, 0, &cm->cm_dmamap) == 0) { - if (i <= sc->facts->HighPriorityCredit) + if (i <= sc->num_prireqs) mpr_free_high_priority_command(sc, cm); else mpr_free_command(sc, cm); Modified: stable/11/sys/dev/mpr/mpr_sas.c ============================================================================== --- stable/11/sys/dev/mpr/mpr_sas.c Mon Feb 12 23:53:38 2018 (r329188) +++ stable/11/sys/dev/mpr/mpr_sas.c Tue Feb 13 02:11:39 2018 (r329189) @@ -728,7 +728,7 @@ mpr_attach_sas(struct mpr_softc *sc) { struct mprsas_softc *sassc; cam_status status; - int unit, error = 0; + int unit, error = 0, reqs; MPR_FUNCTRACE(sc); @@ -757,7 +757,8 @@ mpr_attach_sas(struct mpr_softc *sc) sc->sassc = sassc; sassc->sc = sc; - if ((sassc->devq = cam_simq_alloc(sc->num_reqs)) == NULL) { + reqs = sc->num_reqs - sc->num_prireqs - 1; + if ((sassc->devq = cam_simq_alloc(reqs)) == NULL) { mpr_dprint(sc, MPR_ERROR, "Cannot allocate SIMQ\n"); error = ENOMEM; goto out; @@ -765,7 +766,7 @@ mpr_attach_sas(struct mpr_softc *sc) unit = device_get_unit(sc->mpr_dev); sassc->sim = cam_sim_alloc(mprsas_action, mprsas_poll, "mpr", sassc, - unit, &sc->mpr_mtx, sc->num_reqs, sc->num_reqs, sassc->devq); + unit, &sc->mpr_mtx, reqs, reqs, sassc->devq); if (sassc->sim == NULL) { mpr_dprint(sc, MPR_ERROR, "Cannot allocate SIM\n"); error = EINVAL; Modified: stable/11/sys/dev/mpr/mprvar.h ============================================================================== --- stable/11/sys/dev/mpr/mprvar.h Mon Feb 12 23:53:38 2018 (r329188) +++ stable/11/sys/dev/mpr/mprvar.h Tue Feb 13 02:11:39 2018 (r329189) @@ -38,6 +38,7 @@ #define MPR_DB_MAX_WAIT 2500 #define MPR_REQ_FRAMES 1024 +#define MPR_PRI_REQ_FRAMES 128 #define MPR_EVT_REPLY_FRAMES 32 #define MPR_REPLY_FRAMES MPR_REQ_FRAMES #define MPR_CHAIN_FRAMES 2048 @@ -328,6 +329,7 @@ struct mpr_softc { MPI2_IOC_FACTS_REPLY *facts; int num_reqs; + int num_prireqs; int num_replies; int fqdepth; /* Free queue */ int pqdepth; /* Post queue */ Modified: stable/11/sys/dev/mps/mps.c ============================================================================== --- stable/11/sys/dev/mps/mps.c Mon Feb 12 23:53:38 2018 (r329188) +++ stable/11/sys/dev/mps/mps.c Tue Feb 13 02:11:39 2018 (r329189) @@ -489,7 +489,10 @@ mps_iocfacts_allocate(struct mps_softc *sc, uint8_t at * Size the queues. Since the reply queues always need one free * entry, we'll just deduct one reply message here. */ - sc->num_reqs = MIN(MPS_REQ_FRAMES, sc->facts->RequestCredit); + sc->num_prireqs = MIN(MPS_PRI_REQ_FRAMES, + sc->facts->HighPriorityCredit); + sc->num_reqs = MIN(MPS_REQ_FRAMES, sc->facts->RequestCredit) + + sc->num_prireqs; sc->num_replies = MIN(MPS_REPLY_FRAMES + MPS_EVT_REPLY_FRAMES, sc->facts->MaxReplyDescriptorPostQueueDepth) - 1; @@ -1303,7 +1306,7 @@ mps_alloc_requests(struct mps_softc *sc) /* XXX Is a failure here a critical problem? */ if (bus_dmamap_create(sc->buffer_dmat, 0, &cm->cm_dmamap) == 0) - if (i <= sc->facts->HighPriorityCredit) + if (i <= sc->num_prireqs) mps_free_high_priority_command(sc, cm); else mps_free_command(sc, cm); Modified: stable/11/sys/dev/mps/mps_sas.c ============================================================================== --- stable/11/sys/dev/mps/mps_sas.c Mon Feb 12 23:53:38 2018 (r329188) +++ stable/11/sys/dev/mps/mps_sas.c Tue Feb 13 02:11:39 2018 (r329189) @@ -716,7 +716,7 @@ mps_attach_sas(struct mps_softc *sc) { struct mpssas_softc *sassc; cam_status status; - int unit, error = 0; + int unit, error = 0, reqs; MPS_FUNCTRACE(sc); @@ -745,7 +745,8 @@ mps_attach_sas(struct mps_softc *sc) sc->sassc = sassc; sassc->sc = sc; - if ((sassc->devq = cam_simq_alloc(sc->num_reqs)) == NULL) { + reqs = sc->num_reqs - sc->num_prireqs - 1; + if ((sassc->devq = cam_simq_alloc(reqs)) == NULL) { mps_dprint(sc, MPS_ERROR, "Cannot allocate SIMQ\n"); error = ENOMEM; goto out; @@ -753,7 +754,7 @@ mps_attach_sas(struct mps_softc *sc) unit = device_get_unit(sc->mps_dev); sassc->sim = cam_sim_alloc(mpssas_action, mpssas_poll, "mps", sassc, - unit, &sc->mps_mtx, sc->num_reqs, sc->num_reqs, sassc->devq); + unit, &sc->mps_mtx, reqs, reqs, sassc->devq); if (sassc->sim == NULL) { mps_dprint(sc, MPS_ERROR, "Cannot allocate SIM\n"); error = EINVAL; Modified: stable/11/sys/dev/mps/mpsvar.h ============================================================================== --- stable/11/sys/dev/mps/mpsvar.h Mon Feb 12 23:53:38 2018 (r329188) +++ stable/11/sys/dev/mps/mpsvar.h Tue Feb 13 02:11:39 2018 (r329189) @@ -38,6 +38,7 @@ #define MPS_DB_MAX_WAIT 2500 #define MPS_REQ_FRAMES 1024 +#define MPS_PRI_REQ_FRAMES 128 #define MPS_EVT_REPLY_FRAMES 32 #define MPS_REPLY_FRAMES MPS_REQ_FRAMES #define MPS_CHAIN_FRAMES 2048 @@ -314,6 +315,7 @@ struct mps_softc { MPI2_IOC_FACTS_REPLY *facts; int num_reqs; + int num_prireqs; int num_replies; int fqdepth; /* Free queue */ int pqdepth; /* Post queue */