From owner-svn-src-head@freebsd.org Tue Feb 6 15:58:24 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D7C8AEE3974; Tue, 6 Feb 2018 15:58:23 +0000 (UTC) (envelope-from ken@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7BFBC7098B; Tue, 6 Feb 2018 15:58:23 +0000 (UTC) (envelope-from ken@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 76CE219FF5; Tue, 6 Feb 2018 15:58:23 +0000 (UTC) (envelope-from ken@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w16FwNnb046852; Tue, 6 Feb 2018 15:58:23 GMT (envelope-from ken@FreeBSD.org) Received: (from ken@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w16FwMcS046846; Tue, 6 Feb 2018 15:58:22 GMT (envelope-from ken@FreeBSD.org) Message-Id: <201802061558.w16FwMcS046846@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: ken set sender to ken@FreeBSD.org using -f From: "Kenneth D. Merry" Date: Tue, 6 Feb 2018 15:58:22 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r328936 - in head/sys/dev: mpr mps X-SVN-Group: head X-SVN-Commit-Author: ken X-SVN-Commit-Paths: in head/sys/dev: mpr mps X-SVN-Commit-Revision: 328936 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 06 Feb 2018 15:58:24 -0000 Author: ken Date: Tue Feb 6 15:58:22 2018 New Revision: 328936 URL: https://svnweb.freebsd.org/changeset/base/328936 Log: Diagnostic buffer fixes for the mps(4) and mpr(4) drivers. In mp{r,s}_diag_register(), which is used to register diagnostic buffers with the mp{r,s}(4) firmware, we allocate DMAable memory. There were several issues here: o No checking of the bus_dmamap_load() return value. If the load failed or got deferred, mp{r,s}_diag_register() continued on as if nothing had happened. We now check the return value and bail out if it fails. o No waiting for a deferred load callback. bus_dmamap_load() calls a supplied callback when the mapping is done. This is generally done immediately, but it can be deferred. mp{r,s}_diag_register() did not check to see whether the callback was already done before proceeding on. We now sleep until the callback is done if it is deferred. o No call to bus_dmamap_sync(... BUS_DMASYNC_PREREAD) after the memory is allocated and loaded. This is necessary on some platforms to synchronize host memory that is going to be updated by a device. Both drivers would also panic if the firmware was reinitialized while a diagnostic buffer operation was in progress. This fixes that problem as well. (The driver will reinitialize the firmware in various circumstances, but the problem I ran into was that the firmware would generate an IOC Fault due to a PCIe error.) mp{r,s}var.h: Add a new structure, struct mpr_busdma_context, that is used for deferred busdma load callbacks. Add a prototype for mp{r,s}_memaddr_wait_cb(). mp{r,s}.c: Add a new busdma callback function, mp{r,s}_memaddr_wait_cb(). This provides synchronization for callers that want to wait on a deferred bus_dmamap_load() callback. mp{r,s}_user.c: In bus_dmamap_register(), add a call to bus_dmamap_sync() with the BUS_DMASYNC_PREREAD flag set after an allocation is loaded. Also, check the return value of bus_dmamap_load(). If it fails, bail out. If it is EINPROGRESS, wait for the callback to happen. We use an interruptible sleep (msleep with PCATCH) and let the callback clean things up if we get interrupted. In mpr_diag_read_buffer() and mps_diag_read_buffer(), call bus_dmamap_sync(..., BUS_DMASYNC_POSTREAD) before copying the data out to make sure the data is in stable storage. In mp{r,s}_post_fw_diag_buffer() and mp{r,s}_release_fw_diag_buffer(), check the reply to see whether it is NULL. It can be NULL (and the command non-NULL) if the controller gets reinitialized while we're waiting for the command to complete but the driver structures aren't reallocated. The driver structures generally won't be reallocated unless there is a firmware upgrade that changes one of the IOCFacts. When freeing diagnostic buffers in mp{r,s}_diag_register() and mp{r,s}_diag_unregister(), zero/NULL out the buffer after freeing it. This will prevent a duplicate free in some situations. Sponsored by: Spectra Logic Reviewed by: mav, scottl MFC after: 1 week Differential Revision: D13453 Modified: head/sys/dev/mpr/mpr.c head/sys/dev/mpr/mpr_user.c head/sys/dev/mpr/mprvar.h head/sys/dev/mps/mps.c head/sys/dev/mps/mps_user.c head/sys/dev/mps/mpsvar.h Modified: head/sys/dev/mpr/mpr.c ============================================================================== --- head/sys/dev/mpr/mpr.c Tue Feb 6 15:41:45 2018 (r328935) +++ head/sys/dev/mpr/mpr.c Tue Feb 6 15:58:22 2018 (r328936) @@ -1183,6 +1183,42 @@ mpr_memaddr_cb(void *arg, bus_dma_segment_t *segs, int *addr = segs[0].ds_addr; } +void +mpr_memaddr_wait_cb(void *arg, bus_dma_segment_t *segs, int nsegs, int error) +{ + struct mpr_busdma_context *ctx; + int need_unload, need_free; + + ctx = (struct mpr_busdma_context *)arg; + need_unload = 0; + need_free = 0; + + mpr_lock(ctx->softc); + ctx->error = error; + ctx->completed = 1; + if ((error == 0) && (ctx->abandoned == 0)) { + *ctx->addr = segs[0].ds_addr; + } else { + if (nsegs != 0) + need_unload = 1; + if (ctx->abandoned != 0) + need_free = 1; + } + if (need_free == 0) + wakeup(ctx); + + mpr_unlock(ctx->softc); + + if (need_unload != 0) { + bus_dmamap_unload(ctx->buffer_dmat, + ctx->buffer_dmamap); + *ctx->addr = 0; + } + + if (need_free != 0) + free(ctx, M_MPR); +} + static int mpr_alloc_queues(struct mpr_softc *sc) { Modified: head/sys/dev/mpr/mpr_user.c ============================================================================== --- head/sys/dev/mpr/mpr_user.c Tue Feb 6 15:41:45 2018 (r328935) +++ head/sys/dev/mpr/mpr_user.c Tue Feb 6 15:58:22 2018 (r328936) @@ -1314,6 +1314,13 @@ mpr_post_fw_diag_buffer(struct mpr_softc *sc, * Process POST reply. */ reply = (MPI2_DIAG_BUFFER_POST_REPLY *)cm->cm_reply; + if (reply == NULL) { + mpr_printf(sc, "%s: reply is NULL, probably due to " + "reinitialization", __func__); + status = MPR_DIAG_FAILURE; + goto done; + } + if ((le16toh(reply->IOCStatus) & MPI2_IOCSTATUS_MASK) != MPI2_IOCSTATUS_SUCCESS) { status = MPR_DIAG_FAILURE; @@ -1401,6 +1408,12 @@ mpr_release_fw_diag_buffer(struct mpr_softc *sc, * Process RELEASE reply. */ reply = (MPI2_DIAG_RELEASE_REPLY *)cm->cm_reply; + if (reply == NULL) { + mpr_printf(sc, "%s: reply is NULL, probably due to " + "reinitialization", __func__); + status = MPR_DIAG_FAILURE; + goto done; + } if (((le16toh(reply->IOCStatus) & MPI2_IOCSTATUS_MASK) != MPI2_IOCSTATUS_SUCCESS) || pBuffer->owned_by_firmware) { status = MPR_DIAG_FAILURE; @@ -1436,15 +1449,19 @@ mpr_diag_register(struct mpr_softc *sc, mpr_fw_diag_re uint32_t *return_code) { mpr_fw_diagnostic_buffer_t *pBuffer; + struct mpr_busdma_context *ctx; uint8_t extended_type, buffer_type, i; uint32_t buffer_size; uint32_t unique_id; int status; + int error; extended_type = diag_register->ExtendedType; buffer_type = diag_register->BufferType; buffer_size = diag_register->RequestedBufferSize; unique_id = diag_register->UniqueId; + ctx = NULL; + error = 0; /* * Check for valid buffer type @@ -1493,7 +1510,7 @@ mpr_diag_register(struct mpr_softc *sc, mpr_fw_diag_re *return_code = MPR_FW_DIAG_ERROR_NO_BUFFER; return (MPR_DIAG_FAILURE); } - if (bus_dma_tag_create( sc->mpr_parent_dmat, /* parent */ + if (bus_dma_tag_create( sc->mpr_parent_dmat, /* parent */ 1, 0, /* algnmnt, boundary */ BUS_SPACE_MAXADDR_32BIT,/* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ @@ -1506,17 +1523,83 @@ mpr_diag_register(struct mpr_softc *sc, mpr_fw_diag_re &sc->fw_diag_dmat)) { mpr_dprint(sc, MPR_ERROR, "Cannot allocate FW diag buffer DMA tag\n"); - return (ENOMEM); - } + *return_code = MPR_FW_DIAG_ERROR_NO_BUFFER; + status = MPR_DIAG_FAILURE; + goto bailout; + } if (bus_dmamem_alloc(sc->fw_diag_dmat, (void **)&sc->fw_diag_buffer, BUS_DMA_NOWAIT, &sc->fw_diag_map)) { mpr_dprint(sc, MPR_ERROR, "Cannot allocate FW diag buffer memory\n"); - return (ENOMEM); - } - bzero(sc->fw_diag_buffer, buffer_size); - bus_dmamap_load(sc->fw_diag_dmat, sc->fw_diag_map, sc->fw_diag_buffer, - buffer_size, mpr_memaddr_cb, &sc->fw_diag_busaddr, 0); + *return_code = MPR_FW_DIAG_ERROR_NO_BUFFER; + status = MPR_DIAG_FAILURE; + goto bailout; + } + bzero(sc->fw_diag_buffer, buffer_size); + + ctx = malloc(sizeof(*ctx), M_MPR, M_WAITOK | M_ZERO); + if (ctx == NULL) { + device_printf(sc->mpr_dev, "%s: context malloc failed\n", + __func__); + *return_code = MPR_FW_DIAG_ERROR_NO_BUFFER; + status = MPR_DIAG_FAILURE; + goto bailout; + } + ctx->addr = &sc->fw_diag_busaddr; + ctx->buffer_dmat = sc->fw_diag_dmat; + ctx->buffer_dmamap = sc->fw_diag_map; + ctx->softc = sc; + error = bus_dmamap_load(sc->fw_diag_dmat, sc->fw_diag_map, + sc->fw_diag_buffer, buffer_size, mpr_memaddr_wait_cb, + ctx, 0); + if (error == EINPROGRESS) { + + /* XXX KDM */ + device_printf(sc->mpr_dev, "%s: Deferred bus_dmamap_load\n", + __func__); + /* + * Wait for the load to complete. If we're interrupted, + * bail out. + */ + mpr_lock(sc); + if (ctx->completed == 0) { + error = msleep(ctx, &sc->mpr_mtx, PCATCH, "mprwait", 0); + if (error != 0) { + /* + * We got an error from msleep(9). This is + * most likely due to a signal. Tell + * mpr_memaddr_wait_cb() that we've abandoned + * the context, so it needs to clean up when + * it is called. + */ + ctx->abandoned = 1; + + /* The callback will free this memory */ + ctx = NULL; + mpr_unlock(sc); + + device_printf(sc->mpr_dev, "Cannot " + "bus_dmamap_load FW diag buffer, error = " + "%d returned from msleep\n", error); + *return_code = MPR_FW_DIAG_ERROR_NO_BUFFER; + status = MPR_DIAG_FAILURE; + goto bailout; + } + } + mpr_unlock(sc); + } + + if ((error != 0) || (ctx->error != 0)) { + device_printf(sc->mpr_dev, "Cannot bus_dmamap_load FW diag " + "buffer, %serror = %d\n", error ? "" : "callback ", + error ? error : ctx->error); + *return_code = MPR_FW_DIAG_ERROR_NO_BUFFER; + status = MPR_DIAG_FAILURE; + goto bailout; + } + + bus_dmamap_sync(sc->fw_diag_dmat, sc->fw_diag_map, BUS_DMASYNC_PREREAD); + pBuffer->size = buffer_size; /* @@ -1535,19 +1618,30 @@ mpr_diag_register(struct mpr_softc *sc, mpr_fw_diag_re pBuffer->unique_id = unique_id; status = mpr_post_fw_diag_buffer(sc, pBuffer, return_code); +bailout: + /* * In case there was a failure, free the DMA buffer. */ if (status == MPR_DIAG_FAILURE) { - if (sc->fw_diag_busaddr != 0) + if (sc->fw_diag_busaddr != 0) { bus_dmamap_unload(sc->fw_diag_dmat, sc->fw_diag_map); - if (sc->fw_diag_buffer != NULL) + sc->fw_diag_busaddr = 0; + } + if (sc->fw_diag_buffer != NULL) { bus_dmamem_free(sc->fw_diag_dmat, sc->fw_diag_buffer, sc->fw_diag_map); - if (sc->fw_diag_dmat != NULL) + sc->fw_diag_buffer = NULL; + } + if (sc->fw_diag_dmat != NULL) { bus_dma_tag_destroy(sc->fw_diag_dmat); + sc->fw_diag_dmat = NULL; + } } + if (ctx != NULL) + free(ctx, M_MPR); + return (status); } @@ -1592,13 +1686,19 @@ mpr_diag_unregister(struct mpr_softc *sc, */ pBuffer->unique_id = MPR_FW_DIAG_INVALID_UID; if (status == MPR_DIAG_SUCCESS) { - if (sc->fw_diag_busaddr != 0) + if (sc->fw_diag_busaddr != 0) { bus_dmamap_unload(sc->fw_diag_dmat, sc->fw_diag_map); - if (sc->fw_diag_buffer != NULL) + sc->fw_diag_busaddr = 0; + } + if (sc->fw_diag_buffer != NULL) { bus_dmamem_free(sc->fw_diag_dmat, sc->fw_diag_buffer, sc->fw_diag_map); - if (sc->fw_diag_dmat != NULL) + sc->fw_diag_buffer = NULL; + } + if (sc->fw_diag_dmat != NULL) { bus_dma_tag_destroy(sc->fw_diag_dmat); + sc->fw_diag_dmat = NULL; + } } return (status); @@ -1707,6 +1807,10 @@ mpr_diag_read_buffer(struct mpr_softc *sc, *return_code = MPR_FW_DIAG_ERROR_INVALID_PARAMETER; return (MPR_DIAG_FAILURE); } + + /* Sync the DMA map before we copy to userland. */ + bus_dmamap_sync(sc->fw_diag_dmat, sc->fw_diag_map, + BUS_DMASYNC_POSTREAD); /* * Copy the requested data from DMA to the diag_read_buffer. The DMA Modified: head/sys/dev/mpr/mprvar.h ============================================================================== --- head/sys/dev/mpr/mprvar.h Tue Feb 6 15:41:45 2018 (r328935) +++ head/sys/dev/mpr/mprvar.h Tue Feb 6 15:58:22 2018 (r328936) @@ -265,6 +265,16 @@ struct mpr_event_handle { uint8_t mask[16]; }; +struct mpr_busdma_context { + int completed; + int abandoned; + int error; + bus_addr_t *addr; + struct mpr_softc *softc; + bus_dmamap_t buffer_dmamap; + bus_dma_tag_t buffer_dmat; +}; + struct mpr_queue { struct mpr_softc *sc; int qnum; @@ -752,6 +762,7 @@ int mpr_detach_sas(struct mpr_softc *sc); int mpr_read_config_page(struct mpr_softc *, struct mpr_config_params *); int mpr_write_config_page(struct mpr_softc *, struct mpr_config_params *); void mpr_memaddr_cb(void *, bus_dma_segment_t *, int , int ); +void mpr_memaddr_wait_cb(void *, bus_dma_segment_t *, int , int ); void mpr_init_sge(struct mpr_command *cm, void *req, void *sge); int mpr_attach_user(struct mpr_softc *); void mpr_detach_user(struct mpr_softc *); Modified: head/sys/dev/mps/mps.c ============================================================================== --- head/sys/dev/mps/mps.c Tue Feb 6 15:41:45 2018 (r328935) +++ head/sys/dev/mps/mps.c Tue Feb 6 15:58:22 2018 (r328936) @@ -111,6 +111,7 @@ static void mps_parse_debug(struct mps_softc *sc, char SYSCTL_NODE(_hw, OID_AUTO, mps, CTLFLAG_RD, 0, "MPS Driver Parameters"); MALLOC_DEFINE(M_MPT2, "mps", "mpt2 driver memory"); +MALLOC_DECLARE(M_MPSUSER); /* * Do a "Diagnostic Reset" aka a hard reset. This should get the chip out of @@ -1158,6 +1159,42 @@ mps_memaddr_cb(void *arg, bus_dma_segment_t *segs, int addr = arg; *addr = segs[0].ds_addr; +} + +void +mps_memaddr_wait_cb(void *arg, bus_dma_segment_t *segs, int nsegs, int error) +{ + struct mps_busdma_context *ctx; + int need_unload, need_free; + + ctx = (struct mps_busdma_context *)arg; + need_unload = 0; + need_free = 0; + + mps_lock(ctx->softc); + ctx->error = error; + ctx->completed = 1; + if ((error == 0) && (ctx->abandoned == 0)) { + *ctx->addr = segs[0].ds_addr; + } else { + if (nsegs != 0) + need_unload = 1; + if (ctx->abandoned != 0) + need_free = 1; + } + if (need_free == 0) + wakeup(ctx); + + mps_unlock(ctx->softc); + + if (need_unload != 0) { + bus_dmamap_unload(ctx->buffer_dmat, + ctx->buffer_dmamap); + *ctx->addr = 0; + } + + if (need_free != 0) + free(ctx, M_MPSUSER); } static int Modified: head/sys/dev/mps/mps_user.c ============================================================================== --- head/sys/dev/mps/mps_user.c Tue Feb 6 15:41:45 2018 (r328935) +++ head/sys/dev/mps/mps_user.c Tue Feb 6 15:58:22 2018 (r328936) @@ -180,7 +180,7 @@ static int mps_user_event_report(struct mps_softc *sc, static int mps_user_reg_access(struct mps_softc *sc, mps_reg_access_t *data); static int mps_user_btdh(struct mps_softc *sc, mps_btdh_mapping_t *data); -static MALLOC_DEFINE(M_MPSUSER, "mps_user", "Buffers for mps(4) ioctls"); +MALLOC_DEFINE(M_MPSUSER, "mps_user", "Buffers for mps(4) ioctls"); /* Macros from compat/freebsd32/freebsd32.h */ #define PTRIN(v) (void *)(uintptr_t)(v) @@ -1222,6 +1222,12 @@ mps_post_fw_diag_buffer(struct mps_softc *sc, * Process POST reply. */ reply = (MPI2_DIAG_BUFFER_POST_REPLY *)cm->cm_reply; + if (reply == NULL) { + mps_printf(sc, "%s: reply is NULL, probably due to " + "reinitialization\n", __func__); + status = MPS_DIAG_FAILURE; + goto done; + } if ((le16toh(reply->IOCStatus) & MPI2_IOCSTATUS_MASK) != MPI2_IOCSTATUS_SUCCESS) { status = MPS_DIAG_FAILURE; @@ -1309,6 +1315,12 @@ mps_release_fw_diag_buffer(struct mps_softc *sc, * Process RELEASE reply. */ reply = (MPI2_DIAG_RELEASE_REPLY *)cm->cm_reply; + if (reply == NULL) { + mps_printf(sc, "%s: reply is NULL, probably due to " + "reinitialization\n", __func__); + status = MPS_DIAG_FAILURE; + goto done; + } if (((le16toh(reply->IOCStatus) & MPI2_IOCSTATUS_MASK) != MPI2_IOCSTATUS_SUCCESS) || pBuffer->owned_by_firmware) { status = MPS_DIAG_FAILURE; @@ -1344,15 +1356,19 @@ mps_diag_register(struct mps_softc *sc, mps_fw_diag_re uint32_t *return_code) { mps_fw_diagnostic_buffer_t *pBuffer; + struct mps_busdma_context *ctx; uint8_t extended_type, buffer_type, i; uint32_t buffer_size; uint32_t unique_id; int status; + int error; extended_type = diag_register->ExtendedType; buffer_type = diag_register->BufferType; buffer_size = diag_register->RequestedBufferSize; unique_id = diag_register->UniqueId; + ctx = NULL; + error = 0; /* * Check for valid buffer type @@ -1401,7 +1417,7 @@ mps_diag_register(struct mps_softc *sc, mps_fw_diag_re *return_code = MPS_FW_DIAG_ERROR_NO_BUFFER; return (MPS_DIAG_FAILURE); } - if (bus_dma_tag_create( sc->mps_parent_dmat, /* parent */ + if (bus_dma_tag_create( sc->mps_parent_dmat, /* parent */ 1, 0, /* algnmnt, boundary */ BUS_SPACE_MAXADDR_32BIT,/* lowaddr */ BUS_SPACE_MAXADDR, /* highaddr */ @@ -1414,17 +1430,84 @@ mps_diag_register(struct mps_softc *sc, mps_fw_diag_re &sc->fw_diag_dmat)) { mps_dprint(sc, MPS_ERROR, "Cannot allocate FW diag buffer DMA tag\n"); - return (ENOMEM); - } - if (bus_dmamem_alloc(sc->fw_diag_dmat, (void **)&sc->fw_diag_buffer, + *return_code = MPS_FW_DIAG_ERROR_NO_BUFFER; + status = MPS_DIAG_FAILURE; + goto bailout; + } + if (bus_dmamem_alloc(sc->fw_diag_dmat, (void **)&sc->fw_diag_buffer, BUS_DMA_NOWAIT, &sc->fw_diag_map)) { mps_dprint(sc, MPS_ERROR, "Cannot allocate FW diag buffer memory\n"); - return (ENOMEM); + *return_code = MPS_FW_DIAG_ERROR_NO_BUFFER; + status = MPS_DIAG_FAILURE; + goto bailout; } bzero(sc->fw_diag_buffer, buffer_size); - bus_dmamap_load(sc->fw_diag_dmat, sc->fw_diag_map, sc->fw_diag_buffer, - buffer_size, mps_memaddr_cb, &sc->fw_diag_busaddr, 0); + + ctx = malloc(sizeof(*ctx), M_MPSUSER, M_WAITOK | M_ZERO); + if (ctx == NULL) { + device_printf(sc->mps_dev, "%s: context malloc failed\n", + __func__); + *return_code = MPS_FW_DIAG_ERROR_NO_BUFFER; + status = MPS_DIAG_FAILURE; + goto bailout; + } + ctx->addr = &sc->fw_diag_busaddr; + ctx->buffer_dmat = sc->fw_diag_dmat; + ctx->buffer_dmamap = sc->fw_diag_map; + ctx->softc = sc; + error = bus_dmamap_load(sc->fw_diag_dmat, sc->fw_diag_map, + sc->fw_diag_buffer, buffer_size, mps_memaddr_wait_cb, + ctx, 0); + + if (error == EINPROGRESS) { + + /* XXX KDM */ + device_printf(sc->mps_dev, "%s: Deferred bus_dmamap_load\n", + __func__); + /* + * Wait for the load to complete. If we're interrupted, + * bail out. + */ + mps_lock(sc); + if (ctx->completed == 0) { + error = msleep(ctx, &sc->mps_mtx, PCATCH, "mpswait", 0); + if (error != 0) { + /* + * We got an error from msleep(9). This is + * most likely due to a signal. Tell + * mpr_memaddr_wait_cb() that we've abandoned + * the context, so it needs to clean up when + * it is called. + */ + ctx->abandoned = 1; + + /* The callback will free this memory */ + ctx = NULL; + mps_unlock(sc); + + device_printf(sc->mps_dev, "Cannot " + "bus_dmamap_load FW diag buffer, error = " + "%d returned from msleep\n", error); + *return_code = MPS_FW_DIAG_ERROR_NO_BUFFER; + status = MPS_DIAG_FAILURE; + goto bailout; + } + } + mps_unlock(sc); + } + + if ((error != 0) || (ctx->error != 0)) { + device_printf(sc->mps_dev, "Cannot bus_dmamap_load FW diag " + "buffer, %serror = %d\n", error ? "" : "callback ", + error ? error : ctx->error); + *return_code = MPS_FW_DIAG_ERROR_NO_BUFFER; + status = MPS_DIAG_FAILURE; + goto bailout; + } + + bus_dmamap_sync(sc->fw_diag_dmat, sc->fw_diag_map, BUS_DMASYNC_PREREAD); + pBuffer->size = buffer_size; /* @@ -1443,19 +1526,29 @@ mps_diag_register(struct mps_softc *sc, mps_fw_diag_re pBuffer->unique_id = unique_id; status = mps_post_fw_diag_buffer(sc, pBuffer, return_code); +bailout: /* * In case there was a failure, free the DMA buffer. */ if (status == MPS_DIAG_FAILURE) { - if (sc->fw_diag_busaddr != 0) + if (sc->fw_diag_busaddr != 0) { bus_dmamap_unload(sc->fw_diag_dmat, sc->fw_diag_map); - if (sc->fw_diag_buffer != NULL) + sc->fw_diag_busaddr = 0; + } + if (sc->fw_diag_buffer != NULL) { bus_dmamem_free(sc->fw_diag_dmat, sc->fw_diag_buffer, sc->fw_diag_map); - if (sc->fw_diag_dmat != NULL) + sc->fw_diag_buffer = NULL; + } + if (sc->fw_diag_dmat != NULL) { bus_dma_tag_destroy(sc->fw_diag_dmat); + sc->fw_diag_dmat = NULL; + } } + if (ctx != NULL) + free(ctx, M_MPSUSER); + return (status); } @@ -1500,13 +1593,19 @@ mps_diag_unregister(struct mps_softc *sc, */ pBuffer->unique_id = MPS_FW_DIAG_INVALID_UID; if (status == MPS_DIAG_SUCCESS) { - if (sc->fw_diag_busaddr != 0) + if (sc->fw_diag_busaddr != 0) { bus_dmamap_unload(sc->fw_diag_dmat, sc->fw_diag_map); - if (sc->fw_diag_buffer != NULL) + sc->fw_diag_busaddr = 0; + } + if (sc->fw_diag_buffer != NULL) { bus_dmamem_free(sc->fw_diag_dmat, sc->fw_diag_buffer, sc->fw_diag_map); - if (sc->fw_diag_dmat != NULL) + sc->fw_diag_buffer = NULL; + } + if (sc->fw_diag_dmat != NULL) { bus_dma_tag_destroy(sc->fw_diag_dmat); + sc->fw_diag_dmat = NULL; + } } return (status); @@ -1615,6 +1714,10 @@ mps_diag_read_buffer(struct mps_softc *sc, *return_code = MPS_FW_DIAG_ERROR_INVALID_PARAMETER; return (MPS_DIAG_FAILURE); } + + /* Sync the DMA map before we copy to userland. */ + bus_dmamap_sync(sc->fw_diag_dmat, sc->fw_diag_map, + BUS_DMASYNC_POSTREAD); /* * Copy the requested data from DMA to the diag_read_buffer. The DMA Modified: head/sys/dev/mps/mpsvar.h ============================================================================== --- head/sys/dev/mps/mpsvar.h Tue Feb 6 15:41:45 2018 (r328935) +++ head/sys/dev/mps/mpsvar.h Tue Feb 6 15:58:22 2018 (r328936) @@ -263,6 +263,16 @@ struct mps_event_handle { u32 mask[MPI2_EVENT_NOTIFY_EVENTMASK_WORDS]; }; +struct mps_busdma_context { + int completed; + int abandoned; + int error; + bus_addr_t *addr; + struct mps_softc *softc; + bus_dmamap_t buffer_dmamap; + bus_dma_tag_t buffer_dmat; +}; + struct mps_queue { struct mps_softc *sc; int qnum; @@ -719,6 +729,7 @@ int mps_detach_sas(struct mps_softc *sc); int mps_read_config_page(struct mps_softc *, struct mps_config_params *); int mps_write_config_page(struct mps_softc *, struct mps_config_params *); void mps_memaddr_cb(void *, bus_dma_segment_t *, int , int ); +void mps_memaddr_wait_cb(void *, bus_dma_segment_t *, int , int ); void mpi_init_sge(struct mps_command *cm, void *req, void *sge); int mps_attach_user(struct mps_softc *); void mps_detach_user(struct mps_softc *);