From owner-svn-src-stable@freebsd.org Sun Jan 27 19:04:30 2019 Return-Path: Delivered-To: svn-src-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AE90D14B8EEA; Sun, 27 Jan 2019 19:04:30 +0000 (UTC) (envelope-from marius@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 5FBEE77953; Sun, 27 Jan 2019 19:04:30 +0000 (UTC) (envelope-from marius@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 54B00C513; Sun, 27 Jan 2019 19:04:30 +0000 (UTC) (envelope-from marius@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x0RJ4UnN085187; Sun, 27 Jan 2019 19:04:30 GMT (envelope-from marius@FreeBSD.org) Received: (from marius@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x0RJ4Sfe085179; Sun, 27 Jan 2019 19:04:28 GMT (envelope-from marius@FreeBSD.org) Message-Id: <201901271904.x0RJ4Sfe085179@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: marius set sender to marius@FreeBSD.org using -f From: Marius Strobl Date: Sun, 27 Jan 2019 19:04:28 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r343504 - in stable/11/sys: arm/broadcom/bcm2835 arm/nvidia arm/ti dev/sdhci X-SVN-Group: stable-11 X-SVN-Commit-Author: marius X-SVN-Commit-Paths: in stable/11/sys: arm/broadcom/bcm2835 arm/nvidia arm/ti dev/sdhci X-SVN-Commit-Revision: 343504 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 5FBEE77953 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.998,0]; NEURAL_HAM_SHORT(-0.97)[-0.975,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-stable@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for all the -stable branches of the src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Jan 2019 19:04:31 -0000 Author: marius Date: Sun Jan 27 19:04:28 2019 New Revision: 343504 URL: https://svnweb.freebsd.org/changeset/base/343504 Log: MFC: r342634 (partial) o Don't allocate resources for SDMA in sdhci(4) if the controller or the front-end doesn't support SDMA or the latter implements a platform- specific transfer method instead. While at it, factor out allocation and freeing of SDMA resources to sdhci_dma_{alloc,free}() in order to keep the code more readable when adding support for ADMA variants. o Base the size of the SDMA bounce buffer on MAXPHYS up to the maximum of 512 KiB instead of using a fixed 4-KiB-buffer. With the default MAXPHYS of 128 KiB and depending on the controller and medium, this reduces the number of SDHCI interrupts by a factor of ~16 to ~32 on sequential reads while an increase of throughput of up to ~84 % was seen. Front-ends for broken controllers that only support an SDMA buffer boundary of a specific size may set SDHCI_QUIRK_BROKEN_SDMA_BOUNDARY and supply a size via struct sdhci_slot. According to Linux, only - unsupported in stable/11 anyway - Qualcomm MSM-type SDHCI controllers are affected by this, though. Requested by: Shreyank Amartya (unconditional bump to 512 KiB) o Introduce a SDHCI_DEPEND macro for specifying the dependency of the front-end modules on the sdhci(4) one and bump the module version of sdhci(4) to 2 via an also newly introduced SDHCI_VERSION in order to ensure that all components are in sync WRT struct sdhci_slot. o In sdhci(4): - Make pointers const were applicable, and - replace a few device_printf(9) calls with slot_printf() for consistency. Modified: stable/11/sys/arm/broadcom/bcm2835/bcm2835_sdhci.c stable/11/sys/arm/nvidia/tegra_sdhci.c stable/11/sys/arm/ti/ti_sdhci.c stable/11/sys/dev/sdhci/fsl_sdhci.c stable/11/sys/dev/sdhci/sdhci.c stable/11/sys/dev/sdhci/sdhci.h stable/11/sys/dev/sdhci/sdhci_acpi.c stable/11/sys/dev/sdhci/sdhci_fdt.c stable/11/sys/dev/sdhci/sdhci_pci.c Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/arm/broadcom/bcm2835/bcm2835_sdhci.c ============================================================================== --- stable/11/sys/arm/broadcom/bcm2835/bcm2835_sdhci.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/arm/broadcom/bcm2835/bcm2835_sdhci.c Sun Jan 27 19:04:28 2019 (r343504) @@ -679,5 +679,5 @@ static driver_t bcm_sdhci_driver = { DRIVER_MODULE(sdhci_bcm, simplebus, bcm_sdhci_driver, bcm_sdhci_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_bcm, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_bcm); MMC_DECLARE_BRIDGE(sdhci_bcm); Modified: stable/11/sys/arm/nvidia/tegra_sdhci.c ============================================================================== --- stable/11/sys/arm/nvidia/tegra_sdhci.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/arm/nvidia/tegra_sdhci.c Sun Jan 27 19:04:28 2019 (r343504) @@ -463,5 +463,5 @@ static DEFINE_CLASS_0(sdhci, tegra_sdhci_driver, tegra sizeof(struct tegra_sdhci_softc)); DRIVER_MODULE(sdhci_tegra, simplebus, tegra_sdhci_driver, tegra_sdhci_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_tegra, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_tegra); MMC_DECLARE_BRIDGE(sdhci); Modified: stable/11/sys/arm/ti/ti_sdhci.c ============================================================================== --- stable/11/sys/arm/ti/ti_sdhci.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/arm/ti/ti_sdhci.c Sun Jan 27 19:04:28 2019 (r343504) @@ -729,5 +729,5 @@ static driver_t ti_sdhci_driver = { DRIVER_MODULE(sdhci_ti, simplebus, ti_sdhci_driver, ti_sdhci_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_ti, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_ti); MMC_DECLARE_BRIDGE(sdhci_ti); Modified: stable/11/sys/dev/sdhci/fsl_sdhci.c ============================================================================== --- stable/11/sys/dev/sdhci/fsl_sdhci.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/dev/sdhci/fsl_sdhci.c Sun Jan 27 19:04:28 2019 (r343504) @@ -1009,5 +1009,5 @@ static driver_t fsl_sdhci_driver = { DRIVER_MODULE(sdhci_fsl, simplebus, fsl_sdhci_driver, fsl_sdhci_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_fsl, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_fsl); MMC_DECLARE_BRIDGE(sdhci_fsl); Modified: stable/11/sys/dev/sdhci/sdhci.c ============================================================================== --- stable/11/sys/dev/sdhci/sdhci.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/dev/sdhci/sdhci.c Sun Jan 27 19:04:28 2019 (r343504) @@ -34,6 +34,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include #include @@ -98,22 +99,25 @@ static void sdhci_retune(void *arg); static void sdhci_set_clock(struct sdhci_slot *slot, uint32_t clock); static void sdhci_set_power(struct sdhci_slot *slot, u_char power); static void sdhci_set_transfer_mode(struct sdhci_slot *slot, - struct mmc_data *data); + const struct mmc_data *data); static void sdhci_start(struct sdhci_slot *slot); static void sdhci_timeout(void *arg); static void sdhci_start_command(struct sdhci_slot *slot, struct mmc_command *cmd); -static void sdhci_start_data(struct sdhci_slot *slot, struct mmc_data *data); +static void sdhci_start_data(struct sdhci_slot *slot, + const struct mmc_data *data); static void sdhci_write_block_pio(struct sdhci_slot *slot); static void sdhci_transfer_pio(struct sdhci_slot *slot); /* helper routines */ +static int sdhci_dma_alloc(struct sdhci_slot *slot); +static void sdhci_dma_free(struct sdhci_slot *slot); static void sdhci_dumpregs(struct sdhci_slot *slot); static void sdhci_getaddr(void *arg, bus_dma_segment_t *segs, int nsegs, int error); -static int slot_printf(struct sdhci_slot *slot, const char * fmt, ...) +static int slot_printf(const struct sdhci_slot *slot, const char * fmt, ...) __printflike(2, 3); -static uint32_t sdhci_tuning_intmask(struct sdhci_slot *slot); +static uint32_t sdhci_tuning_intmask(const struct sdhci_slot *slot); #define SDHCI_LOCK(_slot) mtx_lock(&(_slot)->mtx) #define SDHCI_UNLOCK(_slot) mtx_unlock(&(_slot)->mtx) @@ -157,7 +161,7 @@ sdhci_getaddr(void *arg, bus_dma_segment_t *segs, int } static int -slot_printf(struct sdhci_slot *slot, const char * fmt, ...) +slot_printf(const struct sdhci_slot *slot, const char * fmt, ...) { va_list ap; int retval; @@ -268,7 +272,7 @@ sdhci_reset(struct sdhci_slot *slot, uint8_t mask) } static uint32_t -sdhci_tuning_intmask(struct sdhci_slot *slot) +sdhci_tuning_intmask(const struct sdhci_slot *slot) { uint32_t intmask; @@ -649,54 +653,93 @@ sdhci_card_poll(void *arg) sdhci_card_poll, slot); } -int -sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num) +static int +sdhci_dma_alloc(struct sdhci_slot *slot) { - kobjop_desc_t kobj_desc; - kobj_method_t *kobj_method; - uint32_t caps, caps2, freq, host_caps; int err; - SDHCI_LOCK_INIT(slot); - slot->num = num; - slot->bus = dev; - - /* Allocate DMA tag. */ - err = bus_dma_tag_create(bus_get_dma_tag(dev), - DMA_BLOCK_SIZE, 0, BUS_SPACE_MAXADDR_32BIT, - BUS_SPACE_MAXADDR, NULL, NULL, - DMA_BLOCK_SIZE, 1, DMA_BLOCK_SIZE, - BUS_DMA_ALLOCNOW, NULL, NULL, - &slot->dmatag); + if (!(slot->quirks & SDHCI_QUIRK_BROKEN_SDMA_BOUNDARY)) { + if (MAXPHYS <= 1024 * 4) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_4K; + else if (MAXPHYS <= 1024 * 8) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_8K; + else if (MAXPHYS <= 1024 * 16) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_16K; + else if (MAXPHYS <= 1024 * 32) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_32K; + else if (MAXPHYS <= 1024 * 64) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_64K; + else if (MAXPHYS <= 1024 * 128) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_128K; + else if (MAXPHYS <= 1024 * 256) + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_256K; + else + slot->sdma_boundary = SDHCI_BLKSZ_SDMA_BNDRY_512K; + } + slot->sdma_bbufsz = SDHCI_SDMA_BNDRY_TO_BBUFSZ(slot->sdma_boundary); + /* + * Allocate the DMA tag for an SDMA bounce buffer. + * Note that the SDHCI specification doesn't state any alignment + * constraint for the SDMA system address. However, controllers + * typically ignore the SDMA boundary bits in SDHCI_DMA_ADDRESS when + * forming the actual address of data, requiring the SDMA buffer to + * be aligned to the SDMA boundary. + */ + err = bus_dma_tag_create(bus_get_dma_tag(slot->bus), slot->sdma_bbufsz, + 0, BUS_SPACE_MAXADDR_32BIT, BUS_SPACE_MAXADDR, NULL, NULL, + slot->sdma_bbufsz, 1, slot->sdma_bbufsz, BUS_DMA_ALLOCNOW, + NULL, NULL, &slot->dmatag); if (err != 0) { - device_printf(dev, "Can't create DMA tag\n"); - SDHCI_LOCK_DESTROY(slot); + slot_printf(slot, "Can't create DMA tag for SDMA\n"); return (err); } - /* Allocate DMA memory. */ + /* Allocate DMA memory for the SDMA bounce buffer. */ err = bus_dmamem_alloc(slot->dmatag, (void **)&slot->dmamem, BUS_DMA_NOWAIT, &slot->dmamap); if (err != 0) { - device_printf(dev, "Can't alloc DMA memory\n"); + slot_printf(slot, "Can't alloc DMA memory for SDMA\n"); bus_dma_tag_destroy(slot->dmatag); - SDHCI_LOCK_DESTROY(slot); return (err); } - /* Map the memory. */ + /* Map the memory of the SDMA bounce buffer. */ err = bus_dmamap_load(slot->dmatag, slot->dmamap, - (void *)slot->dmamem, DMA_BLOCK_SIZE, - sdhci_getaddr, &slot->paddr, 0); + (void *)slot->dmamem, slot->sdma_bbufsz, sdhci_getaddr, + &slot->paddr, 0); if (err != 0 || slot->paddr == 0) { - device_printf(dev, "Can't load DMA memory\n"); + slot_printf(slot, "Can't load DMA memory for SDMA\n"); bus_dmamem_free(slot->dmatag, slot->dmamem, slot->dmamap); bus_dma_tag_destroy(slot->dmatag); - SDHCI_LOCK_DESTROY(slot); if (err) return (err); else return (EFAULT); } + return (0); +} + +static void +sdhci_dma_free(struct sdhci_slot *slot) +{ + + bus_dmamap_unload(slot->dmatag, slot->dmamap); + bus_dmamem_free(slot->dmatag, slot->dmamem, slot->dmamap); + bus_dma_tag_destroy(slot->dmatag); +} + +int +sdhci_init_slot(device_t dev, struct sdhci_slot *slot, int num) +{ + kobjop_desc_t kobj_desc; + kobj_method_t *kobj_method; + uint32_t caps, caps2, freq, host_caps; + int err; + + SDHCI_LOCK_INIT(slot); + + slot->num = num; + slot->bus = dev; + slot->version = (RD2(slot, SDHCI_HOST_VERSION) >> SDHCI_SPEC_VER_SHIFT) & SDHCI_SPEC_VER_MASK; if (slot->quirks & SDHCI_QUIRK_MISSING_CAPS) { @@ -712,12 +755,8 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, if (slot->version >= SDHCI_SPEC_300) { if ((caps & SDHCI_SLOTTYPE_MASK) != SDHCI_SLOTTYPE_REMOVABLE && (caps & SDHCI_SLOTTYPE_MASK) != SDHCI_SLOTTYPE_EMBEDDED) { - device_printf(dev, + slot_printf(slot, "Driver doesn't support shared bus slots\n"); - bus_dmamap_unload(slot->dmatag, slot->dmamap); - bus_dmamem_free(slot->dmatag, slot->dmamem, - slot->dmamap); - bus_dma_tag_destroy(slot->dmatag); SDHCI_LOCK_DESTROY(slot); return (ENXIO); } else if ((caps & SDHCI_SLOTTYPE_MASK) == @@ -741,7 +780,7 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, */ if (slot->max_clk == 0) { slot->max_clk = SDHCI_DEFAULT_MAX_FREQ * 1000000; - device_printf(dev, "Hardware doesn't specify base clock " + slot_printf(slot, "Hardware doesn't specify base clock " "frequency, using %dMHz as default.\n", SDHCI_DEFAULT_MAX_FREQ); } @@ -762,7 +801,7 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, * max timeout, but still mention it. */ if (slot->timeout_clk == 0) { - device_printf(dev, "Hardware doesn't specify timeout clock " + slot_printf(slot, "Hardware doesn't specify timeout clock " "frequency, setting BROKEN_TIMEOUT quirk.\n"); slot->quirks |= SDHCI_QUIRK_BROKEN_TIMEOUT_VAL; } @@ -778,7 +817,7 @@ sdhci_init_slot(device_t dev, struct sdhci_slot *slot, if ((caps & SDHCI_CAN_VDD_180) && (slot->opt & SDHCI_SLOT_EMBEDDED)) slot->host.host_ocr |= MMC_OCR_LOW_VOLTAGE; if (slot->host.host_ocr == 0) { - device_printf(dev, "Hardware doesn't report any " + slot_printf(slot, "Hardware doesn't report any " "support voltages.\n"); } @@ -864,7 +903,7 @@ no_tuning: slot->retune_count = (caps2 & SDHCI_RETUNE_CNT_MASK) >> SDHCI_RETUNE_CNT_SHIFT; if (slot->retune_count > 0xb) { - device_printf(dev, "Unknown re-tuning count " + slot_printf(slot, "Unknown re-tuning count " "%x, using 1 sec\n", slot->retune_count); slot->retune_count = 1; } else if (slot->retune_count != 0) @@ -923,6 +962,19 @@ no_tuning: if (slot->opt & SDHCI_PLATFORM_TRANSFER) slot->opt &= ~SDHCI_HAVE_DMA; + if (slot->opt & SDHCI_HAVE_DMA) { + err = sdhci_dma_alloc(slot); + if (err != 0) { + if (slot->opt & SDHCI_TUNING_SUPPORTED) { + free(slot->tune_req, M_DEVBUF); + free(slot->tune_cmd, M_DEVBUF); + free(slot->tune_data, M_DEVBUF); + } + SDHCI_LOCK_DESTROY(slot); + return (err); + } + } + if (bootverbose || sdhci_debug) { slot_printf(slot, "%uMHz%s %s VDD:%s%s%s VCCQ: 3.3V%s%s DRV: B%s%s%s %s %s\n", @@ -1018,9 +1070,8 @@ sdhci_cleanup_slot(struct sdhci_slot *slot) SDHCI_LOCK(slot); sdhci_reset(slot, SDHCI_RESET_ALL); SDHCI_UNLOCK(slot); - bus_dmamap_unload(slot->dmatag, slot->dmamap); - bus_dmamem_free(slot->dmatag, slot->dmamem, slot->dmamap); - bus_dma_tag_destroy(slot->dmatag); + if (slot->opt & SDHCI_HAVE_DMA) + sdhci_dma_free(slot); if (slot->opt & SDHCI_TUNING_SUPPORTED) { free(slot->tune_req, M_DEVBUF); free(slot->tune_cmd, M_DEVBUF); @@ -1084,7 +1135,7 @@ sdhci_generic_get_card_present(device_t brdev __unused void sdhci_generic_set_uhs_timing(device_t brdev __unused, struct sdhci_slot *slot) { - struct mmc_ios *ios; + const struct mmc_ios *ios; uint16_t hostctrl2; if (slot->version < SDHCI_SPEC_300) @@ -1217,7 +1268,7 @@ int sdhci_generic_tune(device_t brdev __unused, device_t reqdev, bool hs400) { struct sdhci_slot *slot = device_get_ivars(reqdev); - struct mmc_ios *ios = &slot->host.ios; + const struct mmc_ios *ios = &slot->host.ios; struct mmc_command *tune_cmd; struct mmc_data *tune_data; uint32_t opcode; @@ -1448,7 +1499,7 @@ sdhci_timeout(void *arg) } static void -sdhci_set_transfer_mode(struct sdhci_slot *slot, struct mmc_data *data) +sdhci_set_transfer_mode(struct sdhci_slot *slot, const struct mmc_data *data) { uint16_t mode; @@ -1628,9 +1679,9 @@ sdhci_finish_command(struct sdhci_slot *slot) } static void -sdhci_start_data(struct sdhci_slot *slot, struct mmc_data *data) +sdhci_start_data(struct sdhci_slot *slot, const struct mmc_data *data) { - uint32_t target_timeout, current_timeout; + uint32_t blkcnt, blksz, current_timeout, sdma_bbufsz, target_timeout; uint8_t div; if (data == NULL && (slot->curcmd->flags & MMC_RSP_BUSY) == 0) { @@ -1666,7 +1717,7 @@ sdhci_start_data(struct sdhci_slot *slot, struct mmc_d /* Use DMA if possible. */ if ((slot->opt & SDHCI_HAVE_DMA)) slot->flags |= SDHCI_USE_DMA; - /* If data is small, broken DMA may return zeroes instead of data, */ + /* If data is small, broken DMA may return zeroes instead of data. */ if ((slot->quirks & SDHCI_QUIRK_BROKEN_TIMINGS) && (data->len <= 512)) slot->flags &= ~SDHCI_USE_DMA; @@ -1676,20 +1727,22 @@ sdhci_start_data(struct sdhci_slot *slot, struct mmc_d slot->flags &= ~SDHCI_USE_DMA; /* Load DMA buffer. */ if (slot->flags & SDHCI_USE_DMA) { + sdma_bbufsz = slot->sdma_bbufsz; if (data->flags & MMC_DATA_READ) bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_PREREAD); else { - memcpy(slot->dmamem, data->data, - (data->len < DMA_BLOCK_SIZE) ? - data->len : DMA_BLOCK_SIZE); + memcpy(slot->dmamem, data->data, ulmin(data->len, + sdma_bbufsz)); bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_PREWRITE); } WR4(slot, SDHCI_DMA_ADDRESS, slot->paddr); - /* Interrupt aggregation: Mask border interrupt - * for the last page and unmask else. */ - if (data->len == DMA_BLOCK_SIZE) + /* + * Interrupt aggregation: Mask border interrupt for the last + * bounce buffer and unmask otherwise. + */ + if (data->len == sdma_bbufsz) slot->intmask &= ~SDHCI_INT_DMA_END; else slot->intmask |= SDHCI_INT_DMA_END; @@ -1697,11 +1750,12 @@ sdhci_start_data(struct sdhci_slot *slot, struct mmc_d } /* Current data offset for both PIO and DMA. */ slot->offset = 0; - /* Set block size and request IRQ on 4K border. */ - WR2(slot, SDHCI_BLOCK_SIZE, SDHCI_MAKE_BLKSZ(DMA_BOUNDARY, - (data->len < 512) ? data->len : 512)); + /* Set block size and request border interrupts on the SDMA boundary. */ + blksz = SDHCI_MAKE_BLKSZ(slot->sdma_boundary, ulmin(data->len, 512)); + WR2(slot, SDHCI_BLOCK_SIZE, blksz); /* Set block count. */ - WR2(slot, SDHCI_BLOCK_COUNT, (data->len + 511) / 512); + blkcnt = howmany(data->len, 512); + WR2(slot, SDHCI_BLOCK_COUNT, blkcnt); } void @@ -1725,7 +1779,7 @@ sdhci_finish_data(struct sdhci_slot *slot) bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_POSTREAD); memcpy((u_char*)data->data + slot->offset, slot->dmamem, - (left < DMA_BLOCK_SIZE) ? left : DMA_BLOCK_SIZE); + ulmin(left, slot->sdma_bbufsz)); } else bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_POSTWRITE); @@ -1748,7 +1802,7 @@ sdhci_finish_data(struct sdhci_slot *slot) static void sdhci_start(struct sdhci_slot *slot) { - struct mmc_request *req; + const struct mmc_request *req; req = slot->req; if (req == NULL) @@ -1876,6 +1930,7 @@ sdhci_data_irq(struct sdhci_slot *slot, uint32_t intma { struct mmc_data *data; size_t left; + uint32_t sdma_bbufsz; if (!slot->curcmd) { slot_printf(slot, "Got data interrupt 0x%08x, but " @@ -1930,6 +1985,7 @@ sdhci_data_irq(struct sdhci_slot *slot, uint32_t intma /* Handle DMA border. */ if (intmask & SDHCI_INT_DMA_END) { data = slot->curcmd->data; + sdma_bbufsz = slot->sdma_bbufsz; /* Unload DMA buffer ... */ left = data->len - slot->offset; @@ -1937,26 +1993,28 @@ sdhci_data_irq(struct sdhci_slot *slot, uint32_t intma bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_POSTREAD); memcpy((u_char*)data->data + slot->offset, slot->dmamem, - (left < DMA_BLOCK_SIZE) ? left : DMA_BLOCK_SIZE); + ulmin(left, sdma_bbufsz)); } else { bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_POSTWRITE); } /* ... and reload it again. */ - slot->offset += DMA_BLOCK_SIZE; + slot->offset += sdma_bbufsz; left = data->len - slot->offset; if (data->flags & MMC_DATA_READ) { bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_PREREAD); } else { memcpy(slot->dmamem, (u_char*)data->data + slot->offset, - (left < DMA_BLOCK_SIZE)? left : DMA_BLOCK_SIZE); + ulmin(left, sdma_bbufsz)); bus_dmamap_sync(slot->dmatag, slot->dmamap, BUS_DMASYNC_PREWRITE); } - /* Interrupt aggregation: Mask border interrupt - * for the last page. */ - if (left == DMA_BLOCK_SIZE) { + /* + * Interrupt aggregation: Mask border interrupt for the last + * bounce buffer. + */ + if (left == sdma_bbufsz) { slot->intmask &= ~SDHCI_INT_DMA_END; WR4(slot, SDHCI_SIGNAL_ENABLE, slot->intmask); } @@ -2079,7 +2137,7 @@ int sdhci_generic_read_ivar(device_t bus, device_t child, int which, uintptr_t *result) { - struct sdhci_slot *slot = device_get_ivars(child); + const struct sdhci_slot *slot = device_get_ivars(child); switch (which) { default: @@ -2236,4 +2294,4 @@ sdhci_generic_write_ivar(device_t bus, device_t child, return (0); } -MODULE_VERSION(sdhci, 1); +MODULE_VERSION(sdhci, SDHCI_VERSION); Modified: stable/11/sys/dev/sdhci/sdhci.h ============================================================================== --- stable/11/sys/dev/sdhci/sdhci.h Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/dev/sdhci/sdhci.h Sun Jan 27 19:04:28 2019 (r343504) @@ -28,8 +28,8 @@ #ifndef __SDHCI_H__ #define __SDHCI_H__ -#define DMA_BLOCK_SIZE 4096 -#define DMA_BOUNDARY 0 /* DMA reload every 4K */ +/* Macro for sizing the SDMA bounce buffer on the SDMA buffer boundary. */ +#define SDHCI_SDMA_BNDRY_TO_BBUFSZ(bndry) (4096 * (1 << bndry)) /* Controller doesn't honor resets unless we touch the clock register */ #define SDHCI_QUIRK_CLOCK_BEFORE_RESET (1 << 0) @@ -91,6 +91,8 @@ #define SDHCI_QUIRK_BROKEN_AUTO_STOP (1 << 28) /* Controller supports eMMC HS400 mode if SDHCI_CAN_SDR104 is set. */ #define SDHCI_QUIRK_MMC_HS400_IF_CAN_SDR104 (1 << 29) +/* SDMA boundary in SDHCI_BLOCK_SIZE broken - use front-end supplied value. */ +#define SDHCI_QUIRK_BROKEN_SDMA_BOUNDARY (1 << 30) /* * Controller registers @@ -98,6 +100,14 @@ #define SDHCI_DMA_ADDRESS 0x00 #define SDHCI_BLOCK_SIZE 0x04 +#define SDHCI_BLKSZ_SDMA_BNDRY_4K 0x00 +#define SDHCI_BLKSZ_SDMA_BNDRY_8K 0x01 +#define SDHCI_BLKSZ_SDMA_BNDRY_16K 0x02 +#define SDHCI_BLKSZ_SDMA_BNDRY_32K 0x03 +#define SDHCI_BLKSZ_SDMA_BNDRY_64K 0x04 +#define SDHCI_BLKSZ_SDMA_BNDRY_128K 0x05 +#define SDHCI_BLKSZ_SDMA_BNDRY_256K 0x06 +#define SDHCI_BLKSZ_SDMA_BNDRY_512K 0x07 #define SDHCI_MAKE_BLKSZ(dma, blksz) (((dma & 0x7) << 12) | (blksz & 0xFFF)) #define SDHCI_BLOCK_COUNT 0x06 @@ -358,6 +368,8 @@ struct sdhci_slot { bus_dmamap_t dmamap; u_char *dmamem; bus_addr_t paddr; /* DMA buffer address */ + uint32_t sdma_bbufsz; /* SDMA bounce buffer size */ + uint8_t sdma_boundary; /* SDMA boundary */ struct task card_task; /* Card presence check task */ struct timeout_task card_delayed_task;/* Card insert delayed task */ @@ -420,5 +432,10 @@ uint32_t sdhci_generic_min_freq(device_t brdev, struct bool sdhci_generic_get_card_present(device_t brdev, struct sdhci_slot *slot); void sdhci_generic_set_uhs_timing(device_t brdev, struct sdhci_slot *slot); void sdhci_handle_card_present(struct sdhci_slot *slot, bool is_present); + +#define SDHCI_VERSION 2 + +#define SDHCI_DEPEND(name) \ + MODULE_DEPEND(name, sdhci, SDHCI_VERSION, SDHCI_VERSION, SDHCI_VERSION); #endif /* __SDHCI_H__ */ Modified: stable/11/sys/dev/sdhci/sdhci_acpi.c ============================================================================== --- stable/11/sys/dev/sdhci/sdhci_acpi.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/dev/sdhci/sdhci_acpi.c Sun Jan 27 19:04:28 2019 (r343504) @@ -456,5 +456,5 @@ static devclass_t sdhci_acpi_devclass; DRIVER_MODULE(sdhci_acpi, acpi, sdhci_acpi_driver, sdhci_acpi_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_acpi, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_acpi); MMC_DECLARE_BRIDGE(sdhci_acpi); Modified: stable/11/sys/dev/sdhci/sdhci_fdt.c ============================================================================== --- stable/11/sys/dev/sdhci/sdhci_fdt.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/dev/sdhci/sdhci_fdt.c Sun Jan 27 19:04:28 2019 (r343504) @@ -325,5 +325,5 @@ static devclass_t sdhci_fdt_devclass; DRIVER_MODULE(sdhci_fdt, simplebus, sdhci_fdt_driver, sdhci_fdt_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_fdt, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_fdt); MMC_DECLARE_BRIDGE(sdhci_fdt); Modified: stable/11/sys/dev/sdhci/sdhci_pci.c ============================================================================== --- stable/11/sys/dev/sdhci/sdhci_pci.c Sun Jan 27 19:04:02 2019 (r343503) +++ stable/11/sys/dev/sdhci/sdhci_pci.c Sun Jan 27 19:04:28 2019 (r343504) @@ -521,5 +521,5 @@ static devclass_t sdhci_pci_devclass; DRIVER_MODULE(sdhci_pci, pci, sdhci_pci_driver, sdhci_pci_devclass, NULL, NULL); -MODULE_DEPEND(sdhci_pci, sdhci, 1, 1, 1); +SDHCI_DEPEND(sdhci_pci); MMC_DECLARE_BRIDGE(sdhci_pci);