From nobody Thu Feb 17 13:40:25 2022 X-Original-To: dev-commits-src-all@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 8EC8619D40F0; Thu, 17 Feb 2022 13:40:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Jzwtx3Kl8z3NXf; Thu, 17 Feb 2022 13:40:25 +0000 (UTC) (envelope-from git@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645105225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ThDBEu5ChXUr+zGRlQ6Qn4QpFBi+4KxsU3nGUiQrUFU=; b=IVceVTCGBh9Iqoem//q/QUjefLBP0NvphOD0dbj+i+GH6pTjqS+oFgdK/ThsF97akwMuom nYm1uY5w6GBWpxk8H7FNpqwLe0nUwtYGP5wY/lw2+XbNYkz2HNAc+UDiGHbueoJrd/x0In CUWjiBB34bf7zi3AvHGz9qZBSHaxTWXC8Dagy/SSGWPgJE7endBy1p5e+w+o7LUa27SY4O MyZJIpJOJPjLo+gg99FFOiS04clbYpviCly1z/c1PPBVjd6mv4XvtZpecygw8vXNz2Ppu4 BHnx+awfkN+WOJg4chOOyc2MDRkScAidq5Nw5Qyxs6bv2F6PqJR1EqN0HeiM0A== Received: from gitrepo.freebsd.org (gitrepo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:5]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 537632B21; Thu, 17 Feb 2022 13:40:25 +0000 (UTC) (envelope-from git@FreeBSD.org) Received: from gitrepo.freebsd.org ([127.0.1.44]) by gitrepo.freebsd.org (8.16.1/8.16.1) with ESMTP id 21HDeP5B043377; Thu, 17 Feb 2022 13:40:25 GMT (envelope-from git@gitrepo.freebsd.org) Received: (from git@localhost) by gitrepo.freebsd.org (8.16.1/8.16.1/Submit) id 21HDePG0043376; Thu, 17 Feb 2022 13:40:25 GMT (envelope-from git) Date: Thu, 17 Feb 2022 13:40:25 GMT Message-Id: <202202171340.21HDePG0043376@gitrepo.freebsd.org> To: src-committers@FreeBSD.org, dev-commits-src-all@FreeBSD.org, dev-commits-src-main@FreeBSD.org From: "Bjoern A. Zeeb" Subject: git: 95edb10b47fc - main - LinuxKPI: implement dma_sync_single_for_*, apply to (un)map single/sg List-Id: Commit messages for all branches of the src repository List-Archive: https://lists.freebsd.org/archives/dev-commits-src-all List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-dev-commits-src-all@freebsd.org X-BeenThere: dev-commits-src-all@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Git-Committer: bz X-Git-Repository: src X-Git-Refname: refs/heads/main X-Git-Reftype: branch X-Git-Commit: 95edb10b47fc1a919cd1687aaf16be9e14456c89 Auto-Submitted: auto-generated ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1645105225; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=ThDBEu5ChXUr+zGRlQ6Qn4QpFBi+4KxsU3nGUiQrUFU=; b=VJRSfZnkwripQu4Gck5PbxMVFKGKmyA19Muo7QoEJv81M0mj9UDGuz6LhtVy1oKl1NLzCe GaisVBB4VwNNw0PfhAsavia9LkfA675oUKIIHhJPqfBMwMZRX3efnpjFoqjn2A5ceGY/Gi eC2g0emUxcP5/kLgynN+6nyHLIhOYuKzp2b5ax6i5TjklztrABsXT3WasbNGTpDTsFSARW P7yapTkml+uohcJKsxMasVA/p4Mwm+wSb97cwt1RToBMD5osQFS+/MkaYZI/k0EitwX1XZ n75rWC4h61SYBxvuBXutveHb2P6TLrdq2VzOPS7OevAeiSXnPy4FLYsYvC3F1w== ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1645105225; a=rsa-sha256; cv=none; b=ufxr99rTMYzjdHZn6FzNgAibqgzcYluT89VbKY++yWuHsWON00vGwA1M6iVsxV/pGu7dRv aNTXLMk1+BnyK3emnsC1hoRrVYbBXzNQ4ZbXi9O89NM0H55XWzTLXhvCnnvWe/pe8+Wi07 2XR8c/fSQBwJMZ476tMcBt2pgeDz+/hRKxvrk6IiYL86L5yskgJI5kY5NUd0eJHrN9pcbT SEUyzrSdFeVm08MZWKJgQztLIfwGi44L3jFs6+VGq7e6qzqLMOzxc7HARMxRI1NZxkS6DH l30za96/Whk6mNy8lNpG2w7zlAgYPFfdHY/emaXSm0E3Prx41DTzjcplebYqug== ARC-Authentication-Results: i=1; mx1.freebsd.org; none X-ThisMailContainsUnwantedMimeParts: N The branch main has been updated by bz: URL: https://cgit.FreeBSD.org/src/commit/?id=95edb10b47fc1a919cd1687aaf16be9e14456c89 commit 95edb10b47fc1a919cd1687aaf16be9e14456c89 Author: Bjoern A. Zeeb AuthorDate: 2021-10-01 10:51:50 +0000 Commit: Bjoern A. Zeeb CommitDate: 2022-02-17 13:39:15 +0000 LinuxKPI: implement dma_sync_single_for_*, apply to (un)map single/sg Implement dma_sync_single_for_{cpu,device} translating the Linux DMA_ flags to BUS_DMASYNC_ combinations. Make map_single/unmap_single* functions call the respective sync function. Apply the same logic to the scatter-gather list map/unmap functions. Sponsored by: The FreeBSD Foundation MFC after: 3 days Reviewed by: hselasky Differential Revision: https://reviews.freebsd.org/D32255 --- .../linuxkpi/common/include/linux/dma-mapping.h | 75 +++++++++++++++++++--- sys/compat/linuxkpi/common/src/linux_pci.c | 60 ++++++++++++++++- 2 files changed, 124 insertions(+), 11 deletions(-) diff --git a/sys/compat/linuxkpi/common/include/linux/dma-mapping.h b/sys/compat/linuxkpi/common/include/linux/dma-mapping.h index f80258892574..fcb50f4b5217 100644 --- a/sys/compat/linuxkpi/common/include/linux/dma-mapping.h +++ b/sys/compat/linuxkpi/common/include/linux/dma-mapping.h @@ -103,6 +103,7 @@ int linux_dma_map_sg_attrs(struct device *dev, struct scatterlist *sgl, void linux_dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sg, int nents __unused, enum dma_data_direction dir __unused, unsigned long attrs __unused); +void linuxkpi_dma_sync(struct device *, dma_addr_t, size_t, bus_dmasync_op_t); static inline int dma_supported(struct device *dev, u64 dma_mask) @@ -167,12 +168,6 @@ dma_free_coherent(struct device *dev, size_t size, void *cpu_addr, kmem_free((vm_offset_t)cpu_addr, size); } -#define dma_map_single_attrs(dev, ptr, size, dir, attrs) \ - linux_dma_map_phys(dev, vtophys(ptr), size) - -#define dma_unmap_single_attrs(dev, dma_addr, size, dir, attrs) \ - linux_dma_unmap(dev, dma_addr, size) - static inline dma_addr_t dma_map_page_attrs(struct device *dev, struct page *page, size_t offset, size_t size, enum dma_data_direction dir, unsigned long attrs) @@ -205,9 +200,28 @@ dma_unmap_page(struct device *dev, dma_addr_t dma_address, size_t size, } static inline void -dma_sync_single_for_cpu(struct device *dev, dma_addr_t dma_handle, size_t size, +dma_sync_single_for_cpu(struct device *dev, dma_addr_t dma, size_t size, enum dma_data_direction direction) { + bus_dmasync_op_t op; + + switch (direction) { + case DMA_BIDIRECTIONAL: + op = BUS_DMASYNC_POSTREAD; + linuxkpi_dma_sync(dev, dma, size, op); + op = BUS_DMASYNC_PREREAD; + break; + case DMA_TO_DEVICE: + op = BUS_DMASYNC_POSTWRITE; + break; + case DMA_FROM_DEVICE: + op = BUS_DMASYNC_POSTREAD; + break; + default: + return; + } + + linuxkpi_dma_sync(dev, dma, size, op); } static inline void @@ -218,9 +232,26 @@ dma_sync_single(struct device *dev, dma_addr_t addr, size_t size, } static inline void -dma_sync_single_for_device(struct device *dev, dma_addr_t dma_handle, +dma_sync_single_for_device(struct device *dev, dma_addr_t dma, size_t size, enum dma_data_direction direction) { + bus_dmasync_op_t op; + + switch (direction) { + case DMA_BIDIRECTIONAL: + op = BUS_DMASYNC_PREWRITE; + break; + case DMA_TO_DEVICE: + op = BUS_DMASYNC_PREREAD; + break; + case DMA_FROM_DEVICE: + op = BUS_DMASYNC_PREWRITE; + break; + default: + return; + } + + linuxkpi_dma_sync(dev, dma, size, op); } static inline void @@ -260,6 +291,34 @@ static inline unsigned int dma_set_max_seg_size(struct device *dev, return (0); } +static inline dma_addr_t +_dma_map_single_attrs(struct device *dev, void *ptr, size_t size, + enum dma_data_direction direction, unsigned long attrs __unused) +{ + dma_addr_t dma; + + dma = linux_dma_map_phys(dev, vtophys(ptr), size); + if (!dma_mapping_error(dev, dma)) + dma_sync_single_for_device(dev, dma, size, direction); + + return (dma); +} + +static inline void +_dma_unmap_single_attrs(struct device *dev, dma_addr_t dma, size_t size, + enum dma_data_direction direction, unsigned long attrs __unused) +{ + + dma_sync_single_for_cpu(dev, dma, size, direction); + linux_dma_unmap(dev, dma, size); +} + +#define dma_map_single_attrs(dev, ptr, size, dir, attrs) \ + _dma_map_single_attrs(dev, ptr, size, dir, 0) + +#define dma_unmap_single_attrs(dev, dma_addr, size, dir, attrs) \ + _dma_unmap_single_attrs(dev, dma_addr, size, dir, 0) + #define dma_map_single(d, a, s, r) dma_map_single_attrs(d, a, s, r, 0) #define dma_unmap_single(d, a, s, r) dma_unmap_single_attrs(d, a, s, r, 0) #define dma_map_sg(d, s, n, r) dma_map_sg_attrs(d, s, n, r, 0) diff --git a/sys/compat/linuxkpi/common/src/linux_pci.c b/sys/compat/linuxkpi/common/src/linux_pci.c index c388da87ac15..7ee5e2b2e06a 100644 --- a/sys/compat/linuxkpi/common/src/linux_pci.c +++ b/sys/compat/linuxkpi/common/src/linux_pci.c @@ -1,7 +1,7 @@ /*- * Copyright (c) 2015-2016 Mellanox Technologies, Ltd. * All rights reserved. - * Copyright (c) 2020-2021 The FreeBSD Foundation + * Copyright (c) 2020-2022 The FreeBSD Foundation * * Portions of this software were developed by Björn Zeeb * under sponsorship from the FreeBSD Foundation. @@ -957,9 +957,32 @@ linux_dma_alloc_coherent(struct device *dev, size_t size, return (mem); } +void +linuxkpi_dma_sync(struct device *dev, dma_addr_t dma_addr, size_t size, + bus_dmasync_op_t op) +{ + struct linux_dma_priv *priv; + struct linux_dma_obj *obj; + + priv = dev->dma_priv; + + if (pctrie_is_empty(&priv->ptree)) + return; + + DMA_PRIV_LOCK(priv); + obj = LINUX_DMA_PCTRIE_LOOKUP(&priv->ptree, dma_addr); + if (obj == NULL) { + DMA_PRIV_UNLOCK(priv); + return; + } + + bus_dmamap_sync(obj->dmat, obj->dmamap, op); + DMA_PRIV_UNLOCK(priv); +} + int linux_dma_map_sg_attrs(struct device *dev, struct scatterlist *sgl, int nents, - enum dma_data_direction dir __unused, unsigned long attrs __unused) + enum dma_data_direction direction, unsigned long attrs __unused) { struct linux_dma_priv *priv; struct scatterlist *sg; @@ -992,6 +1015,21 @@ linux_dma_map_sg_attrs(struct device *dev, struct scatterlist *sgl, int nents, sg_dma_address(sg) = seg.ds_addr; } + + switch (direction) { + case DMA_BIDIRECTIONAL: + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_PREWRITE); + break; + case DMA_TO_DEVICE: + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_PREREAD); + break; + case DMA_FROM_DEVICE: + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_PREWRITE); + break; + default: + break; + } + DMA_PRIV_UNLOCK(priv); return (nents); @@ -999,7 +1037,7 @@ linux_dma_map_sg_attrs(struct device *dev, struct scatterlist *sgl, int nents, void linux_dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sgl, - int nents __unused, enum dma_data_direction dir __unused, + int nents __unused, enum dma_data_direction direction, unsigned long attrs __unused) { struct linux_dma_priv *priv; @@ -1007,6 +1045,22 @@ linux_dma_unmap_sg_attrs(struct device *dev, struct scatterlist *sgl, priv = dev->dma_priv; DMA_PRIV_LOCK(priv); + + switch (direction) { + case DMA_BIDIRECTIONAL: + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_POSTREAD); + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_PREREAD); + break; + case DMA_TO_DEVICE: + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_POSTWRITE); + break; + case DMA_FROM_DEVICE: + bus_dmamap_sync(priv->dmat, sgl->dma_map, BUS_DMASYNC_POSTREAD); + break; + default: + break; + } + bus_dmamap_unload(priv->dmat, sgl->dma_map); bus_dmamap_destroy(priv->dmat, sgl->dma_map); DMA_PRIV_UNLOCK(priv);