From owner-svn-src-head@freebsd.org Wed Dec 9 22:45:53 2015 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D99519D6388; Wed, 9 Dec 2015 22:45:53 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 8807D13B3; Wed, 9 Dec 2015 22:45:53 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id tB9MjqWp052874; Wed, 9 Dec 2015 22:45:52 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id tB9Mjp5l052870; Wed, 9 Dec 2015 22:45:51 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201512092245.tB9Mjp5l052870@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: "Conrad E. Meyer" Date: Wed, 9 Dec 2015 22:45:51 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r292031 - in head: share/man/man4 sys/dev/ioat X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 09 Dec 2015 22:45:53 -0000 Author: cem Date: Wed Dec 9 22:45:51 2015 New Revision: 292031 URL: https://svnweb.freebsd.org/changeset/base/292031 Log: ioat(4): Add ioat_copy_8k_aligned KPI The hardware supports descriptors with two non-contiguous pages. This allows issuing one descriptor for an 8k copy from/to non-contiguous but otherwise page-aligned memory. Sponsored by: EMC / Isilon Storage Division Modified: head/share/man/man4/ioat.4 head/sys/dev/ioat/ioat.c head/sys/dev/ioat/ioat.h head/sys/dev/ioat/ioat_internal.h Modified: head/share/man/man4/ioat.4 ============================================================================== --- head/share/man/man4/ioat.4 Wed Dec 9 22:09:33 2015 (r292030) +++ head/share/man/man4/ioat.4 Wed Dec 9 22:45:51 2015 (r292031) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd October 31, 2015 +.Dd December 9, 2015 .Dt IOAT 4 .Os .Sh NAME @@ -78,6 +78,17 @@ In .Fa "uint32_t flags" .Fc .Ft struct bus_dmadesc * +.Fo ioat_copy_8k_aligned +.Fa "bus_dmaengine_t dmaengine" +.Fa "bus_addr_t dst1" +.Fa "bus_addr_t dst2" +.Fa "bus_addr_t src1" +.Fa "bus_addr_t src2" +.Fa "bus_dmaengine_callback_t callback_fn" +.Fa "void *callback_arg" +.Fa "uint32_t flags" +.Fc +.Ft struct bus_dmadesc * .Fo ioat_blockfill .Fa "bus_dmaengine_t dmaengine" .Fa "bus_addr_t dst" @@ -150,7 +161,7 @@ Then, they will submit one or more opera .Fn ioat_copy , or .Fn ioat_null . -After queueing one or more individual DMA operations, they will +After queuing one or more individual DMA operations, they will .Fn ioat_release the .Ar bus_dmaengine_t Modified: head/sys/dev/ioat/ioat.c ============================================================================== --- head/sys/dev/ioat/ioat.c Wed Dec 9 22:09:33 2015 (r292030) +++ head/sys/dev/ioat/ioat.c Wed Dec 9 22:45:51 2015 (r292031) @@ -833,6 +833,51 @@ ioat_copy(bus_dmaengine_t dmaengine, bus } struct bus_dmadesc * +ioat_copy_8k_aligned(bus_dmaengine_t dmaengine, bus_addr_t dst1, + bus_addr_t dst2, bus_addr_t src1, bus_addr_t src2, + bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) +{ + struct ioat_dma_hw_descriptor *hw_desc; + struct ioat_descriptor *desc; + struct ioat_softc *ioat; + + CTR0(KTR_IOAT, __func__); + ioat = to_ioat_softc(dmaengine); + + if (((src1 | src2 | dst1 | dst2) & (0xffffull << 48)) != 0) { + ioat_log_message(0, "%s: High 16 bits of src/dst invalid\n", + __func__); + return (NULL); + } + if (((src1 | src2 | dst1 | dst2) & PAGE_MASK) != 0) { + ioat_log_message(0, "%s: Addresses must be page-aligned\n", + __func__); + return (NULL); + } + + desc = ioat_op_generic(ioat, IOAT_OP_COPY, 2 * PAGE_SIZE, src1, dst1, + callback_fn, callback_arg, flags); + if (desc == NULL) + return (NULL); + + hw_desc = desc->u.dma; + if (src2 != src1 + PAGE_SIZE) { + hw_desc->u.control.src_page_break = 1; + hw_desc->next_src_addr = src2; + } + if (dst2 != dst1 + PAGE_SIZE) { + hw_desc->u.control.dest_page_break = 1; + hw_desc->next_dest_addr = dst2; + } + + if (g_ioat_debug_level >= 3) + dump_descriptor(hw_desc); + + ioat_submit_single(ioat); + return (&desc->bus_dmadesc); +} + +struct bus_dmadesc * ioat_blockfill(bus_dmaengine_t dmaengine, bus_addr_t dst, uint64_t fillpattern, bus_size_t len, bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags) Modified: head/sys/dev/ioat/ioat.h ============================================================================== --- head/sys/dev/ioat/ioat.h Wed Dec 9 22:09:33 2015 (r292030) +++ head/sys/dev/ioat/ioat.h Wed Dec 9 22:45:51 2015 (r292031) @@ -84,6 +84,19 @@ struct bus_dmadesc *ioat_copy(bus_dmaeng void *callback_arg, uint32_t flags); /* + * Issue a copy data operation, with constraints: + * - src1, src2, dst1, dst2 are all page-aligned addresses + * - The quantity to copy is exactly 2 pages; + * - src1 -> dst1, src2 -> dst2 + * + * Why use this instead of normal _copy()? You can copy two non-contiguous + * pages (src, dst, or both) with one descriptor. + */ +struct bus_dmadesc *ioat_copy_8k_aligned(bus_dmaengine_t dmaengine, + bus_addr_t dst1, bus_addr_t dst2, bus_addr_t src1, bus_addr_t src2, + bus_dmaengine_callback_t callback_fn, void *callback_arg, uint32_t flags); + +/* * Issues a null operation. This issues the operation to the hardware, but the * hardware doesn't do anything with it. */ Modified: head/sys/dev/ioat/ioat_internal.h ============================================================================== --- head/sys/dev/ioat/ioat_internal.h Wed Dec 9 22:09:33 2015 (r292030) +++ head/sys/dev/ioat/ioat_internal.h Wed Dec 9 22:45:51 2015 (r292031) @@ -175,8 +175,8 @@ struct ioat_dma_hw_descriptor { uint64_t src_addr; uint64_t dest_addr; uint64_t next; - uint64_t reserved; - uint64_t reserved2; + uint64_t next_src_addr; + uint64_t next_dest_addr; uint64_t user1; uint64_t user2; };