From owner-svn-src-all@freebsd.org Fri Jan 15 01:34:45 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id E292AA8378B; Fri, 15 Jan 2016 01:34:44 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 9C59A1FDE; Fri, 15 Jan 2016 01:34:44 +0000 (UTC) (envelope-from cem@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u0F1YhPc025802; Fri, 15 Jan 2016 01:34:43 GMT (envelope-from cem@FreeBSD.org) Received: (from cem@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u0F1YhKi025798; Fri, 15 Jan 2016 01:34:43 GMT (envelope-from cem@FreeBSD.org) Message-Id: <201601150134.u0F1YhKi025798@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: cem set sender to cem@FreeBSD.org using -f From: "Conrad E. Meyer" Date: Fri, 15 Jan 2016 01:34:43 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r294062 - in head: share/man/man4 sys/dev/ioat X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 15 Jan 2016 01:34:45 -0000 Author: cem Date: Fri Jan 15 01:34:43 2016 New Revision: 294062 URL: https://svnweb.freebsd.org/changeset/base/294062 Log: ioat(4): Add support for 'fence' bit with DMA_FENCE flag Some classes of IOAT hardware prefetch reads. DMA operations that depend on the result of prior DMA operations must use the DMA_FENCE flag to prevent stale reads. (E.g., I've hit this personally on Broadwell-EP. The Broadwell-DE has a different IOAT unit that is documented to not pipeline DMA operations.) Sponsored by: EMC / Isilon Storage Division Modified: head/share/man/man4/ioat.4 head/sys/dev/ioat/ioat.c head/sys/dev/ioat/ioat.h Modified: head/share/man/man4/ioat.4 ============================================================================== --- head/share/man/man4/ioat.4 Fri Jan 15 01:26:32 2016 (r294061) +++ head/share/man/man4/ioat.4 Fri Jan 15 01:34:43 2016 (r294062) @@ -24,7 +24,7 @@ .\" .\" $FreeBSD$ .\" -.Dd January 7, 2016 +.Dd January 14, 2016 .Dt IOAT 4 .Os .Sh NAME @@ -134,7 +134,7 @@ Null operations do nothing, but may be u mechanism. .Pp All operations can optionally trigger an interrupt at completion with the -.Ar DMA_EN_INT +.Ar DMA_INT_EN flag. For example, a user might submit multiple operations to the same channel and only enable an interrupt and callback for the last operation. @@ -160,6 +160,17 @@ flag. .Ar DMA_NO_WAIT may return NULL.) .Pp +Operations that depend on the result of prior operations should use +.Ar DMA_FENCE . +For example, such a scenario can happen when two related DMA operations are +queued. +First, a DMA copy to one location (A), followed directly by a DMA copy +from A to B. +In this scenario, some classes of I/OAT hardware may prefetch A for the second +operation before it is written by the first operation. +To avoid reading a stale value in sequences of dependent operations, use +.Ar DMA_FENCE . +.Pp All operations, as well as .Fn ioat_get_dmaengine , can return NULL in special circumstances. Modified: head/sys/dev/ioat/ioat.c ============================================================================== --- head/sys/dev/ioat/ioat.c Fri Jan 15 01:26:32 2016 (r294061) +++ head/sys/dev/ioat/ioat.c Fri Jan 15 01:34:43 2016 (r294062) @@ -852,6 +852,8 @@ ioat_op_generic(struct ioat_softc *ioat, if ((flags & DMA_INT_EN) != 0) hw_desc->u.control_generic.int_enable = 1; + if ((flags & DMA_FENCE) != 0) + hw_desc->u.control_generic.fence = 1; hw_desc->size = size; hw_desc->src_addr = src; Modified: head/sys/dev/ioat/ioat.h ============================================================================== --- head/sys/dev/ioat/ioat.h Fri Jan 15 01:26:32 2016 (r294061) +++ head/sys/dev/ioat/ioat.h Fri Jan 15 01:34:43 2016 (r294062) @@ -46,7 +46,13 @@ __FBSDID("$FreeBSD$"); * descriptor without blocking. */ #define DMA_NO_WAIT 0x2 -#define DMA_ALL_FLAGS (DMA_INT_EN | DMA_NO_WAIT) +/* + * Disallow prefetching the source of the following operation. Ordinarily, DMA + * operations can be pipelined on some hardware. E.g., operation 2's source + * may be prefetched before operation 1 completes. + */ +#define DMA_FENCE 0x4 +#define DMA_ALL_FLAGS (DMA_INT_EN | DMA_NO_WAIT | DMA_FENCE) /* * Hardware revision number. Different hardware revisions support different