From owner-freebsd-mips@FreeBSD.ORG Mon Aug 27 22:12:56 2012 Return-Path: Delivered-To: freebsd-mips@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C81371065670; Mon, 27 Aug 2012 22:12:56 +0000 (UTC) (envelope-from ticso@cicely7.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 55C148FC0A; Mon, 27 Aug 2012 22:12:55 +0000 (UTC) Received: from mail.cicely.de ([10.1.1.37]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id q7RMCjUO059568 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Tue, 28 Aug 2012 00:12:46 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (cicely7.cicely.de [10.1.1.9]) by mail.cicely.de (8.14.4/8.14.4) with ESMTP id q7RMCWPU082659 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 28 Aug 2012 00:12:32 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: from cicely7.cicely.de (localhost [127.0.0.1]) by cicely7.cicely.de (8.14.2/8.14.2) with ESMTP id q7RMCWaC074506; Tue, 28 Aug 2012 00:12:32 +0200 (CEST) (envelope-from ticso@cicely7.cicely.de) Received: (from ticso@localhost) by cicely7.cicely.de (8.14.2/8.14.2/Submit) id q7RMCWMQ074505; Tue, 28 Aug 2012 00:12:32 +0200 (CEST) (envelope-from ticso) Date: Tue, 28 Aug 2012 00:12:32 +0200 From: Bernd Walter To: Warner Losh Message-ID: <20120827221231.GB73992@cicely7.cicely.de> References: <6D83AF9D-577B-4C83-84B7-C4E3B32695FC@bsdimp.com> <45C204C3-D4CE-4B00-886A-A88A0FE7CAD7@bsdimp.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <45C204C3-D4CE-4B00-886A-A88A0FE7CAD7@bsdimp.com> X-Operating-System: FreeBSD cicely7.cicely.de 7.0-STABLE i386 User-Agent: Mutt/1.5.11 X-Spam-Status: No, score=-2.9 required=5.0 tests=ALL_TRUSTED=-1, BAYES_00=-1.9, T_RP_MATCHES_RCVD=-0.01 autolearn=unavailable version=3.3.0 X-Spam-Checker-Version: SpamAssassin 3.3.0 (2010-01-18) on spamd.cicely.de Cc: Tim Kientzle , freebsd-arm@freebsd.org, freebsd-arch@freebsd.org, freebsd-mips@freebsd.org, Hans Petter Selasky Subject: Re: Partial cacheline flush problems on ARM and MIPS X-BeenThere: freebsd-mips@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Porting FreeBSD to MIPS List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2012 22:12:57 -0000 On Mon, Aug 27, 2012 at 09:34:30AM -0600, Warner Losh wrote: > > On Aug 27, 2012, at 9:12 AM, Tim Kientzle wrote: > > > > > On Aug 27, 2012, at 6:38 AM, Warner Losh wrote: > > > >> > >> On Aug 27, 2012, at 12:06 AM, Hans Petter Selasky wrote: > >> > >>> Hi, > >>> Correct. > >>> > >>>> We also need some rules about working with buffers obtained from > >>>> bus_dmamem_alloc() and external buffers passed to bus_dmamap_load(). I > >>>> think the rule should be that a buffer obtained from bus_dmamem_alloc(), > >>>> or more formally any region of memory mapped by a bus_dmamap_load(), is > >>>> a single logical object which can only be accessed by one entity at a > >>>> time. That means that there cannot be two concurrent DMA operations > >>>> happening in different regions of the same buffer, nor can DMA and CPU > >>>> access be happening concurrently even if in different parts of the > >>>> buffer. > >>> > >>> Is this something which we can fix using a simple __align(USB_DMA_ALIGN) on elements in C-structures which are allowed to be DMA loaded. > >> > >> No. I don't think so. the reason is that you can't define USB_DMA_ALGIN to be a constant on MIPS, at least, or I think ARM because that's determined at run time. > > > > But don't mbuf structures do pretty much what Hans is suggesting? > > They kinda do, and kinda don't. > > > Why is mbuf okay? > > mbuf is OK because it is never changed while the DMA is pending to the buffers. That is, m_hdr and m_ext are invariant while the device owns the memory. In addition, mbuf allocations are so large that no two mbufs share the same cache line (although it looks like 256 might be too small to avoid that on some MIPS processors). usb buffers do not obey these same rules, otherwise none of the stuff we're talking about would matter. I've always hated the design of USB controllers with their zillions of tiny DMA buffers. -- B.Walter http://www.bwct.de Modbus/TCP Ethernet I/O Baugruppen, ARM basierte FreeBSD Rechner uvm.