From owner-freebsd-arch@FreeBSD.ORG Sat Sep 29 15:12:26 2007 Return-Path: Delivered-To: freebsd-arch@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 05DEB16A421 for ; Sat, 29 Sep 2007 15:12:26 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id 96F9913C47E for ; Sat, 29 Sep 2007 15:12:25 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.1/8.13.4) with ESMTP id l8TF9rXE034409; Sat, 29 Sep 2007 09:09:54 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Sat, 29 Sep 2007 09:09:56 -0600 (MDT) Message-Id: <20070929.090956.-1889954517.imp@bsdimp.com> To: hselasky@c2i.net From: "M. Warner Losh" In-Reply-To: <200709291309.05747.hselasky@c2i.net> References: <200709281753.30367.hselasky@c2i.net> <20070928.141345.78789158.imp@bsdimp.com> <200709291309.05747.hselasky@c2i.net> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-2.0 (harmony.bsdimp.com [127.0.0.1]); Sat, 29 Sep 2007 09:09:54 -0600 (MDT) Cc: scottl@samsco.org, freebsd-arch@freebsd.org Subject: Re: Request for feedback on common data backstore in the kernel X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Sep 2007 15:12:26 -0000 In message: <200709291309.05747.hselasky@c2i.net> Hans Petter Selasky writes: : On Friday 28 September 2007, Warner Losh wrote: : > Hans Petter Selasky wrote: : > > > Well, the point is that I'm not sure why you're so worried about : > > > performance issues with USB and busdma. Do you have any test data that : > > > shows that it's a problem? : > > : > > It is a problem on embedded devices. Typically not for an ordinary PC : > > user. : > : > I'm curious. What's the problem? : : Technically, if you load a buffer pointer and length that is not cache aligned : nor host aligned then you should bounce all the data no matter what. If you : do not bounce under those conditions you might get trouble. Why do you say this? It doesn't match my experience. : struct my_softc { : uint8_t hdr[2]; : uint8_t eth_pkt[1500]; : uint8_t other_important_stuff[546]; : } __aligned(PAGE_SIZE); : : Can "eth_pkt" be loaded into DMA ? No, it has an offset of 2. This may or may not be able to load. It all depends on ethernet controller's requirements for alignment. : struct my_softc { : uint8_t eth_pkt[1500]; : uint8_t other_important_stuff[548]; : } __aligned(PAGE_SIZE); : : Can "eth_pkt" be loaded into DMA ? No, "other_important_stuff" might get lost : when the cache is invalidated. On MIPS and ARM you can invalidate/flush specific ranges of memory without the need for them to be aligned to a cache-line. There may be other reasons why you can't, but mere alignment isn't it. : struct my_softc { : uint8_t eth_pkt[1500]; : } __aligned(PAGE_SIZE); : : Can "eth_pkt" be loaded into DMA ? Yes, I see no problem. Maybe. Some designs require that you do DMA only to/from a specific range of memory. So while it meets alignment requirements, it may not be allocated from the proper range of memory. In these designs, bouncing is almost forced on the driver writer. : Is it guaranteed that the "m_data" field of a mbuf will always be correctly : aligned for the CPU/DMA cache in addition to the host alignment ? : : Does bus_dma support bouncing using multiple pages instead of using one : contiguous segment ? : : Please correct me if I am wrong. The busdma interface is designed to hide these issues. There are some problems with the busdma interface, but they are minor in comparison to the portability it buys. I use the busdma interface for my Ethernet driver on the AT91RM9200. There's some weirdness for that part because it had alignment requirements that are bad for IP: the ethernet header is longword aligned, necessarily forcing the ip header not to be. Warner