From owner-freebsd-arm@FreeBSD.ORG Wed Jul 8 14:29:25 2009 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EC6881065676; Wed, 8 Jul 2009 14:29:25 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from harmony.bsdimp.com (bsdimp.com [199.45.160.85]) by mx1.freebsd.org (Postfix) with ESMTP id A855F8FC15; Wed, 8 Jul 2009 14:29:25 +0000 (UTC) (envelope-from imp@bsdimp.com) Received: from localhost (localhost [127.0.0.1]) by harmony.bsdimp.com (8.14.3/8.14.1) with ESMTP id n68EN5GH053804; Wed, 8 Jul 2009 08:23:06 -0600 (MDT) (envelope-from imp@bsdimp.com) Date: Wed, 08 Jul 2009 08:23:43 -0600 (MDT) Message-Id: <20090708.082343.-1343671115.imp@bsdimp.com> To: hselasky@c2i.net From: "M. Warner Losh" In-Reply-To: <200907081103.45388.hselasky@c2i.net> References: <200907071846.42086.hselasky@c2i.net> <37C51279-42D8-49DE-8249-0DA386EBB062@semihalf.com> <200907081103.45388.hselasky@c2i.net> X-Mailer: Mew version 5.2 on Emacs 21.3 / Mule 5.0 (SAKAKI) Mime-Version: 1.0 Content-Type: Text/Plain; charset=us-ascii Content-Transfer-Encoding: 7bit Cc: freebsd-usb@freebsd.org, freebsd-arm@freebsd.org, kosmo@semihalf.com, thompsa@freebsd.org Subject: Re: CPU Cache and busdma usage in USB X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 08 Jul 2009 14:29:26 -0000 In message: <200907081103.45388.hselasky@c2i.net> Hans Petter Selasky writes: : On Wednesday 08 July 2009 10:58:17 Rafal Jaworowski wrote: : > On 2009-07-07, at 18:46, Hans Petter Selasky wrote: : > >> I had Checked USB behaviour on PowerPC without hardware cache : > >> coherency. : > >> The problem also exists here and patch helps. : > > : > > In my source code view the busdma sync function is empty for power- : > > pc. Your : > > patch should have no effect at all for sync operations? : > : > This was about the PPC4xx PowerPC port, not committed to the tree yet. : > Unlike most PowerPC this system has a de-coherent DMA and therefore : > its busdma sync is non empty, but needs to enforce coherency by : > software. : > : > The point is this is another platform on which USB stack (usb_pc_cpu_* : > functions) shows similar issues as reported for ARM and MIPS. : : And what about my patch suggestion in my previous e-mail having the same : subject. Does it work? : : Regarding my testing on the AT91RM9200 I was short of time yesterday and will : try to get it done today. I think that the root cause of this issue is two fold. First, The 4 operations for busdma are being collapsed to only 2 operations. There are four for a reason (since you have to do different things for each case). Second, and I think this is more important, I think that we're seeing some cache-line poisoning. We're flushing the entire busdma tag rather than just one cache line since we don't have the API to do that. In addition, I don't think that the USB code is being careful enough to ensure that we don't have buffers that live in the same cache line that are simultaneously being used for read and write. The corruption we're seeing is a classic signal that this may be going on, although I must admit that I've not done the detailed analysis to prove it. Warner