From owner-freebsd-arm@FreeBSD.ORG Fri Dec 14 20:54:36 2012 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (unknown [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 7428E1EED for ; Fri, 14 Dec 2012 20:54:36 +0000 (UTC) (envelope-from gonzo@id.bluezbox.com) Received: from id.bluezbox.com (id.bluezbox.com [88.198.91.248]) by mx1.freebsd.org (Postfix) with ESMTP id E573C8FC12 for ; Fri, 14 Dec 2012 20:54:35 +0000 (UTC) Received: from [88.198.91.248] (helo=[IPv6:::1]) by id.bluezbox.com with esmtpsa (TLSv1:CAMELLIA256-SHA:256) (Exim 4.77 (FreeBSD)) (envelope-from ) id 1TjcH3-000OFi-SF for freebsd-arm@freebsd.org; Fri, 14 Dec 2012 12:54:28 -0800 Message-ID: <50CB91FE.7050908@bluezbox.com> Date: Fri, 14 Dec 2012 12:54:22 -0800 From: Oleksandr Tymoshenko User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:17.0) Gecko/17.0 Thunderbird/17.0 MIME-Version: 1.0 To: freebsd-arm@freebsd.org Subject: Re: Unsolved problem with WB caches on ARMv6 References: <37F24B3C-4600-4E57-96EB-98C91FCD2B72@bluezbox.com> In-Reply-To: <37F24B3C-4600-4E57-96EB-98C91FCD2B72@bluezbox.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Sender: gonzo@id.bluezbox.com X-Spam-Level: -- X-Spam-Report: Spam detection software, running on the system "id.bluezbox.com", has identified this incoming email as possible spam. The original message has been attached to this so you can view it (if it isn't spam) or label similar future email. If you have any questions, see The administrator of that system for details. Content preview: On 12/9/2012 10:24 PM, Oleksandr Tymoshenko wrote: > Hello, > > One of the long-time issues with FreeBSD/ARMv6 is that Write-Back cache > mode does not work properly. On PandaBoard changing cache mode to WB from WT > causesUSB glitches (starting from stalls to network packets corruption) and random > memory corruptions that manifest themselves as a userland programs crashes. > > gber@ tracked down one of the bugs several month ago, but it's still unusable > at least on my setup. > > I spent some time debugging through busdma and USB code but failed to find > anything fishy. PandaBoard's USB host controller is EHCI. QH and QTDs are > flushed properly. Corruption pattern in packets is weird: it's not cacheline-size > it's like chunk of data is just missing from bulk transfer DMA buffer. L2 cache > is disabled. > > The issue is not reproducible in QEMU. > > Fix for arm/160431 applied to busdma-v6.c didn't help. > > I'm out of ideas for now. May be Ian or Alan will have some suggestions where to look? > > If you have setup with armv6 devices, I'd appreciate test results for this patch: > http://people.freebsd.org/~gonzo/patches/armv6-wb.diff > Just wondering if results differ from platform to platform. Following up on this one. [...] Content analysis details: (-2.9 points, 5.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- -1.0 ALL_TRUSTED Passed through trusted hosts only via SMTP -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 14 Dec 2012 20:54:36 -0000 On 12/9/2012 10:24 PM, Oleksandr Tymoshenko wrote: > Hello, > > One of the long-time issues with FreeBSD/ARMv6 is that Write-Back cache > mode does not work properly. On PandaBoard changing cache mode to WB from WT > causesUSB glitches (starting from stalls to network packets corruption) and random > memory corruptions that manifest themselves as a userland programs crashes. > > gber@ tracked down one of the bugs several month ago, but it's still unusable > at least on my setup. > > I spent some time debugging through busdma and USB code but failed to find > anything fishy. PandaBoard's USB host controller is EHCI. QH and QTDs are > flushed properly. Corruption pattern in packets is weird: it's not cacheline-size > it's like chunk of data is just missing from bulk transfer DMA buffer. L2 cache > is disabled. > > The issue is not reproducible in QEMU. > > Fix for arm/160431 applied to busdma-v6.c didn't help. > > I'm out of ideas for now. May be Ian or Alan will have some suggestions where to look? > > If you have setup with armv6 devices, I'd appreciate test results for this patch: > http://people.freebsd.org/~gonzo/patches/armv6-wb.diff > Just wondering if results differ from platform to platform. Following up on this one. For pandaboard it seems to be a bunch of problems, actually, not just one: Issues with known fixes: - Some additional pmap flushes/syncs, andrew@ is working on it. - I-Cache should be synced in pmap_enter_locked (otherwise userland programs cash in random places) - PL310 errata workaround wasn't functioning properly because DEBUG and CONTROL registers are not available for direct access in secureboot mode. So when I told it was disabled - actually it wasn't. Still working on: - Switching from WT to WB cache mode somehow affects EHCI internal works so occasionally parts of bulk IN transfers are lost. Adding delay after setting IAAD flag cures corruption. It's definitely not DMA/cache problem per se.