From owner-freebsd-arm@FreeBSD.ORG  Tue Jun  2 18:39:30 2009
Return-Path: <owner-freebsd-arm@FreeBSD.ORG>
Delivered-To: freebsd-arm@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id D958D1065680;
	Tue,  2 Jun 2009 18:39:30 +0000 (UTC)
	(envelope-from tinguely@casselton.net)
Received: from casselton.net (casselton.net [63.165.140.2])
	by mx1.freebsd.org (Postfix) with ESMTP id 8AD5A8FC1B;
	Tue,  2 Jun 2009 18:39:30 +0000 (UTC)
	(envelope-from tinguely@casselton.net)
Received: from casselton.net (localhost [127.0.0.1])
	by casselton.net (8.14.3/8.14.3) with ESMTP id n52IdTHZ002808;
	Tue, 2 Jun 2009 13:39:29 -0500 (CDT)
	(envelope-from tinguely@casselton.net)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=casselton.net;
	s=ccnMail; t=1243967969;
	bh=7mxrFCEzPAk5y4MsFPecUFA/2GqSDo5aorOXlNh9AHs=;
	h=Date:From:Message-Id:To:Subject:Cc:In-Reply-To;
	b=eEYN8F68eWQ+LMLNOsUt59omjMpN/d+a/jd+U04c4csIFHqVq2+NsEmyiFbfzBGnD
	g8v4wbgHKLQEEMJq2WQN9VKmlUA1KCHNaXCUp1+hzaHNPM/S2XtG0tFF8ub+KtHENn
	5jGWLdF/zpXyIsptgDI7dYW+2YV1NbTM70xWDsfo=
Received: (from tinguely@localhost)
	by casselton.net (8.14.3/8.14.2/Submit) id n52IdSrY002806;
	Tue, 2 Jun 2009 13:39:28 -0500 (CDT) (envelope-from tinguely)
Date: Tue, 2 Jun 2009 13:39:28 -0500 (CDT)
From: Mark Tinguely <tinguely@casselton.net>
Message-Id: <200906021839.n52IdSrY002806@casselton.net>
To: sam@freebsd.org, tinguely@casselton.net
In-Reply-To: <4A25566C.7020806@freebsd.org>
X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.3.2
	(casselton.net [127.0.0.1]); Tue, 02 Jun 2009 13:39:29 -0500 (CDT)
Cc: freebsd-arm@freebsd.org
Subject: Re: cache corruption patch
X-BeenThere: freebsd-arm@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Porting FreeBSD to the StrongARM Processor <freebsd-arm.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arm>,
	<mailto:freebsd-arm-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-arm>
List-Post: <mailto:freebsd-arm@freebsd.org>
List-Help: <mailto:freebsd-arm-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-arm>,
	<mailto:freebsd-arm-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Tue, 02 Jun 2009 18:39:31 -0000


>  Have you measured the overhead of this additional state tracking 
>  (time+space)?  It appears to affect only dma buffers.

no, I haven't tracked the overhead of tracking and turning off the cache for
the multiply mapped page versus flushing the entire cache.

For space, the patch costs us 4 bytes/page. I removed 16 bytes/page
when I re-implemented the cache fixing routines (from pmap_vac_me_harder()
from NetBSD and in -7.x to pmap_fix_cache() which is new for -8.x). The
main reason to re-implement the cache fixing routines was to fix the
"big O" factor on the routines.

Right now we have the potential of stale caches any time a kernel mapped
page is shared with a user or another kernel mapping. Things like mmap
comes to mind.

>  Not sure what "added cache invalidation commands" you're referring to, 
>  can you explain?  Perhaps you could just provide a separate patch that 
>  shows which bus_dma ops you think can go away.

Yes, without the patch, some people placed cache flushes in the DMA sync
routines. That did solve the obvious problem. Those flushes are not in
the ARM code and the patch does not remove them.

A few months back, when new USB stack brought the cache problems to the
foreground, others mention strange intermittent memory corruption that
pointed to stale caches. We could start sticking in cache flushes for
each new one that pops up, but it seemed like a game of russian roulette.

--Mark.