From owner-freebsd-arm@FreeBSD.ORG Sun Feb 5 00:03:53 2012 Return-Path: Delivered-To: freebsd-arm@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBC53106567A for ; Sun, 5 Feb 2012 00:03:53 +0000 (UTC) (envelope-from jmg@h2.funkthat.com) Received: from h2.funkthat.com (gate2.funkthat.com [208.87.223.18]) by mx1.freebsd.org (Postfix) with ESMTP id 817158FC16 for ; Sun, 5 Feb 2012 00:03:53 +0000 (UTC) Received: from h2.funkthat.com (localhost [127.0.0.1]) by h2.funkthat.com (8.14.3/8.14.3) with ESMTP id q14NhKW3050245 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 4 Feb 2012 15:43:20 -0800 (PST) (envelope-from jmg@h2.funkthat.com) Received: (from jmg@localhost) by h2.funkthat.com (8.14.3/8.14.3/Submit) id q14NhJL3050233; Sat, 4 Feb 2012 15:43:19 -0800 (PST) (envelope-from jmg) Date: Sat, 4 Feb 2012 15:43:19 -0800 From: John-Mark Gurney To: Ian Lepore Message-ID: <20120204234319.GR52468@funkthat.com> Mail-Followup-To: Ian Lepore , Warner Losh , freebsd-arm@freebsd.org References: <1327980703.1662.240.camel@revolution.hippie.lan> <1328025245.1662.289.camel@revolution.hippie.lan> <5FB4965A-66C9-4C99-8B61-5AC605F9ECC5@bsdimp.com> <1328030999.1662.324.camel@revolution.hippie.lan> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1328030999.1662.324.camel@revolution.hippie.lan> User-Agent: Mutt/1.4.2.3i X-Operating-System: FreeBSD 7.2-RELEASE i386 X-Files: The truth is out there X-URL: http://resnet.uoregon.edu/~gurney_j/ X-Resume: http://resnet.uoregon.edu/~gurney_j/resume.html X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.2 (h2.funkthat.com [127.0.0.1]); Sat, 04 Feb 2012 15:43:20 -0800 (PST) Cc: freebsd-arm@freebsd.org Subject: Re: Performance of SheevaPlug on 8-stable X-BeenThere: freebsd-arm@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Porting FreeBSD to the StrongARM Processor List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 05 Feb 2012 00:03:53 -0000 Ian Lepore wrote this message on Tue, Jan 31, 2012 at 10:29 -0700: > On Tue, 2012-01-31 at 09:37 -0700, Warner Losh wrote: > > On Jan 31, 2012, at 8:54 AM, Ian Lepore wrote: > > > > > On Mon, 2012-01-30 at 22:39 -0700, Warner Losh wrote: > > >> Hi Ian, > > >> > > >> Do you have any data on what 9.0 does? > > >> > > >> Warner > > > > > > No. Do you have reason to believe it will be different than 8.x? > > > > > > It would be a major effort right now to get anything later than 8.2 > > > built and running on one of our arm platforms. Maybe not as hard as the > > > 6.2 -> 8.2 conversion was, but we're still carrying a lot of diffs from > > > stock FreeBSD that have to be analyzed and merged by hand. Actually > > > before that can even happen I'd have to grab a snapshot of 9.0 and do an > > > svn->Hg conversion to even be able to start merging the diffs (and I'm > > > hardly an Hg expert, but those in the company who are let me know last > > > week that they're just as busy as me, and I'm on my own for this kind of > > > work). It's work I want to do, but I suspect it's going to happen later > > > rather than sooner because product deadlines are beginning to loom and > > > my ability to spend most of my time working on the OS side of things is > > > waning. > > > > > > If there are some specific changes you've got in mind that affect this > > > problem I might be able to backport and test them faster than I could > > > get a full 9.0 or -current build environment working, just point me at > > > them. > > > > I thought that we'd done a root cause of this and had put a fix into the vm system. Lemme look... > > > > ------------------------------------------------------------------------ > > r224049 | marcel | 2011-07-14 20:11:26 -0600 (Thu, 14 Jul 2011) | 2 lines > > > > In pmap_protect(), don't call vm_page_dirty() if the page is unmanaged. > > > > > > ------------------------------------------------------------------------ > > r221844 | cognet | 2011-05-13 09:54:12 -0600 (Fri, 13 May 2011) | 4 lines > > > > In pmap_change_wiring(), use the right argument for pmap_modify_pv(). > > It only worked because the only consumer calls pmap_change_wiring() to remove > > the wiring. > > > > ------------------------------------------------------------------------ > > r212507 | cognet | 2010-09-12 14:46:32 -0600 (Sun, 12 Sep 2010) | 5 lines > > > > In pmap_remove_all(), do not decrease pm_stats.wired_count if the mapping was > > wired, as it's been done later in pmap_nuke_pv(). > > > > Submitted by: Mark Tinguely > > > > > > ------------------------------------------------------------------------ > > r209223 | cognet | 2010-06-15 16:16:02 -0600 (Tue, 15 Jun 2010) | 4 lines > > > > Turn off cache if there's more than one kernel mapping, and one is writable. > > > > Submitted by: Mark Tinguely > > > > ------------------------------------------------------------------------ > > r205028 | raj | 2010-03-11 14:16:54 -0700 (Thu, 11 Mar 2010) | 12 lines > > > > Fix ARM cache handling yet more. > > > > 1) vm_machdep.c: remove the dangling allocations so they do not > > un-necessarily turn off the cache upon consecutive access. > > > > 2) busdma_machdep.c: remove the same amount than shadow mapped. > > > > Reported by: Maks Verver > > Submitted by: Mark Tinguely > > Reviewed by: Grzegorz Bernacki > > MFC after: 3 days > > > > ------------------------------------------------------------------------ > > r203637 | raj | 2010-02-07 13:48:57 -0700 (Sun, 07 Feb 2010) | 19 lines > > > > Improve checking whether an ARM VA has a valid mapping before performing cache > > sync. > > > > VIPT/PIPT caches need valid VA-PA mapping in PTE for a cache operation to > > succeed (unlike VIVT). Prior to this fix pmap was using l2pte_valid() for that > > check, but this is not sufficient as the function merely checks if a PTE > > exists (there can be existing but _invalid_ entries in the table). > > > > A new pmap_has_valid_mapping() routine is introduced to do this job right by > > checking proper PTE flags. > > > > Among other potential problems this cures coherency issues with L2 caches on > > MV-78100. > > > > Submitted by: Grzegorz Bernacki, Piotr Ziecik > > Reviewed, tested by: marcel > > Obtained from: Semihalf > > MFC after: 1 week > > > > > > Only the last two have MFC, so you can start there and see which of these changes are in... > > > > Just thought you might have a reference board that would be easy to test... > > > > Warner > > I think we may have all those changes incorporated except perhaps > r224049; I'll make sure of that. > > r209223 is the change that exposed this situation. > > I'm skeptical that any of the changes you cite (or any change at all in > the pmap layer) will fix the problem, because the problem seems to be > rooted in the fact that the vfs buffer cache establishes a kva mapping > of the buffer pages with the protections set to READ|WRITE|EXEC and > leaves that mapping in place as long as the buffer is in the cache, and > r209223 says that as long as there are multiple mappings of a page with > at least one writable, that page's i-cache and d-cache bits stay off. > (The multiple mappings being the one for the buffer cache that includes > write access and one or more READ|EXEC mappings made by pmap() when the > executable or library is loaded/relocated.) > > If my analysis is correct (and I'm fairly sure, if not 100% positive, > that it is), then it seems to me that the only fix available is going to > be at the vfs layer, and it's going to involve dropping the write access > to the pages in the buffer cache once any physical IO and/or uio > operations needing write access are completed. > > Even if I could figure out a patchset to fix the problem, it's going to > need a lot of input from the vm gurus to answer questions such as what > the performance impact will be to non-VIVT platforms that don't need > this extra work done. If the extra work is expensive enough (and I'm > not sure I could evaluate that properly) it may need to be conditional > on whether the platform needs it. I'm also vaguely uneasy with all this > on a purely philosphical level, since this could end up basically > infecting MI code with a platform-specific concept. What is an easy to figure out if a system we have is effected by this issue? I have a GW2348-4 board running FreeBSD 9.0-RC1 w/ some minor modifications to get pf to work... I think the system is effected by this since userland seems really slow.. Thanks. -- John-Mark Gurney Voice: +1 415 225 5579 "All that I will do, has been done, All that I have, has not."