From owner-svn-src-all@FreeBSD.ORG  Thu Mar  5 18:11:27 2009
Return-Path: <owner-svn-src-all@FreeBSD.ORG>
Delivered-To: svn-src-all@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id 37DC1106564A;
	Thu,  5 Mar 2009 18:11:27 +0000 (UTC) (envelope-from alc@FreeBSD.org)
Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c])
	by mx1.freebsd.org (Postfix) with ESMTP id 25AB68FC1E;
	Thu,  5 Mar 2009 18:11:27 +0000 (UTC) (envelope-from alc@FreeBSD.org)
Received: from svn.freebsd.org (localhost [127.0.0.1])
	by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id n25IBRsY095552;
	Thu, 5 Mar 2009 18:11:27 GMT (envelope-from alc@svn.freebsd.org)
Received: (from alc@localhost)
	by svn.freebsd.org (8.14.3/8.14.3/Submit) id n25IBRj7095551;
	Thu, 5 Mar 2009 18:11:27 GMT (envelope-from alc@svn.freebsd.org)
Message-Id: <200903051811.n25IBRj7095551@svn.freebsd.org>
From: Alan Cox <alc@FreeBSD.org>
Date: Thu, 5 Mar 2009 18:11:27 +0000 (UTC)
To: src-committers@freebsd.org, svn-src-all@freebsd.org,
	svn-src-head@freebsd.org
X-SVN-Group: head
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Cc: 
Subject: svn commit: r189415 - head/sys/amd64/amd64
X-BeenThere: svn-src-all@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: "SVN commit messages for the entire src tree \(except for &quot;
	user&quot; and &quot; projects&quot; \)" <svn-src-all.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/svn-src-all>
List-Post: <mailto:svn-src-all@freebsd.org>
List-Help: <mailto:svn-src-all-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/svn-src-all>,
	<mailto:svn-src-all-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 05 Mar 2009 18:11:27 -0000

Author: alc
Date: Thu Mar  5 18:11:26 2009
New Revision: 189415
URL: http://svn.freebsd.org/changeset/base/189415

Log:
  Make pmap_copy() more TLB friendly.  Specifically, make it use the kernel's
  direct map instead of the pmap's recursive mapping to access the lowest
  level in the page table.
  
  MFC after:	6 weeks

Modified:
  head/sys/amd64/amd64/pmap.c

Modified: head/sys/amd64/amd64/pmap.c
==============================================================================
--- head/sys/amd64/amd64/pmap.c	Thu Mar  5 18:03:46 2009	(r189414)
+++ head/sys/amd64/amd64/pmap.c	Thu Mar  5 18:11:26 2009	(r189415)
@@ -3481,9 +3481,6 @@ pmap_copy(pmap_t dst_pmap, pmap_t src_pm
 	if (dst_addr != src_addr)
 		return;
 
-	if (!pmap_is_current(src_pmap))
-		return;
-
 	vm_page_lock_queues();
 	if (dst_pmap < src_pmap) {
 		PMAP_LOCK(dst_pmap);
@@ -3545,14 +3542,16 @@ pmap_copy(pmap_t dst_pmap, pmap_t src_pm
 			continue;
 		}
 
-		srcmpte = PHYS_TO_VM_PAGE(srcptepaddr & PG_FRAME);
+		srcptepaddr &= PG_FRAME;
+		srcmpte = PHYS_TO_VM_PAGE(srcptepaddr);
 		KASSERT(srcmpte->wire_count > 0,
 		    ("pmap_copy: source page table page is unused"));
 
 		if (va_next > end_addr)
 			va_next = end_addr;
 
-		src_pte = vtopte(addr);
+		src_pte = (pt_entry_t *)PHYS_TO_DMAP(srcptepaddr);
+		src_pte = &src_pte[pmap_pte_index(addr)];
 		while (addr < va_next) {
 			pt_entry_t ptetemp;
 			ptetemp = *src_pte;