From owner-svn-src-stable-7@FreeBSD.ORG Fri Jan 22 20:52:32 2010 Return-Path: Delivered-To: svn-src-stable-7@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 229951065672; Fri, 22 Jan 2010 20:52:32 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id 0FFE68FC08; Fri, 22 Jan 2010 20:52:32 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.3/8.14.3) with ESMTP id o0MKqVGu082473; Fri, 22 Jan 2010 20:52:31 GMT (envelope-from jhb@svn.freebsd.org) Received: (from jhb@localhost) by svn.freebsd.org (8.14.3/8.14.3/Submit) id o0MKqVo9082471; Fri, 22 Jan 2010 20:52:31 GMT (envelope-from jhb@svn.freebsd.org) Message-Id: <201001222052.o0MKqVo9082471@svn.freebsd.org> From: John Baldwin Date: Fri, 22 Jan 2010 20:52:31 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-7@freebsd.org X-SVN-Group: stable-7 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r202836 - stable/7/sys/amd64/amd64 X-BeenThere: svn-src-stable-7@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for only the 7-stable src tree List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2010 20:52:32 -0000 Author: jhb Date: Fri Jan 22 20:52:31 2010 New Revision: 202836 URL: http://svn.freebsd.org/changeset/base/202836 Log: MFC 189610: Eliminate the last use of the recursive mapping to access user-space page table pages. Now, all accesses to user-space page table pages are performed through the direct map. (The recursive mapping is only used to access kernel-space page table pages.) Eliminate the TLB invalidation on the recursive mapping when a user-space page table page is removed from the page table and when a user-space superpage is demoted. Modified: stable/7/sys/amd64/amd64/pmap.c Directory Properties: stable/7/sys/ (props changed) stable/7/sys/cddl/contrib/opensolaris/ (props changed) stable/7/sys/contrib/dev/acpica/ (props changed) stable/7/sys/contrib/pf/ (props changed) Modified: stable/7/sys/amd64/amd64/pmap.c ============================================================================== --- stable/7/sys/amd64/amd64/pmap.c Fri Jan 22 20:46:11 2010 (r202835) +++ stable/7/sys/amd64/amd64/pmap.c Fri Jan 22 20:52:31 2010 (r202836) @@ -1278,7 +1278,6 @@ static int _pmap_unwire_pte_hold(pmap_t pmap, vm_offset_t va, vm_page_t m, vm_page_t *free) { - vm_offset_t pteva; /* * unmap the page table page @@ -1287,19 +1286,16 @@ _pmap_unwire_pte_hold(pmap_t pmap, vm_of /* PDP page */ pml4_entry_t *pml4; pml4 = pmap_pml4e(pmap, va); - pteva = (vm_offset_t) PDPmap + amd64_ptob(m->pindex - (NUPDE + NUPDPE)); *pml4 = 0; } else if (m->pindex >= NUPDE) { /* PD page */ pdp_entry_t *pdp; pdp = pmap_pdpe(pmap, va); - pteva = (vm_offset_t) PDmap + amd64_ptob(m->pindex - NUPDE); *pdp = 0; } else { /* PTE page */ pd_entry_t *pd; pd = pmap_pde(pmap, va); - pteva = (vm_offset_t) PTmap + amd64_ptob(m->pindex); *pd = 0; } --pmap->pm_stats.resident_count; @@ -1325,12 +1321,6 @@ _pmap_unwire_pte_hold(pmap_t pmap, vm_of */ atomic_subtract_rel_int(&cnt.v_wire_count, 1); - /* - * Do an invltlb to make the invalidated mapping - * take effect immediately. - */ - pmap_invalidate_page(pmap, pteva); - /* * Put page on a list so that it is released after * *ALL* TLB shootdown is done @@ -2287,9 +2277,10 @@ pmap_demote_pde(pmap_t pmap, pd_entry_t pde_store(pde, newpde); /* - * Invalidate a stale mapping of the page table page. + * Invalidate a stale recursive mapping of the page table page. */ - pmap_invalidate_page(pmap, (vm_offset_t)vtopte(va)); + if (va >= VM_MAXUSER_ADDRESS) + pmap_invalidate_page(pmap, (vm_offset_t)vtopte(va)); /* * Demote the pv entry. This depends on the earlier demotion @@ -3737,7 +3728,7 @@ pmap_page_is_mapped(vm_page_t m) void pmap_remove_pages(pmap_t pmap) { - pd_entry_t *pde; + pd_entry_t ptepde; pt_entry_t *pte, tpte; vm_page_t free = NULL; vm_page_t m, mpte, mt; @@ -3766,23 +3757,19 @@ pmap_remove_pages(pmap_t pmap) pv = &pc->pc_pventry[idx]; inuse &= ~bitmask; - pde = vtopde(pv->pv_va); - tpte = *pde; - if ((tpte & PG_PS) != 0) - pte = pde; - else { + pte = pmap_pdpe(pmap, pv->pv_va); + ptepde = *pte; + pte = pmap_pdpe_to_pde(pte, pv->pv_va); + tpte = *pte; + if ((tpte & (PG_PS | PG_V)) == PG_V) { + ptepde = tpte; pte = (pt_entry_t *)PHYS_TO_DMAP(tpte & PG_FRAME); pte = &pte[pmap_pte_index(pv->pv_va)]; tpte = *pte & ~PG_PTE_PAT; } - - if (tpte == 0) { - printf( - "TPTE at %p IS ZERO @ VA %08lx\n", - pte, pv->pv_va); + if ((tpte & PG_V) == 0) panic("bad pte"); - } /* * We cannot remove wired pages from a process' mapping at this time @@ -3839,8 +3826,6 @@ pmap_remove_pages(pmap_t pmap) pmap_add_delayed_free_list(mpte, &free, FALSE); atomic_subtract_int(&cnt.v_wire_count, 1); } - pmap_unuse_pt(pmap, pv->pv_va, - *pmap_pdpe(pmap, pv->pv_va), &free); } else { pmap->pm_stats.resident_count--; TAILQ_REMOVE(&m->md.pv_list, pv, pv_list); @@ -3849,8 +3834,8 @@ pmap_remove_pages(pmap_t pmap) if (TAILQ_EMPTY(&pvh->pv_list)) vm_page_flag_clear(m, PG_WRITEABLE); } - pmap_unuse_pt(pmap, pv->pv_va, *pde, &free); } + pmap_unuse_pt(pmap, pv->pv_va, ptepde, &free); } } if (allfree) {