From owner-svn-src-head@FreeBSD.ORG Wed Mar 28 17:25:29 2012 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D1261106564A; Wed, 28 Mar 2012 17:25:29 +0000 (UTC) (envelope-from nwhitehorn@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id BB1268FC0A; Wed, 28 Mar 2012 17:25:29 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q2SHPTu7046845; Wed, 28 Mar 2012 17:25:29 GMT (envelope-from nwhitehorn@svn.freebsd.org) Received: (from nwhitehorn@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q2SHPT5i046842; Wed, 28 Mar 2012 17:25:29 GMT (envelope-from nwhitehorn@svn.freebsd.org) Message-Id: <201203281725.q2SHPT5i046842@svn.freebsd.org> From: Nathan Whitehorn Date: Wed, 28 Mar 2012 17:25:29 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r233618 - head/sys/powerpc/aim X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 28 Mar 2012 17:25:30 -0000 Author: nwhitehorn Date: Wed Mar 28 17:25:29 2012 New Revision: 233618 URL: http://svn.freebsd.org/changeset/base/233618 Log: More PMAP performance improvements: skip 256 MB segments entirely if they are are not mapped during ranged operations and reduce the scope of the tlbie lock only to the actual tlbie instruction instead of the entire sequence. There are a few more optimization possibilities here as well. Modified: head/sys/powerpc/aim/mmu_oea64.c head/sys/powerpc/aim/moea64_native.c Modified: head/sys/powerpc/aim/mmu_oea64.c ============================================================================== --- head/sys/powerpc/aim/mmu_oea64.c Wed Mar 28 17:21:59 2012 (r233617) +++ head/sys/powerpc/aim/mmu_oea64.c Wed Mar 28 17:25:29 2012 (r233618) @@ -1981,10 +1981,18 @@ moea64_protect(mmu_t mmu, pmap_t pm, vm_ LOCK_TABLE_RD(); PMAP_LOCK(pm); if ((eva - sva)/PAGE_SIZE < pm->pm_stats.resident_count) { - for (; sva < eva; sva += PAGE_SIZE) { + while (sva < eva) { + #ifdef __powerpc64__ + if (pm != kernel_pmap && + user_va_to_slb_entry(pm, sva) == NULL) { + sva = roundup2(sva + 1, SEGMENT_LENGTH); + continue; + } + #endif pvo = moea64_pvo_find_va(pm, sva); if (pvo != NULL) moea64_pvo_protect(mmu, pm, pvo, prot); + sva += PAGE_SIZE; } } else { LIST_FOREACH_SAFE(pvo, &pm->pmap_pvo, pvo_plink, tpvo) { @@ -2095,10 +2103,18 @@ moea64_remove(mmu_t mmu, pmap_t pm, vm_o LOCK_TABLE_WR(); PMAP_LOCK(pm); if ((eva - sva)/PAGE_SIZE < pm->pm_stats.resident_count) { - for (; sva < eva; sva += PAGE_SIZE) { + while (sva < eva) { + #ifdef __powerpc64__ + if (pm != kernel_pmap && + user_va_to_slb_entry(pm, sva) == NULL) { + sva = roundup2(sva + 1, SEGMENT_LENGTH); + continue; + } + #endif pvo = moea64_pvo_find_va(pm, sva); if (pvo != NULL) moea64_pvo_remove(mmu, pvo); + sva += PAGE_SIZE; } } else { LIST_FOREACH_SAFE(pvo, &pm->pmap_pvo, pvo_plink, tpvo) { @@ -2566,7 +2582,7 @@ moea64_mapdev_attr(mmu_t mmu, vm_offset_ ppa = trunc_page(pa); offset = pa & PAGE_MASK; - size = roundup(offset + size, PAGE_SIZE); + size = roundup2(offset + size, PAGE_SIZE); va = kmem_alloc_nofault(kernel_map, size); @@ -2597,7 +2613,7 @@ moea64_unmapdev(mmu_t mmu, vm_offset_t v base = trunc_page(va); offset = va & PAGE_MASK; - size = roundup(offset + size, PAGE_SIZE); + size = roundup2(offset + size, PAGE_SIZE); kmem_free(kernel_map, base, size); } Modified: head/sys/powerpc/aim/moea64_native.c ============================================================================== --- head/sys/powerpc/aim/moea64_native.c Wed Mar 28 17:21:59 2012 (r233617) +++ head/sys/powerpc/aim/moea64_native.c Wed Mar 28 17:25:29 2012 (r233618) @@ -103,6 +103,7 @@ __FBSDID("$FreeBSD$"); #include #include #include +#include #include #include @@ -152,15 +153,13 @@ TLBIE(uint64_t vpn) { vpn &= ~(0xffffULL << 48); #ifdef __powerpc64__ + sched_pin(); + __asm __volatile("ptesync"); mtx_lock(&tlbie_mutex); - __asm __volatile("\ - ptesync; \ - tlbie %0; \ - eieio; \ - tlbsync; \ - ptesync;" - :: "r"(vpn) : "memory"); + __asm __volatile("tlbie %0" :: "r"(vpn) : "memory"); mtx_unlock(&tlbie_mutex); + __asm __volatile("eieio; tlbsync; ptesync"); + sched_unpin(); #else vpn_hi = (uint32_t)(vpn >> 32); vpn_lo = (uint32_t)vpn;