From owner-svn-src-head@FreeBSD.ORG Sun Mar 25 06:01:35 2012 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2596B106564A; Sun, 25 Mar 2012 06:01:35 +0000 (UTC) (envelope-from nwhitehorn@FreeBSD.org) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:4f8:fff6::2c]) by mx1.freebsd.org (Postfix) with ESMTP id EB3AA8FC12; Sun, 25 Mar 2012 06:01:34 +0000 (UTC) Received: from svn.freebsd.org (localhost [127.0.0.1]) by svn.freebsd.org (8.14.4/8.14.4) with ESMTP id q2P61YD3066733; Sun, 25 Mar 2012 06:01:34 GMT (envelope-from nwhitehorn@svn.freebsd.org) Received: (from nwhitehorn@localhost) by svn.freebsd.org (8.14.4/8.14.4/Submit) id q2P61Yut066731; Sun, 25 Mar 2012 06:01:34 GMT (envelope-from nwhitehorn@svn.freebsd.org) Message-Id: <201203250601.q2P61Yut066731@svn.freebsd.org> From: Nathan Whitehorn Date: Sun, 25 Mar 2012 06:01:34 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Cc: Subject: svn commit: r233454 - head/sys/powerpc/aim X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 25 Mar 2012 06:01:35 -0000 Author: nwhitehorn Date: Sun Mar 25 06:01:34 2012 New Revision: 233454 URL: http://svn.freebsd.org/changeset/base/233454 Log: More PMAP performance improvements: on powerpc64, when TLBIE can be run with exceptions enabled, leave them enabled and use a regular mutex to guard TLB invalidations instead of a spinlock. Modified: head/sys/powerpc/aim/moea64_native.c Modified: head/sys/powerpc/aim/moea64_native.c ============================================================================== --- head/sys/powerpc/aim/moea64_native.c Sun Mar 25 03:14:31 2012 (r233453) +++ head/sys/powerpc/aim/moea64_native.c Sun Mar 25 06:01:34 2012 (r233454) @@ -138,7 +138,7 @@ __FBSDID("$FreeBSD$"); * Just to add to the fun, exceptions must be off as well * so that we can't trap in 64-bit mode. What a pain. */ -struct mtx tlbie_mutex; +static struct mtx tlbie_mutex; static __inline void TLBIE(uint64_t vpn) { @@ -151,8 +151,8 @@ TLBIE(uint64_t vpn) { vpn <<= ADDR_PIDX_SHFT; vpn &= ~(0xffffULL << 48); - mtx_lock_spin(&tlbie_mutex); #ifdef __powerpc64__ + mtx_lock(&tlbie_mutex); __asm __volatile("\ ptesync; \ tlbie %0; \ @@ -160,10 +160,13 @@ TLBIE(uint64_t vpn) { tlbsync; \ ptesync;" :: "r"(vpn) : "memory"); + mtx_unlock(&tlbie_mutex); #else vpn_hi = (uint32_t)(vpn >> 32); vpn_lo = (uint32_t)vpn; + /* Note: spin mutex is to disable exceptions while fiddling MSR */ + mtx_lock_spin(&tlbie_mutex); __asm __volatile("\ mfmsr %0; \ mr %1, %0; \ @@ -181,8 +184,8 @@ TLBIE(uint64_t vpn) { ptesync;" : "=r"(msr), "=r"(scratch) : "r"(vpn_hi), "r"(vpn_lo), "r"(32), "r"(1) : "memory"); -#endif mtx_unlock_spin(&tlbie_mutex); +#endif } #define DISABLE_TRANS(msr) msr = mfmsr(); mtmsr(msr & ~PSL_DR) @@ -413,7 +416,11 @@ moea64_bootstrap_native(mmu_t mmup, vm_o /* * Initialize the TLBIE lock. TLBIE can only be executed by one CPU. */ - mtx_init(&tlbie_mutex, "tlbie mutex", NULL, MTX_SPIN); +#ifdef __powerpc64__ + mtx_init(&tlbie_mutex, "tlbie", NULL, MTX_DEF); +#else + mtx_init(&tlbie_mutex, "tlbie", NULL, MTX_SPIN); +#endif moea64_mid_bootstrap(mmup, kernelstart, kernelend);