From owner-svn-src-all@freebsd.org Fri Sep 6 02:45:47 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C7A94DC978; Fri, 6 Sep 2019 02:45:47 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 46Phkg4kQvz3DYx; Fri, 6 Sep 2019 02:45:47 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 6BD0AE1F3; Fri, 6 Sep 2019 02:45:47 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x862jlF8060145; Fri, 6 Sep 2019 02:45:47 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x862jlfu060144; Fri, 6 Sep 2019 02:45:47 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201909060245.x862jlfu060144@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Fri, 6 Sep 2019 02:45:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r351921 - head/sys/powerpc/aim X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: head/sys/powerpc/aim X-SVN-Commit-Revision: 351921 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 06 Sep 2019 02:45:47 -0000 Author: jhibbits Date: Fri Sep 6 02:45:46 2019 New Revision: 351921 URL: https://svnweb.freebsd.org/changeset/base/351921 Log: powerpc64/pmap: Simplify the code path for moea64_pte_replace_native() Summary: MOEA64_PTE_REPLACE() is called often with the pmap lock held, and sometimes with the page pv lock held. The less work done while holding a lock, the better. Since we are intending to replace the same PTE (same hash index), we don't need to recalculate anything, just flat replace the PTE. This cuts more than 200 instructions off the invalidating code path. In addition, we don't need to replace a PTE that's not occupied by this PVO. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21515 Modified: head/sys/powerpc/aim/moea64_native.c Modified: head/sys/powerpc/aim/moea64_native.c ============================================================================== --- head/sys/powerpc/aim/moea64_native.c Fri Sep 6 02:28:03 2019 (r351920) +++ head/sys/powerpc/aim/moea64_native.c Fri Sep 6 02:45:46 2019 (r351921) @@ -358,6 +358,45 @@ moea64_pte_unset_native(mmu_t mmu, struct pvo_entry *p } static int64_t +moea64_pte_replace_inval_native(mmu_t mmu, struct pvo_entry *pvo, + volatile struct lpte *pt) +{ + struct lpte properpt; + uint64_t ptelo; + + moea64_pte_from_pvo(pvo, &properpt); + + rw_rlock(&moea64_eviction_lock); + if ((be64toh(pt->pte_hi & LPTE_AVPN_MASK)) != + (properpt.pte_hi & LPTE_AVPN_MASK)) { + /* Evicted */ + STAT_MOEA64(moea64_pte_overflow--); + rw_runlock(&moea64_eviction_lock); + return (-1); + } + + /* + * Replace the pte, briefly locking it to collect RC bits. No + * atomics needed since this is protected against eviction by the lock. + */ + isync(); + critical_enter(); + pt->pte_hi = be64toh((pt->pte_hi & ~LPTE_VALID) | LPTE_LOCKED); + PTESYNC(); + TLBIE(pvo->pvo_vpn); + ptelo = be64toh(pt->pte_lo); + EIEIO(); + pt->pte_lo = htobe64(properpt.pte_lo); + EIEIO(); + pt->pte_hi = htobe64(properpt.pte_hi); /* Release lock */ + PTESYNC(); + critical_exit(); + rw_runlock(&moea64_eviction_lock); + + return (ptelo & (LPTE_CHG | LPTE_REF)); +} + +static int64_t moea64_pte_replace_native(mmu_t mmu, struct pvo_entry *pvo, int flags) { volatile struct lpte *pt = moea64_pteg_table + pvo->pvo_pte.slot; @@ -379,8 +418,7 @@ moea64_pte_replace_native(mmu_t mmu, struct pvo_entry rw_runlock(&moea64_eviction_lock); } else { /* Otherwise, need reinsertion and deletion */ - ptelo = moea64_pte_unset_native(mmu, pvo); - moea64_pte_insert_native(mmu, pvo); + ptelo = moea64_pte_replace_inval_native(mmu, pvo, pt); } return (ptelo);