From owner-svn-src-head@freebsd.org Wed May 30 20:43:49 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 54BBCEFC4BF; Wed, 30 May 2018 20:43:49 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 0756678DE3; Wed, 30 May 2018 20:43:49 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id DB5B410320; Wed, 30 May 2018 20:43:48 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w4UKhmwE083974; Wed, 30 May 2018 20:43:48 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w4UKhmAn083973; Wed, 30 May 2018 20:43:48 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201805302043.w4UKhmAn083973@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Wed, 30 May 2018 20:43:48 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r334400 - head/sys/i386/i386 X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: head/sys/i386/i386 X-SVN-Commit-Revision: 334400 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 May 2018 20:43:49 -0000 Author: kib Date: Wed May 30 20:43:48 2018 New Revision: 334400 URL: https://svnweb.freebsd.org/changeset/base/334400 Log: Extract code for fast mapping of pte from pmap_extract_and_hold() into the helper function pmap_pte_ufast(). Benchmarked by: bde Tested by: pho Sponsored by: The FreeBSD Foundation Modified: head/sys/i386/i386/pmap.c Modified: head/sys/i386/i386/pmap.c ============================================================================== --- head/sys/i386/i386/pmap.c Wed May 30 20:39:22 2018 (r334399) +++ head/sys/i386/i386/pmap.c Wed May 30 20:43:48 2018 (r334400) @@ -1598,6 +1598,26 @@ pmap_pte_quick3(pmap_t pmap, vm_offset_t va) return (0); } +static pt_entry_t +pmap_pte_ufast(pmap_t pmap, vm_offset_t va, pd_entry_t pde) +{ + pt_entry_t *eh_ptep, pte, *ptep; + + PMAP_LOCK_ASSERT(pmap, MA_OWNED); + pde &= PG_FRAME; + critical_enter(); + eh_ptep = (pt_entry_t *)PCPU_GET(pmap_eh_ptep); + if ((*eh_ptep & PG_FRAME) != pde) { + *eh_ptep = pde | PG_RW | PG_V | PG_A | PG_M; + invlcaddr((void *)PCPU_GET(pmap_eh_va)); + } + ptep = (pt_entry_t *)PCPU_GET(pmap_eh_va) + (i386_btop(va) & + (NPTEPG - 1)); + pte = *ptep; + critical_exit(); + return (pte); +} + /* * Routine: pmap_extract * Function: @@ -1637,8 +1657,8 @@ pmap_extract(pmap_t pmap, vm_offset_t va) vm_page_t pmap_extract_and_hold(pmap_t pmap, vm_offset_t va, vm_prot_t prot) { - pd_entry_t pde, newpf; - pt_entry_t *eh_ptep, pte, *ptep; + pd_entry_t pde; + pt_entry_t pte; vm_page_t m; vm_paddr_t pa; @@ -1658,17 +1678,7 @@ retry: vm_page_hold(m); } } else { - newpf = pde & PG_FRAME; - critical_enter(); - eh_ptep = (pt_entry_t *)PCPU_GET(pmap_eh_ptep); - if ((*eh_ptep & PG_FRAME) != newpf) { - *eh_ptep = newpf | PG_RW | PG_V | PG_A | PG_M; - invlcaddr((void *)PCPU_GET(pmap_eh_va)); - } - ptep = (pt_entry_t *)PCPU_GET(pmap_eh_va) + - (i386_btop(va) & (NPTEPG - 1)); - pte = *ptep; - critical_exit(); + pte = pmap_pte_ufast(pmap, va, pde); if (pte != 0 && ((pte & PG_RW) || (prot & VM_PROT_WRITE) == 0)) { if (vm_page_pa_tryrelock(pmap, pte & PG_FRAME,