From owner-svn-src-all@freebsd.org Sat Oct 7 20:22:06 2017 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 93B9EE41641; Sat, 7 Oct 2017 20:22:06 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 254E86F18D; Sat, 7 Oct 2017 20:22:06 +0000 (UTC) (envelope-from alc@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id v97KM5Yx062904; Sat, 7 Oct 2017 20:22:05 GMT (envelope-from alc@FreeBSD.org) Received: (from alc@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id v97KM5jf062759; Sat, 7 Oct 2017 20:22:05 GMT (envelope-from alc@FreeBSD.org) Message-Id: <201710072022.v97KM5jf062759@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: alc set sender to alc@FreeBSD.org using -f From: Alan Cox Date: Sat, 7 Oct 2017 20:22:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-11@freebsd.org Subject: svn commit: r324399 - stable/11/sys/vm X-SVN-Group: stable-11 X-SVN-Commit-Author: alc X-SVN-Commit-Paths: stable/11/sys/vm X-SVN-Commit-Revision: 324399 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Oct 2017 20:22:06 -0000 Author: alc Date: Sat Oct 7 20:22:04 2017 New Revision: 324399 URL: https://svnweb.freebsd.org/changeset/base/324399 Log: MFC r321386,321393 Utilize pmap_enter(..., psind=1) in vm_fault_soft_fast() on amd64. (The Differential Revision discusses the benefits of this change.) Add a function, vm_reserv_to_superpage(), that returns the superpage containing the specified base page. Modified: stable/11/sys/vm/vm_fault.c stable/11/sys/vm/vm_reserv.c stable/11/sys/vm/vm_reserv.h Directory Properties: stable/11/ (props changed) Modified: stable/11/sys/vm/vm_fault.c ============================================================================== --- stable/11/sys/vm/vm_fault.c Sat Oct 7 20:20:01 2017 (r324398) +++ stable/11/sys/vm/vm_fault.c Sat Oct 7 20:22:04 2017 (r324399) @@ -267,8 +267,12 @@ static int vm_fault_soft_fast(struct faultstate *fs, vm_offset_t vaddr, vm_prot_t prot, int fault_type, int fault_flags, boolean_t wired, vm_page_t *m_hold) { - vm_page_t m; - int rv; + vm_page_t m, m_map; +#if defined(__amd64__) && VM_NRESERVLEVEL > 0 + vm_page_t m_super; + int flags; +#endif + int psind, rv; MPASS(fs->vp == NULL); m = vm_page_lookup(fs->first_object, fs->first_pindex); @@ -276,14 +280,46 @@ vm_fault_soft_fast(struct faultstate *fs, vm_offset_t if (m == NULL || ((prot & VM_PROT_WRITE) != 0 && vm_page_busied(m)) || m->valid != VM_PAGE_BITS_ALL) return (KERN_FAILURE); - rv = pmap_enter(fs->map->pmap, vaddr, m, prot, fault_type | - PMAP_ENTER_NOSLEEP | (wired ? PMAP_ENTER_WIRED : 0), 0); + m_map = m; + psind = 0; +#if defined(__amd64__) && VM_NRESERVLEVEL > 0 + if ((m->flags & PG_FICTITIOUS) == 0 && + (m_super = vm_reserv_to_superpage(m)) != NULL && + rounddown2(vaddr, pagesizes[m_super->psind]) >= fs->entry->start && + roundup2(vaddr + 1, pagesizes[m_super->psind]) <= fs->entry->end && + (vaddr & (pagesizes[m_super->psind] - 1)) == (VM_PAGE_TO_PHYS(m) & + (pagesizes[m_super->psind] - 1)) && + pmap_ps_enabled(fs->map->pmap)) { + flags = PS_ALL_VALID; + if ((prot & VM_PROT_WRITE) != 0) { + /* + * Create a superpage mapping allowing write access + * only if none of the constituent pages are busy and + * all of them are already dirty (except possibly for + * the page that was faulted on). + */ + flags |= PS_NONE_BUSY; + if ((fs->first_object->flags & OBJ_UNMANAGED) == 0) + flags |= PS_ALL_DIRTY; + } + if (vm_page_ps_test(m_super, flags, m)) { + m_map = m_super; + psind = m_super->psind; + vaddr = rounddown2(vaddr, pagesizes[psind]); + /* Preset the modified bit for dirty superpages. */ + if ((flags & PS_ALL_DIRTY) != 0) + fault_type |= VM_PROT_WRITE; + } + } +#endif + rv = pmap_enter(fs->map->pmap, vaddr, m_map, prot, fault_type | + PMAP_ENTER_NOSLEEP | (wired ? PMAP_ENTER_WIRED : 0), psind); if (rv != KERN_SUCCESS) return (rv); vm_fault_fill_hold(m_hold, m); vm_fault_dirty(fs->entry, m, prot, fault_type, fault_flags, false); VM_OBJECT_RUNLOCK(fs->first_object); - if (!wired) + if (psind == 0 && !wired) vm_fault_prefault(fs, vaddr, PFBAK, PFFOR); vm_map_lookup_done(fs->map, fs->entry); curthread->td_ru.ru_minflt++; Modified: stable/11/sys/vm/vm_reserv.c ============================================================================== --- stable/11/sys/vm/vm_reserv.c Sat Oct 7 20:20:01 2017 (r324398) +++ stable/11/sys/vm/vm_reserv.c Sat Oct 7 20:22:04 2017 (r324399) @@ -1120,4 +1120,18 @@ vm_reserv_startup(vm_offset_t *vaddr, vm_paddr_t end, return (new_end); } +/* + * Returns the superpage containing the given page. + */ +vm_page_t +vm_reserv_to_superpage(vm_page_t m) +{ + vm_reserv_t rv; + + VM_OBJECT_ASSERT_LOCKED(m->object); + rv = vm_reserv_from_page(m); + return (rv->object == m->object && rv->popcnt == VM_LEVEL_0_NPAGES ? + rv->pages : NULL); +} + #endif /* VM_NRESERVLEVEL > 0 */ Modified: stable/11/sys/vm/vm_reserv.h ============================================================================== --- stable/11/sys/vm/vm_reserv.h Sat Oct 7 20:20:01 2017 (r324398) +++ stable/11/sys/vm/vm_reserv.h Sat Oct 7 20:22:04 2017 (r324399) @@ -64,6 +64,7 @@ void vm_reserv_rename(vm_page_t m, vm_object_t new_ob int vm_reserv_size(int level); vm_paddr_t vm_reserv_startup(vm_offset_t *vaddr, vm_paddr_t end, vm_paddr_t high_water); +vm_page_t vm_reserv_to_superpage(vm_page_t m); #endif /* VM_NRESERVLEVEL > 0 */ #endif /* _KERNEL */