From owner-svn-src-all@FreeBSD.ORG Mon Aug 25 21:21:30 2014 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 6D90DCA7; Mon, 25 Aug 2014 21:21:30 +0000 (UTC) Received: from svn.freebsd.org (svn.freebsd.org [IPv6:2001:1900:2254:2068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4D9B43E54; Mon, 25 Aug 2014 21:21:30 +0000 (UTC) Received: from svn.freebsd.org ([127.0.1.70]) by svn.freebsd.org (8.14.9/8.14.9) with ESMTP id s7PLLUaN043684; Mon, 25 Aug 2014 21:21:30 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by svn.freebsd.org (8.14.9/8.14.9/Submit) id s7PLLUlC043683; Mon, 25 Aug 2014 21:21:30 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201408252121.s7PLLUlC043683@svn.freebsd.org> X-Authentication-Warning: svn.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Mon, 25 Aug 2014 21:21:30 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-stable@freebsd.org, svn-src-stable-10@freebsd.org Subject: svn commit: r270630 - stable/10/sys/vm X-SVN-Group: stable-10 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Aug 2014 21:21:30 -0000 Author: kib Date: Mon Aug 25 21:21:29 2014 New Revision: 270630 URL: http://svnweb.freebsd.org/changeset/base/270630 Log: MFC r270011: Implement 'fast path' for the vm page fault handler. MFC r270387 (by alc): Relax one of the conditions for mapping a page on the fast path. Approved by: re (gjb) Modified: stable/10/sys/vm/vm_fault.c Directory Properties: stable/10/ (props changed) Modified: stable/10/sys/vm/vm_fault.c ============================================================================== --- stable/10/sys/vm/vm_fault.c Mon Aug 25 21:19:08 2014 (r270629) +++ stable/10/sys/vm/vm_fault.c Mon Aug 25 21:21:29 2014 (r270630) @@ -237,6 +237,7 @@ vm_fault_hold(vm_map_t map, vm_offset_t int hardfault; struct faultstate fs; struct vnode *vp; + vm_page_t m; int locked, error; hardfault = 0; @@ -290,6 +291,56 @@ RetryFault:; goto RetryFault; } + if (wired) + fault_type = prot | (fault_type & VM_PROT_COPY); + + if (fs.vp == NULL /* avoid locked vnode leak */ && + (fault_flags & (VM_FAULT_CHANGE_WIRING | VM_FAULT_DIRTY)) == 0 && + /* avoid calling vm_object_set_writeable_dirty() */ + ((prot & VM_PROT_WRITE) == 0 || + fs.first_object->type != OBJT_VNODE || + (fs.first_object->flags & OBJ_MIGHTBEDIRTY) != 0)) { + VM_OBJECT_RLOCK(fs.first_object); + if ((prot & VM_PROT_WRITE) != 0 && + fs.first_object->type == OBJT_VNODE && + (fs.first_object->flags & OBJ_MIGHTBEDIRTY) == 0) + goto fast_failed; + m = vm_page_lookup(fs.first_object, fs.first_pindex); + /* A busy page can be mapped for read|execute access. */ + if (m == NULL || ((prot & VM_PROT_WRITE) != 0 && + vm_page_busied(m)) || m->valid != VM_PAGE_BITS_ALL) + goto fast_failed; + result = pmap_enter(fs.map->pmap, vaddr, m, prot, + fault_type | PMAP_ENTER_NOSLEEP | (wired ? PMAP_ENTER_WIRED : + 0), 0); + if (result != KERN_SUCCESS) + goto fast_failed; + if (m_hold != NULL) { + *m_hold = m; + vm_page_lock(m); + vm_page_hold(m); + vm_page_unlock(m); + } + if ((fault_type & VM_PROT_WRITE) != 0 && + (m->oflags & VPO_UNMANAGED) == 0) { + vm_page_dirty(m); + vm_pager_page_unswapped(m); + } + VM_OBJECT_RUNLOCK(fs.first_object); + if (!wired) + vm_fault_prefault(&fs, vaddr, 0, 0); + vm_map_lookup_done(fs.map, fs.entry); + curthread->td_ru.ru_minflt++; + return (KERN_SUCCESS); +fast_failed: + if (!VM_OBJECT_TRYUPGRADE(fs.first_object)) { + VM_OBJECT_RUNLOCK(fs.first_object); + VM_OBJECT_WLOCK(fs.first_object); + } + } else { + VM_OBJECT_WLOCK(fs.first_object); + } + /* * Make a reference to this object to prevent its disposal while we * are messing with it. Once we have the reference, the map is free @@ -300,15 +351,11 @@ RetryFault:; * truncation operations) during I/O. This must be done after * obtaining the vnode lock in order to avoid possible deadlocks. */ - VM_OBJECT_WLOCK(fs.first_object); vm_object_reference_locked(fs.first_object); vm_object_pip_add(fs.first_object, 1); fs.lookup_still_valid = TRUE; - if (wired) - fault_type = prot | (fault_type & VM_PROT_COPY); - fs.first_m = NULL; /*