From owner-svn-src-head@freebsd.org Thu Jan 23 05:07:02 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BC48822C690; Thu, 23 Jan 2020 05:07:02 +0000 (UTC) (envelope-from jeff@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4839HV4ZB3z45pF; Thu, 23 Jan 2020 05:07:02 +0000 (UTC) (envelope-from jeff@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 980B19689; Thu, 23 Jan 2020 05:07:02 +0000 (UTC) (envelope-from jeff@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 00N572hQ066345; Thu, 23 Jan 2020 05:07:02 GMT (envelope-from jeff@FreeBSD.org) Received: (from jeff@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 00N572UF066343; Thu, 23 Jan 2020 05:07:02 GMT (envelope-from jeff@FreeBSD.org) Message-Id: <202001230507.00N572UF066343@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jeff set sender to jeff@FreeBSD.org using -f From: Jeff Roberson Date: Thu, 23 Jan 2020 05:07:02 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r357022 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: jeff X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 357022 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 05:07:02 -0000 Author: jeff Date: Thu Jan 23 05:07:01 2020 New Revision: 357022 URL: https://svnweb.freebsd.org/changeset/base/357022 Log: (fault 3/9) Move map relookup into a dedicated function. Add a new VM return code KERN_RESTART which means, deallocate and restart in fault. Reviewed by: kib, markj Differential Revision: https://reviews.freebsd.org/D23303 Modified: head/sys/vm/vm_fault.c head/sys/vm/vm_param.h Modified: head/sys/vm/vm_fault.c ============================================================================== --- head/sys/vm/vm_fault.c Thu Jan 23 05:05:39 2020 (r357021) +++ head/sys/vm/vm_fault.c Thu Jan 23 05:07:01 2020 (r357022) @@ -795,6 +795,57 @@ vm_fault_lookup(struct faultstate *fs) return (KERN_SUCCESS); } +static int +vm_fault_relookup(struct faultstate *fs) +{ + vm_object_t retry_object; + vm_pindex_t retry_pindex; + vm_prot_t retry_prot; + int result; + + if (!vm_map_trylock_read(fs->map)) + return (KERN_RESTART); + + fs->lookup_still_valid = true; + if (fs->map->timestamp == fs->map_generation) + return (KERN_SUCCESS); + + result = vm_map_lookup_locked(&fs->map, fs->vaddr, fs->fault_type, + &fs->entry, &retry_object, &retry_pindex, &retry_prot, + &fs->wired); + if (result != KERN_SUCCESS) { + /* + * If retry of map lookup would have blocked then + * retry fault from start. + */ + if (result == KERN_FAILURE) + return (KERN_RESTART); + return (result); + } + if (retry_object != fs->first_object || + retry_pindex != fs->first_pindex) + return (KERN_RESTART); + + /* + * Check whether the protection has changed or the object has + * been copied while we left the map unlocked. Changing from + * read to write permission is OK - we leave the page + * write-protected, and catch the write fault. Changing from + * write to read permission means that we can't mark the page + * write-enabled after all. + */ + fs->prot &= retry_prot; + fs->fault_type &= retry_prot; + if (fs->prot == 0) + return (KERN_RESTART); + + /* Reassert because wired may have changed. */ + KASSERT(fs->wired || (fs->fault_flags & VM_FAULT_WIRE) == 0, + ("!wired && VM_FAULT_WIRE")); + + return (KERN_SUCCESS); +} + /* * Wait/Retry if the page is busy. We have to do this if the page is * either exclusive or shared busy because the vm_pager may be using @@ -837,10 +888,8 @@ vm_fault(vm_map_t map, vm_offset_t vaddr, vm_prot_t fa { struct faultstate fs; struct domainset *dset; - vm_object_t next_object, retry_object; + vm_object_t next_object; vm_offset_t e_end, e_start; - vm_pindex_t retry_pindex; - vm_prot_t retry_prot; int ahead, alloc_req, behind, cluster_offset, faultcount; int nera, oom, result, rv; u_char behavior; @@ -1363,56 +1412,12 @@ next: * lookup. */ if (!fs.lookup_still_valid) { - if (!vm_map_trylock_read(fs.map)) { + result = vm_fault_relookup(&fs); + if (result != KERN_SUCCESS) { fault_deallocate(&fs); - goto RetryFault; - } - fs.lookup_still_valid = true; - if (fs.map->timestamp != fs.map_generation) { - result = vm_map_lookup_locked(&fs.map, vaddr, fs.fault_type, - &fs.entry, &retry_object, &retry_pindex, &retry_prot, - &fs.wired); - - /* - * If we don't need the page any longer, put it on the inactive - * list (the easiest thing to do here). If no one needs it, - * pageout will grab it eventually. - */ - if (result != KERN_SUCCESS) { - fault_deallocate(&fs); - - /* - * If retry of map lookup would have blocked then - * retry fault from start. - */ - if (result == KERN_FAILURE) - goto RetryFault; - return (result); - } - if ((retry_object != fs.first_object) || - (retry_pindex != fs.first_pindex)) { - fault_deallocate(&fs); + if (result == KERN_RESTART) goto RetryFault; - } - - /* - * Check whether the protection has changed or the object has - * been copied while we left the map unlocked. Changing from - * read to write permission is OK - we leave the page - * write-protected, and catch the write fault. Changing from - * write to read permission means that we can't mark the page - * write-enabled after all. - */ - fs.prot &= retry_prot; - fs.fault_type &= retry_prot; - if (fs.prot == 0) { - fault_deallocate(&fs); - goto RetryFault; - } - - /* Reassert because wired may have changed. */ - KASSERT(fs.wired || (fs.fault_flags & VM_FAULT_WIRE) == 0, - ("!wired && VM_FAULT_WIRE")); + return (result); } } VM_OBJECT_ASSERT_UNLOCKED(fs.object); Modified: head/sys/vm/vm_param.h ============================================================================== --- head/sys/vm/vm_param.h Thu Jan 23 05:05:39 2020 (r357021) +++ head/sys/vm/vm_param.h Thu Jan 23 05:07:01 2020 (r357022) @@ -114,6 +114,7 @@ struct xswdev { #define KERN_NOT_RECEIVER 7 #define KERN_NO_ACCESS 8 #define KERN_OUT_OF_BOUNDS 9 +#define KERN_RESTART 10 #ifndef PA_LOCK_COUNT #ifdef SMP