From owner-svn-src-head@freebsd.org Fri Jul 19 20:47:36 2019 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 13895AB743; Fri, 19 Jul 2019 20:47:36 +0000 (UTC) (envelope-from dougm@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E629885A28; Fri, 19 Jul 2019 20:47:35 +0000 (UTC) (envelope-from dougm@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id BC14126C4C; Fri, 19 Jul 2019 20:47:35 +0000 (UTC) (envelope-from dougm@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x6JKlZHa053731; Fri, 19 Jul 2019 20:47:35 GMT (envelope-from dougm@FreeBSD.org) Received: (from dougm@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x6JKlZru053730; Fri, 19 Jul 2019 20:47:35 GMT (envelope-from dougm@FreeBSD.org) Message-Id: <201907192047.x6JKlZru053730@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: dougm set sender to dougm@FreeBSD.org using -f From: Doug Moore Date: Fri, 19 Jul 2019 20:47:35 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r350155 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: dougm X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 350155 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: E629885A28 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.97 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.97)[-0.969,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 19 Jul 2019 20:47:36 -0000 Author: dougm Date: Fri Jul 19 20:47:35 2019 New Revision: 350155 URL: https://svnweb.freebsd.org/changeset/base/350155 Log: Define vm_map_entry_in_transition to handle an in-transition map entry, combining code currently in vm_map_unwire and vm_map_wire_locked into a single function, called by each of them for entries in transition. Discussed with: kib, markj Reviewed by: alc Approved by: kib, markj (mentors, implicit) Tested by: pho Differential Revision: https://reviews.freebsd.org/D20833 Modified: head/sys/vm/vm_map.c Modified: head/sys/vm/vm_map.c ============================================================================== --- head/sys/vm/vm_map.c Fri Jul 19 20:04:31 2019 (r350154) +++ head/sys/vm/vm_map.c Fri Jul 19 20:47:35 2019 (r350155) @@ -2837,6 +2837,55 @@ vm_map_inherit(vm_map_t map, vm_offset_t start, vm_off } /* + * vm_map_entry_in_transition: + * + * Release the map lock, and sleep until the entry is no longer in + * transition. Awake and acquire the map lock. If the map changed while + * another held the lock, lookup a possibly-changed entry at or after the + * 'start' position of the old entry. + */ +static vm_map_entry_t +vm_map_entry_in_transition(vm_map_t map, vm_offset_t in_start, + vm_offset_t *io_end, bool holes_ok, vm_map_entry_t in_entry) +{ + vm_map_entry_t entry; + vm_offset_t start; + u_int last_timestamp; + + VM_MAP_ASSERT_LOCKED(map); + KASSERT((in_entry->eflags & MAP_ENTRY_IN_TRANSITION) != 0, + ("not in-tranition map entry %p", in_entry)); + /* + * We have not yet clipped the entry. + */ + start = MAX(in_start, in_entry->start); + in_entry->eflags |= MAP_ENTRY_NEEDS_WAKEUP; + last_timestamp = map->timestamp; + if (vm_map_unlock_and_wait(map, 0)) { + /* + * Allow interruption of user wiring/unwiring? + */ + } + vm_map_lock(map); + if (last_timestamp + 1 == map->timestamp) + return (in_entry); + + /* + * Look again for the entry because the map was modified while it was + * unlocked. Specifically, the entry may have been clipped, merged, or + * deleted. + */ + if (!vm_map_lookup_entry(map, start, &entry)) { + if (!holes_ok) { + *io_end = start; + return (NULL); + } + entry = entry->next; + } + return (entry); +} + +/* * vm_map_unwire: * * Implements both kernel and user unwiring. @@ -2845,11 +2894,9 @@ int vm_map_unwire(vm_map_t map, vm_offset_t start, vm_offset_t end, int flags) { - vm_map_entry_t entry, first_entry, tmp_entry; - vm_offset_t saved_start; - unsigned int last_timestamp; + vm_map_entry_t entry, first_entry; int rv; - bool holes_ok, need_wakeup, user_unwire; + bool first_iteration, holes_ok, need_wakeup, user_unwire; if (start == end) return (KERN_SUCCESS); @@ -2865,7 +2912,7 @@ vm_map_unwire(vm_map_t map, vm_offset_t start, vm_offs return (KERN_INVALID_ADDRESS); } } - last_timestamp = map->timestamp; + first_iteration = true; entry = first_entry; rv = KERN_SUCCESS; while (entry->start < end) { @@ -2873,48 +2920,20 @@ vm_map_unwire(vm_map_t map, vm_offset_t start, vm_offs /* * We have not yet clipped the entry. */ - saved_start = (start >= entry->start) ? start : - entry->start; - entry->eflags |= MAP_ENTRY_NEEDS_WAKEUP; - if (vm_map_unlock_and_wait(map, 0)) { - /* - * Allow interruption of user unwiring? - */ - } - vm_map_lock(map); - if (last_timestamp+1 != map->timestamp) { - /* - * Look again for the entry because the map was - * modified while it was unlocked. - * Specifically, the entry may have been - * clipped, merged, or deleted. - */ - if (!vm_map_lookup_entry(map, saved_start, - &tmp_entry)) { - if (holes_ok) - tmp_entry = tmp_entry->next; - else { - if (saved_start == start) { - /* - * First_entry has been deleted. - */ - vm_map_unlock(map); - return (KERN_INVALID_ADDRESS); - } - end = saved_start; - rv = KERN_INVALID_ADDRESS; - break; - } + entry = vm_map_entry_in_transition(map, start, &end, + holes_ok, entry); + if (entry == NULL) { + if (first_iteration) { + vm_map_unlock(map); + return (KERN_INVALID_ADDRESS); } - if (entry == first_entry) - first_entry = tmp_entry; - else - first_entry = NULL; - entry = tmp_entry; + rv = KERN_INVALID_ADDRESS; + break; } - last_timestamp = map->timestamp; + first_entry = first_iteration ? entry : NULL; continue; } + first_iteration = false; vm_map_clip_start(map, entry, start); vm_map_clip_end(map, entry, end); /* @@ -3081,7 +3100,7 @@ vm_map_wire_locked(vm_map_t map, vm_offset_t start, vm u_long npages; u_int last_timestamp; int rv; - bool holes_ok, need_wakeup, user_wire; + bool first_iteration, holes_ok, need_wakeup, user_wire; vm_prot_t prot; VM_MAP_ASSERT_LOCKED(map); @@ -3100,54 +3119,25 @@ vm_map_wire_locked(vm_map_t map, vm_offset_t start, vm else return (KERN_INVALID_ADDRESS); } - last_timestamp = map->timestamp; + first_iteration = true; entry = first_entry; while (entry->start < end) { if (entry->eflags & MAP_ENTRY_IN_TRANSITION) { /* * We have not yet clipped the entry. */ - saved_start = (start >= entry->start) ? start : - entry->start; - entry->eflags |= MAP_ENTRY_NEEDS_WAKEUP; - if (vm_map_unlock_and_wait(map, 0)) { - /* - * Allow interruption of user wiring? - */ + entry = vm_map_entry_in_transition(map, start, &end, + holes_ok, entry); + if (entry == NULL) { + if (first_iteration) + return (KERN_INVALID_ADDRESS); + rv = KERN_INVALID_ADDRESS; + goto done; } - vm_map_lock(map); - if (last_timestamp + 1 != map->timestamp) { - /* - * Look again for the entry because the map was - * modified while it was unlocked. - * Specifically, the entry may have been - * clipped, merged, or deleted. - */ - if (!vm_map_lookup_entry(map, saved_start, - &tmp_entry)) { - if (holes_ok) - tmp_entry = tmp_entry->next; - else { - if (saved_start == start) { - /* - * first_entry has been deleted. - */ - return (KERN_INVALID_ADDRESS); - } - end = saved_start; - rv = KERN_INVALID_ADDRESS; - goto done; - } - } - if (entry == first_entry) - first_entry = tmp_entry; - else - first_entry = NULL; - entry = tmp_entry; - } - last_timestamp = map->timestamp; + first_entry = first_iteration ? entry : NULL; continue; } + first_iteration = false; vm_map_clip_start(map, entry, start); vm_map_clip_end(map, entry, end); /* @@ -3185,6 +3175,7 @@ vm_map_wire_locked(vm_map_t map, vm_offset_t start, vm */ saved_start = entry->start; saved_end = entry->end; + last_timestamp = map->timestamp; vm_map_busy(map); vm_map_unlock(map); @@ -3230,7 +3221,6 @@ vm_map_wire_locked(vm_map_t map, vm_offset_t start, vm entry = entry->next; } } - last_timestamp = map->timestamp; if (rv != KERN_SUCCESS) { vm_map_wire_entry_failure(map, entry, faddr); if (user_wire)