From owner-svn-src-all@freebsd.org Mon Nov 25 07:13:05 2019 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BFD531CAC60; Mon, 25 Nov 2019 07:13:05 +0000 (UTC) (envelope-from jeff@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 47Lyt94RCpz44Sg; Mon, 25 Nov 2019 07:13:05 +0000 (UTC) (envelope-from jeff@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 7B7C8917C; Mon, 25 Nov 2019 07:13:05 +0000 (UTC) (envelope-from jeff@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id xAP7D5sc017402; Mon, 25 Nov 2019 07:13:05 GMT (envelope-from jeff@FreeBSD.org) Received: (from jeff@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id xAP7D5WQ017401; Mon, 25 Nov 2019 07:13:05 GMT (envelope-from jeff@FreeBSD.org) Message-Id: <201911250713.xAP7D5WQ017401@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jeff set sender to jeff@FreeBSD.org using -f From: Jeff Roberson Date: Mon, 25 Nov 2019 07:13:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r355082 - head/sys/vm X-SVN-Group: head X-SVN-Commit-Author: jeff X-SVN-Commit-Paths: head/sys/vm X-SVN-Commit-Revision: 355082 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 25 Nov 2019 07:13:05 -0000 Author: jeff Date: Mon Nov 25 07:13:05 2019 New Revision: 355082 URL: https://svnweb.freebsd.org/changeset/base/355082 Log: Move anonymous object copying for fork into its own routine and so that we can avoid locking non-anonymous objects. Reviewed by: kib Differential Revision: https://reviews.freebsd.org/D22472 Modified: head/sys/vm/vm_map.c Modified: head/sys/vm/vm_map.c ============================================================================== --- head/sys/vm/vm_map.c Mon Nov 25 06:16:53 2019 (r355081) +++ head/sys/vm/vm_map.c Mon Nov 25 07:13:05 2019 (r355082) @@ -3722,7 +3722,57 @@ vm_map_check_protection(vm_map_t map, vm_offset_t star return (TRUE); } + /* + * + * vm_map_copy_anon_object: + * + * Copies an anonymous object from an existing map entry to a + * new one. Carries forward the swap charge. May change the + * src object on return. + */ +static void +vm_map_copy_anon_object(vm_map_entry_t src_entry, vm_map_entry_t dst_entry, + vm_offset_t size, vm_ooffset_t *fork_charge) +{ + vm_object_t src_object; + struct ucred *cred; + int charged; + + src_object = src_entry->object.vm_object; + VM_OBJECT_WLOCK(src_object); + charged = ENTRY_CHARGED(src_entry); + vm_object_collapse(src_object); + if ((src_object->flags & OBJ_ONEMAPPING) != 0) { + vm_object_split(src_entry); + src_object = src_entry->object.vm_object; + } + vm_object_reference_locked(src_object); + vm_object_clear_flag(src_object, OBJ_ONEMAPPING); + if (src_entry->cred != NULL && + !(src_entry->eflags & MAP_ENTRY_NEEDS_COPY)) { + KASSERT(src_object->cred == NULL, + ("OVERCOMMIT: vm_map_copy_anon_entry: cred %p", + src_object)); + src_object->cred = src_entry->cred; + src_object->charge = size; + } + VM_OBJECT_WUNLOCK(src_object); + dst_entry->object.vm_object = src_object; + if (charged) { + cred = curthread->td_ucred; + crhold(cred); + dst_entry->cred = cred; + *fork_charge += size; + if (!(src_entry->eflags & MAP_ENTRY_NEEDS_COPY)) { + crhold(cred); + src_entry->cred = cred; + *fork_charge += size; + } + } +} + +/* * vm_map_copy_entry: * * Copies the contents of the source entry to the destination @@ -3739,8 +3789,6 @@ vm_map_copy_entry( vm_object_t src_object; vm_map_entry_t fake_entry; vm_offset_t size; - struct ucred *cred; - int charged; VM_MAP_ASSERT_LOCKED(dst_map); @@ -3766,39 +3814,14 @@ vm_map_copy_entry( */ size = src_entry->end - src_entry->start; if ((src_object = src_entry->object.vm_object) != NULL) { - VM_OBJECT_WLOCK(src_object); - charged = ENTRY_CHARGED(src_entry); if ((src_object->flags & OBJ_ANON) != 0) { - vm_object_collapse(src_object); - if ((src_object->flags & OBJ_ONEMAPPING) != 0) { - vm_object_split(src_entry); - src_object = - src_entry->object.vm_object; - } - } - vm_object_reference_locked(src_object); - vm_object_clear_flag(src_object, OBJ_ONEMAPPING); - if (src_entry->cred != NULL && - !(src_entry->eflags & MAP_ENTRY_NEEDS_COPY)) { - KASSERT(src_object->cred == NULL, - ("OVERCOMMIT: vm_map_copy_entry: cred %p", - src_object)); - src_object->cred = src_entry->cred; - src_object->charge = size; - } - VM_OBJECT_WUNLOCK(src_object); - dst_entry->object.vm_object = src_object; - if (charged) { - cred = curthread->td_ucred; - crhold(cred); - dst_entry->cred = cred; - *fork_charge += size; - if (!(src_entry->eflags & - MAP_ENTRY_NEEDS_COPY)) { - crhold(cred); - src_entry->cred = cred; - *fork_charge += size; - } + vm_map_copy_anon_object(src_entry, dst_entry, + size, fork_charge); + /* May have split/collapsed, reload obj. */ + src_object = src_entry->object.vm_object; + } else { + vm_object_reference(src_object); + dst_entry->object.vm_object = src_object; } src_entry->eflags |= MAP_ENTRY_COW | MAP_ENTRY_NEEDS_COPY;