From owner-svn-src-all@freebsd.org Thu Jan 23 05:33:06 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 5699A22D09B for ; Thu, 23 Jan 2020 05:33:06 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: from mail-pg1-x534.google.com (mail-pg1-x534.google.com [IPv6:2607:f8b0:4864:20::534]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4839sX6pMTz47tk for ; Thu, 23 Jan 2020 05:33:04 +0000 (UTC) (envelope-from jroberson@jroberson.net) Received: by mail-pg1-x534.google.com with SMTP id k3so786023pgc.3 for ; Wed, 22 Jan 2020 21:33:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=jroberson-net.20150623.gappssmtp.com; s=20150623; h=date:from:to:cc:subject:in-reply-to:message-id:references :user-agent:mime-version; bh=/qtKt17DgkBpWycOZQHUC1QnyjioF5ja76AXOKyO+R8=; b=lA84rbtTTTAeBrK1V/1MuyLP3PyQOAseqK+dp8JjNSjOVhruTBU/yN97UbcosVHrgA VSwyBb1bJhyel58axotmIfd4jOn2XnFeaa5iOVwzSYjTYn3YgYuywaGwq5Po0/ZmPTdW LiM2mnQOZjBHGQyKSo4f6ApvOvANHQx8E4Mn4CLzUo+PLmrzdp7v4Zkm7bCxTAFgbN3I PjLaloxcWUKgBJ7B8RBsrpruGijJ8/q5MYrrkl8yB8PcHLvxYWQx2OhC9isdcju3Wsy2 iIOH6blu97egQQdVOGLs9a2MRJshktrkUiPesOcLDob5HiRNLsYAszF+NvAWT5xi1uhd Ahsg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:from:to:cc:subject:in-reply-to:message-id :references:user-agent:mime-version; bh=/qtKt17DgkBpWycOZQHUC1QnyjioF5ja76AXOKyO+R8=; b=QIAMp+n+LN21YvTpXSm1I49UAtd0wDkx6iR+MKTsAI6hWuwc4dM4QRlV1JIKPttp1S IVYNf2zr/J0JeZXgoaUTkbFhOrhoJGYbcL5KqWlcEv2Le2/1BGy0x26F/5dD4Pc9c7Yh e5qTA/bUFzblhby6UlM4EOFrO3pgVJ5YsV8XfI24uKUv+h5j1IKXATnyZP8nSFOsImgt IFw1S7NPhIL/SxobbVFfIRKVq8FUgwbjuanlPi33CIv/yRAGOgrUXXI6DTbH2x0ROPPw YOuKmHqKLR2/gHrkwsK42Tqibw6mHJSaZ0x3DezGN7lOxUCWQu6E9q1uPLam2kDCADy/ QBIQ== X-Gm-Message-State: APjAAAWHfiBUdWAGoCsRODZcQfgwml17ofX6N0zCC2q3NvDH/SsEE9KQ T4eSVcSyN1naWaEestRbzhRhiQ== X-Google-Smtp-Source: APXvYqx2pPZEfzxGwp4NVHSdeEp8Acb9zYktHMVitOT6mRM7iFYroYjPptTm+mP1XnW0o87vl1GtcQ== X-Received: by 2002:a63:d442:: with SMTP id i2mr2209030pgj.349.1579757582984; Wed, 22 Jan 2020 21:33:02 -0800 (PST) Received: from rrcs-76-81-105-82.west.biz.rr.com (rrcs-76-81-105-82.west.biz.rr.com. [76.81.105.82]) by smtp.gmail.com with ESMTPSA id v4sm950998pgo.63.2020.01.22.21.33.01 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 22 Jan 2020 21:33:02 -0800 (PST) Date: Wed, 22 Jan 2020 19:33:00 -1000 (HST) From: Jeff Roberson X-X-Sender: jroberson@desktop To: Jeff Roberson cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r357028 - head/sys/vm In-Reply-To: <202001230523.00N5NbbB078486@repo.freebsd.org> Message-ID: References: <202001230523.00N5NbbB078486@repo.freebsd.org> User-Agent: Alpine 2.21.9999 (BSF 287 2018-06-16) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4839sX6pMTz47tk X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=jroberson-net.20150623.gappssmtp.com header.s=20150623 header.b=lA84rbtT; dmarc=none; spf=none (mx1.freebsd.org: domain of jroberson@jroberson.net has no SPF policy when checking 2607:f8b0:4864:20::534) smtp.mailfrom=jroberson@jroberson.net X-Spamd-Result: default: False [-4.37 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; R_DKIM_ALLOW(-0.20)[jroberson-net.20150623.gappssmtp.com:s=20150623]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[svn-src-all@freebsd.org]; DMARC_NA(0.00)[jroberson.net]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[jroberson-net.20150623.gappssmtp.com:+]; RCVD_IN_DNSWL_NONE(0.00)[4.3.5.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.b.8.f.7.0.6.2.list.dnswl.org : 127.0.5.0]; R_SPF_NA(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; MID_RHS_NOT_FQDN(0.50)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_TLS_ALL(0.00)[]; IP_SCORE(-2.57)[ip: (-8.93), ipnet: 2607:f8b0::/32(-2.07), asn: 15169(-1.82), country: US(-0.05)] X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 23 Jan 2020 05:33:06 -0000 You may be asking yourself, why did jeff just squish a bunch of code around with little functional change. I could not justify adding more complexity to a 900 line function with a handful of gotos. It is ~300 lines now. I tested the whole set together. I apologize if there are bugs. Other than comment cleanup, I will probably not commit again for a little while. The commits that follow will enable lockless vm object lookup which has given a 10x improvement in massively parallel builds on large machines (100core+). Thanks, Jeff On Thu, 23 Jan 2020, Jeff Roberson wrote: > Author: jeff > Date: Thu Jan 23 05:23:37 2020 > New Revision: 357028 > URL: https://svnweb.freebsd.org/changeset/base/357028 > > Log: > (fault 9/9) Move zero fill into a dedicated function to make the object lock > state more clear. > > Reviewed by: kib > Differential Revision: https://reviews.freebsd.org/D23326 > > Modified: > head/sys/vm/vm_fault.c > > Modified: head/sys/vm/vm_fault.c > ============================================================================== > --- head/sys/vm/vm_fault.c Thu Jan 23 05:22:02 2020 (r357027) > +++ head/sys/vm/vm_fault.c Thu Jan 23 05:23:37 2020 (r357028) > @@ -960,35 +960,8 @@ vm_fault_next(struct faultstate *fs) > */ > VM_OBJECT_ASSERT_WLOCKED(fs->object); > next_object = fs->object->backing_object; > - if (next_object == NULL) { > - /* > - * If there's no object left, fill the page in the top > - * object with zeros. > - */ > - VM_OBJECT_WUNLOCK(fs->object); > - if (fs->object != fs->first_object) { > - vm_object_pip_wakeup(fs->object); > - fs->object = fs->first_object; > - fs->pindex = fs->first_pindex; > - } > - MPASS(fs->first_m != NULL); > - MPASS(fs->m == NULL); > - fs->m = fs->first_m; > - fs->first_m = NULL; > - > - /* > - * Zero the page if necessary and mark it valid. > - */ > - if ((fs->m->flags & PG_ZERO) == 0) { > - pmap_zero_page(fs->m); > - } else { > - VM_CNT_INC(v_ozfod); > - } > - VM_CNT_INC(v_zfod); > - vm_page_valid(fs->m); > - > + if (next_object == NULL) > return (false); > - } > MPASS(fs->first_m != NULL); > KASSERT(fs->object != next_object, ("object loop %p", next_object)); > VM_OBJECT_WLOCK(next_object); > @@ -1002,6 +975,36 @@ vm_fault_next(struct faultstate *fs) > return (true); > } > > +static void > +vm_fault_zerofill(struct faultstate *fs) > +{ > + > + /* > + * If there's no object left, fill the page in the top > + * object with zeros. > + */ > + if (fs->object != fs->first_object) { > + vm_object_pip_wakeup(fs->object); > + fs->object = fs->first_object; > + fs->pindex = fs->first_pindex; > + } > + MPASS(fs->first_m != NULL); > + MPASS(fs->m == NULL); > + fs->m = fs->first_m; > + fs->first_m = NULL; > + > + /* > + * Zero the page if necessary and mark it valid. > + */ > + if ((fs->m->flags & PG_ZERO) == 0) { > + pmap_zero_page(fs->m); > + } else { > + VM_CNT_INC(v_ozfod); > + } > + VM_CNT_INC(v_zfod); > + vm_page_valid(fs->m); > +} > + > /* > * Allocate a page directly or via the object populate method. > */ > @@ -1407,11 +1410,13 @@ RetryFault: > * traverse into a backing object or zero fill if none is > * found. > */ > - if (!vm_fault_next(&fs)) { > - /* Don't try to prefault neighboring pages. */ > - faultcount = 1; > - break; /* break to PAGE HAS BEEN FOUND. */ > - } > + if (vm_fault_next(&fs)) > + continue; > + VM_OBJECT_WUNLOCK(fs.object); > + vm_fault_zerofill(&fs); > + /* Don't try to prefault neighboring pages. */ > + faultcount = 1; > + break; /* break to PAGE HAS BEEN FOUND. */ > } > > /* >