From owner-svn-src-head@freebsd.org Tue Aug 11 21:51:12 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D45A53B7F59; Tue, 11 Aug 2020 21:51:12 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: from mail-io1-xd2e.google.com (mail-io1-xd2e.google.com [IPv6:2607:f8b0:4864:20::d2e]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BR63N3sMLz45ny; Tue, 11 Aug 2020 21:51:12 +0000 (UTC) (envelope-from markjdb@gmail.com) Received: by mail-io1-xd2e.google.com with SMTP id u126so377157iod.12; Tue, 11 Aug 2020 14:51:12 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=Jy/HqZFAf5m6OY3HEyMC8JnrFbexJB3TE29yp2WJI9o=; b=t15qbgTVM+fbk/cpkiILsZjIrRVKm3WjqS8GPemyTGRxS+yuJFctuWgbsT/hXtC9Q7 8OgcuhokGKkeUAb0lrtodqVeYzKzlIdAuVFPu7WYNGRX41fmXdb89jAEu5ZZoHzxtaXj 3rUY4tHTT7LZY7rhHx5QHx5tZZMyA2/oAfhSc29X3lqsZIcainK+WFS75y2bt2/ZtFo8 1aUFhFimgxe0rtkulMRgxC7J7N5xrwjUKG6OvDBiwBUtzIWKcsusu3PXgsdfDLWb4oTo hHQPNBHvgYz4DuFTUYLKehQVntTcNj4noma4eU6VxRJa54u8npUdcglZvBHJskJxTNTb 42tA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:date:from:to:cc:subject:message-id :references:mime-version:content-disposition:in-reply-to; bh=Jy/HqZFAf5m6OY3HEyMC8JnrFbexJB3TE29yp2WJI9o=; b=eF8i9QJdRFXnfiYM/SfsLNfOyRDwks9T4LsK5BDetcymClt3lYpNXxCXr6YDIpmETa b/Lvlz07LgLKNvZ3MrKhfU2/mEiZKC5CjdZOE/PszwtEuQca3hG+8V31VLm+4W7H/um4 oLvZSTBELRNWiUet6N17JHmhe2iESwthNOe/lG3UcAul6j4NiGGDwooHpB49bUKjvMfW q/yeLE+DEfWT1Mk/sdMlUPgUV/+uQ0NiC6w80OOTLo7dSH4cn3gsPjcFYNMfd2tNowg/ dKRuWsC4QC2Gw8Vv9HiuKYmpw2RuIfE+DKzj3bPb56zZ6m2etHTembbq3+eIyhK0n0ZB uqiw== X-Gm-Message-State: AOAM533LSRL5JdDN9Roz+OO8tHh0sS45KrITL6shl76iUfCWcayp9vjL gSCRWY5Qvqk300uTnPiqB2uJFWkR X-Google-Smtp-Source: ABdhPJzdovHRxEacj8FLuTNqCsj8YIsuKWIouX3mZBzcE689DXN+yPc+MTExpJfv3XHwx+ykjoD/NA== X-Received: by 2002:a6b:3954:: with SMTP id g81mr24204050ioa.207.1597182671068; Tue, 11 Aug 2020 14:51:11 -0700 (PDT) Received: from raichu (toroon0560w-lp130-08-67-71-176-35.dsl.bell.ca. [67.71.176.35]) by smtp.gmail.com with ESMTPSA id j10sm4379230ili.86.2020.08.11.14.51.09 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 11 Aug 2020 14:51:10 -0700 (PDT) Sender: Mark Johnston Date: Tue, 11 Aug 2020 17:51:08 -0400 From: Mark Johnston To: Conrad Meyer Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r364129 - head/sys/vm Message-ID: <20200811215108.GB77647@raichu> References: <202008112037.07BKbjsL056699@repo.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <202008112037.07BKbjsL056699@repo.freebsd.org> X-Rspamd-Queue-Id: 4BR63N3sMLz45ny X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; REPLY(-4.00)[] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 11 Aug 2020 21:51:12 -0000 On Tue, Aug 11, 2020 at 08:37:45PM +0000, Conrad Meyer wrote: > Author: cem > Date: Tue Aug 11 20:37:45 2020 > New Revision: 364129 > URL: https://svnweb.freebsd.org/changeset/base/364129 > > Log: > Add support for multithreading the inactive queue pageout within a domain. > > In very high throughput workloads, the inactive scan can become overwhelmed > as you have many cores producing pages and a single core freeing. Since > Mark's introduction of batched pagequeue operations, we can now run multiple > inactive threads working on independent batches. > > To avoid confusing the pid and other control algorithms, I (Jeff) do this in > a mpi-like fan out and collect model that is driven from the primary page > daemon. It decides whether the shortfall can be overcome with a single > thread and if not dispatches multiple threads and waits for their results. > > The heuristic is based on timing the pageout activity and averaging a > pages-per-second variable which is exponentially decayed. This is visible in > sysctl and may be interesting for other purposes. > > I (Jeff) have verified that this does indeed double our paging throughput > when used with two threads. With four we tend to run into other contention > problems. For now I would like to commit this infrastructure with only a > single thread enabled. > > The number of worker threads per domain can be controlled with the > 'vm.pageout_threads_per_domain' tunable. > > Submitted by: jeff (earlier version) > Discussed with: markj > Tested by: pho > Sponsored by: probably Netflix (based on contemporary commits) > Differential Revision: https://reviews.freebsd.org/D21629 > > Modified: > head/sys/vm/vm_meter.c > head/sys/vm/vm_page.c > head/sys/vm/vm_page.h > head/sys/vm/vm_pageout.c > head/sys/vm/vm_pagequeue.h > > Modified: head/sys/vm/vm_page.c > ============================================================================== > --- head/sys/vm/vm_page.c Tue Aug 11 17:54:10 2020 (r364128) > +++ head/sys/vm/vm_page.c Tue Aug 11 20:37:45 2020 (r364129) > @@ -421,7 +421,7 @@ sysctl_vm_page_blacklist(SYSCTL_HANDLER_ARGS) > * In principle, this function only needs to set the flag PG_MARKER. > * Nonetheless, it write busies the page as a safety precaution. > */ > -static void > +void > vm_page_init_marker(vm_page_t marker, int queue, uint16_t aflags) > { > > @@ -2488,7 +2488,7 @@ vm_page_zone_import(void *arg, void **store, int cnt, > * main purpose is to replenish the store of free pages. > */ > if (vmd->vmd_severeset || curproc == pageproc || > - !_vm_domain_allocate(vmd, VM_ALLOC_NORMAL, cnt)) > + !_vm_domain_allocate(vmd, VM_ALLOC_SYSTEM, cnt)) Why did this change? It does not look related to the rest of the change.