From owner-svn-src-head@freebsd.org Tue Apr 10 16:02:41 2018 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 0D202F8366C; Tue, 10 Apr 2018 16:02:41 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 998FE71483; Tue, 10 Apr 2018 16:02:40 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1f5vjB-000CXN-KT; Tue, 10 Apr 2018 19:02:37 +0300 Date: Tue, 10 Apr 2018 19:02:37 +0300 From: Slawa Olhovchenkov To: Mark Johnston Cc: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: Re: svn commit: r332365 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs Message-ID: <20180410160237.GP4305@zxy.spb.ru> References: <201804101356.w3ADu6Jr072766@repo.freebsd.org> <20180410140957.GG6612@zxy.spb.ru> <20180410151733.GA64021@raichu> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20180410151733.GA64021@raichu> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 10 Apr 2018 16:02:41 -0000 On Tue, Apr 10, 2018 at 11:17:33AM -0400, Mark Johnston wrote: > On Tue, Apr 10, 2018 at 05:09:57PM +0300, Slawa Olhovchenkov wrote: > > On Tue, Apr 10, 2018 at 01:56:06PM +0000, Mark Johnston wrote: > > > > > Author: markj > > > Date: Tue Apr 10 13:56:06 2018 > > > New Revision: 332365 > > > URL: https://svnweb.freebsd.org/changeset/base/332365 > > > > > > Log: > > > Set zfs_arc_free_target to v_free_target. > > > > > > Page daemon output is now regulated by a PID controller with a setpoint > > > of v_free_target. Moreover, the page daemon now wakes up regularly > > > rather than waiting for a wakeup from another thread. This means that > > > the free page count is unlikely to drop below the old > > > zfs_arc_free_target value, and as a result the ARC was not readily > > > freeing pages under memory pressure. Address the immediate problem by > > > updating zfs_arc_free_target to match the page daemon's new behaviour. > > > > Can you explain some more about new page daemon algo (and reclaim zone > > free memory)? > > The old algorithm was pretty simple: there was a free page target and > below that, a wakeup threshold. Any time a thread allocated a page and > in so doing caused the free page count to drop below the wakeup > threshold, that thread would wake up the page daemon, which would scan > the inactive queue and free pages until the free target is reached, or > the end of the inactive queue was reached. > > This is simple and easy to reason about, but has some drawbacks. When > memory pressure is constant, it leads to bursts of CPU usage and lock > contention. The static watermarks may also be insufficient for some > demanding workloads. In particular, the wakeup threshold might be too > low, thus allowing the free page count to drop to dangerous levels and > triggering expensive memory shortage handling (i.e., VM_WAIT). > > The new algorithm uses a control loop to dynamically compute a target > for each scan of the inactive queue. The loop takes as input the > magnitude of the page shortage (v_free_target - v_free_count) and keeps > track of the rate of change of this difference (i.e., the rate at which > free pages are being consumed) and the sum of this difference over time > (i.e., a cumulative value for the magnitude of recent page shortages). > These factors are used to compute "shortage", the number of pages to > reclaim with the goal of maintaining a free page count of v_free_target. > > The effect of the new algorithm is that the page daemon runs more > frequently but for shorter durations, so its CPU usage is more even. It > responds dynamically to the demands of the workload, so the shortcomings > of a pair of static watermarks are gone. > > r329882 doesn't really change anything with respect to reclamation of > pages from UMA zones. There are some plans to address shortcomings there > in the near future though. Thank, very nice explain. IMHO, in this case for ZFS best is old zfs_arc_free_target: too close zfs_arc_free_target to vm_cnt.v_free_min can cause too often run arc_target correction and cause CPU consumption and memory subsystem overuse. ZFS need more correct pressure, ZFS-specific, and I am try it in D7538. > > PS: zfs need some more time for free pages from ARC. Also, vanila zfs > > have broken logic for count used and free ARC's memory. For most > > correctly count system-wide used and free memory need accounting > > in-zone free memory. > > Yes, there is a number of problems in this area predating r329882. This > commit is really just a bandaid.