From owner-svn-src-all@FreeBSD.ORG Mon May 18 01:27:37 2009 Return-Path: Delivered-To: svn-src-all@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2D705106564A; Mon, 18 May 2009 01:27:37 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from mx.egr.msu.edu (surfnturf.egr.msu.edu [35.9.37.164]) by mx1.freebsd.org (Postfix) with ESMTP id E30C18FC0A; Mon, 18 May 2009 01:27:36 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from localhost (localhost [127.0.0.1]) by mx.egr.msu.edu (Postfix) with ESMTP id EA9A171F583; Sun, 17 May 2009 21:12:21 -0400 (EDT) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mx.egr.msu.edu ([127.0.0.1]) by localhost (surfnturf.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id FIGNUnOUqjio; Sun, 17 May 2009 21:12:21 -0400 (EDT) Received: from localhost (daemon.egr.msu.edu [35.9.44.65]) by mx.egr.msu.edu (Postfix) with ESMTP id C6D6271F561; Sun, 17 May 2009 21:12:21 -0400 (EDT) Received: by localhost (Postfix, from userid 21281) id B8024471; Sun, 17 May 2009 21:12:21 -0400 (EDT) Date: Sun, 17 May 2009 21:12:21 -0400 From: Adam McDougall To: Kip Macy Message-ID: <20090518011221.GZ82547@egr.msu.edu> References: <200905072057.n47Kv6wW067998@svn.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200905072057.n47Kv6wW067998@svn.freebsd.org> User-Agent: Mutt/1.5.19 (2009-01-05) Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org Subject: Re: svn commit: r191902 - head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 May 2009 01:27:37 -0000 On Thu, May 07, 2009 at 08:57:06PM +0000, Kip Macy wrote: Author: kmacy Date: Thu May 7 20:57:06 2009 New Revision: 191902 URL: http://svn.freebsd.org/changeset/base/191902 Log: Allow the VM to provide backpressure on the ARC cache as it does on Solaris. Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Modified: head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c ============================================================================== --- head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu May 7 20:39:23 2009 (r191901) +++ head/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c Thu May 7 20:57:06 2009 (r191902) @@ -131,6 +131,8 @@ #include #include +#include + static kmutex_t arc_reclaim_thr_lock; static kcondvar_t arc_reclaim_thr_cv; /* used to signal reclaim thr */ static uint8_t arc_thread_exit; @@ -1809,6 +1811,13 @@ arc_reclaim_needed(void) #ifdef _KERNEL + /* + * If pages are needed or we're within 2048 pages + * of needing to page need to reclaim + */ + if (vm_pages_needed || (vm_paging_target() > -2048)) + return (1); + if (needfree) return (1); This seems to cause a problem for me because it appears to put undue pressure on the arc when I think it shouldn't, and it seems the shrunk arc causes my performance to degrade severely. I have an amd64 server with 20G of ram that runs ftp, cvsupd, rsyncd, two http daemons, etc and I have vfs.zfs.arc_max=1024M for now. I ran "top | grep Mem" and sysctl kstat.zfs.misc.arcstats.size every 10 seconds shortly after the system came up and watched the Free column slowly drop over the next few hours. The arcstats.size will generally stay quite close to 1024M. What I see is at this point: Mem: 610M Active, 16G Inact, 2226M Wired, 48M Cache, 144K Buf, 1043M Free kstat.zfs.misc.arcstats.size: 1073922176 Free will temporarily cease to decrease while arcstats.size starts to decrease over the next few minutes at about the same rate Free used to decrease, until it hits: Mem: 613M Active, 16G Inact, 1819M Wired, 50M Cache, 144K Buf, 1023M Free kstat.zfs.misc.arcstats.size: 458326208 then the arcstats.size stays around this depressed size while Free continues to float down towards ~650M and Wired down to ~1796M where they stabilize. When I revert this svn rev, arcstats.size remains around 1024M the entire time, and performance is fine. None of my services seem to be putting any real pressure on the system and it is not swapping, my novice conclusion is this svn change allows the lack of "Free" memory to put undue pressure on the arc ignoring Inact etc. I also tried setting vfs.zfs.arc_min=950M which I think was helpful but perhaps not 100%, but I don't have detailed stats on that; in any case it would be a new workaround to add, and I'm trying hard at this point to be rid of workarounds so I can help test "bare" ZFS and any experimental patches that come up. Thanks for your help and any input on this issue.