From owner-freebsd-fs@FreeBSD.ORG Tue Sep 28 13:36:47 2010 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 1C76E106564A; Tue, 28 Sep 2010 13:36:47 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.freebsd.org (Postfix) with ESMTP id 27D078FC0C; Tue, 28 Sep 2010 13:36:45 +0000 (UTC) Received: from odyssey.starpoint.kiev.ua (alpha-e.starpoint.kiev.ua [212.40.38.101]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id QAA04596; Tue, 28 Sep 2010 16:36:41 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <4CA1EF69.4040402@icyb.net.ua> Date: Tue, 28 Sep 2010 16:36:41 +0300 From: Andriy Gapon User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.9) Gecko/20100920 Lightning/1.0b2 Thunderbird/3.1.4 MIME-Version: 1.0 To: Jeremy Chadwick References: <4CA1D06C.9050305@digiware.nl> <20100928115047.GA62142__15392.0458550148$1285675457$gmane$org@icarus.home.lan> <4CA1DDE9.8090107@icyb.net.ua> <20100928132355.GA63149@icarus.home.lan> In-Reply-To: <20100928132355.GA63149@icarus.home.lan> X-Enigmail-Version: 1.1.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: stable@freebsd.org, fs@freebsd.org Subject: Re: Still getting kmem exhausted panic X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Sep 2010 13:36:47 -0000 on 28/09/2010 16:23 Jeremy Chadwick said the following: > On Tue, Sep 28, 2010 at 03:22:01PM +0300, Andriy Gapon wrote: >> on 28/09/2010 14:50 Jeremy Chadwick said the following: >>> I believe the trick -- Andriy, please correct me if I'm wrong -- is the >> >> Wouldn't hurt to CC me, so that I could do it :-) >> >>> tuning of vfs.zfs.arc_max, which is now a hard limit rather than a "high >>> watermark". >> >> Not sure what you mean here. >> What is hard limit, what is high watermark, what is the difference and when is >> "now"? :-) > > There was some speculation on the part of users a while back which lead > to this understanding. Folks were seeing actual ARC usage higher than > what vfs.zfs.arc_max was set to (automatically or administratively). I > believe it started here: > > http://www.mailinglistarchive.com/freebsd-current@freebsd.org/msg28884.html > > With the "high-water mark" statements being here: > > http://www.mailinglistarchive.com/freebsd-current@freebsd.org/msg28887.html > http://unix.derkeiler.com/Mailing-Lists/FreeBSD/stable/2010-04/msg00129.html > > The term implies that there is not an explicitly hard limit on the ARC > utilisation/growth. As stated in the unix.derkeiler.com URL above, this > behaviour was in fact changed. Why/when/how? I had to go digging up > the commits -- this took me some time. Here they are, labelled r197816, > for RELENG_8 and RELENG_7 respectively. These were both committed on > 2010/01/08 UTC: > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c#rev1.22.2.2 > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c#rev1.15.2.6 > > In HEAD/CURRENT (yet to be MFC'd), it looks like above code got removed > on 2010/09/17 UTC, citing they should be "enforced by actual > calculations of delta": > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c#rev1.46 > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c#rev1.45 > > So what's this "delta" code piece that's mentioned? That appears to be > have been committed to RELENG_8 on 2010/05/24 UTC (thus, between the > above two dates): > > http://www.freebsd.org/cgi/cvsweb.cgi/src/sys/cddl/contrib/opensolaris/uts/common/fs/zfs/arc.c#rev1.22.2.4 > > (Side note: the "delta stuff" was never committed to RELENG_7 -- and > that's fine. I'm pointing this out not out of retaliation or insult, > but because people will almost certainly Google, find this post, and > wonder if their 7.x machines might be affected.) > > This situation with the ARC, and all its changes over time, is one of > the reasons why I rant aggressively about the need for more > communication transparency (re: what the changes actually affect). Most > SAs and users don't follow commits. Well, no time for me to dig through all that history. arc_max should be a hard limit and it is now. If it ever wasn't then it was a bug. Besides, "high watermark" is still an ambiguous term, for you it "implies" that it is not a hard limit, but for me it "implies" exactly a hard limit. Additionally, going from "non-hard limit" to a "hard limit" on ARC size should improve things memory-wise, not vice versa, right? :) P.S. All that I said above is a hint that this is a pointless branch of the thread :) -- Andriy Gapon