From owner-freebsd-fs@FreeBSD.ORG  Wed Feb 22 18:55:56 2012
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34])
	by hub.freebsd.org (Postfix) with ESMTP id CCF68106566C
	for <freebsd-fs@freebsd.org>; Wed, 22 Feb 2012 18:55:56 +0000 (UTC)
	(envelope-from ian@ndwns.net)
Received: from smtpauth.rollernet.us (smtpauth.rollernet.us
	[IPv6:2607:fe70:0:3::d])
	by mx1.freebsd.org (Postfix) with ESMTP id AD8348FC1C
	for <freebsd-fs@freebsd.org>; Wed, 22 Feb 2012 18:55:56 +0000 (UTC)
Received: from smtpauth.rollernet.us (localhost [127.0.0.1])
	by smtpauth.rollernet.us (Postfix) with ESMTP id EB73859446F
	for <freebsd-fs@freebsd.org>; Wed, 22 Feb 2012 10:55:34 -0800 (PST)
Received: from localhost (c-76-126-116-195.hsd1.ca.comcast.net
	[76.126.116.195])
	(using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits))
	(No client certificate requested)
	by smtpauth.rollernet.us (Postfix) with ESMTPSA
	for <freebsd-fs@freebsd.org>; Wed, 22 Feb 2012 10:55:34 -0800 (PST)
Date: Wed, 22 Feb 2012 10:55:52 -0800
From: Ian Downes <ian@ndwns.net>
To: freebsd-fs@freebsd.org
Message-ID: <20120222185552.GA86902@weta.local>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.5.21 (2010-09-15)
X-Rollernet-Abuse: Processed by Roller Network Mail Services. Contact
	abuse@rollernet.us to report violations. Abuse policy:
	http://www.rollernet.us/policy
X-Rollernet-Submit: Submit ID 5cf5.4f453a26.679b9.0
Subject: ZFS: arc_meta consumes *all* ram
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
	<mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Wed, 22 Feb 2012 18:55:56 -0000

Is vfs.zfs.arc_meta_limit supposed to be a (relatively) hard limit on
cached metadata?

I've limited the arc size with arc_max but how do I effectively limit
the caching of meta data?

Suggestions appreciated!

details:
ZFS is exceeding vfs.zfs.arc_meta_limit on some of my boxes; consuming
all available RAM, paging everything out and bringing the system to its
knees.

$ uname -a
FreeBSD local 8.2-RELEASE FreeBSD 8.2-RELEASE #0: Fri Jul  8 00:54:56 UTC 2011     root@8.8.8.8:/usr/obj/usr/src/sys/XENHVM  amd64
$ sysctl vfs.zfs | grep arc_meta
vfs.zfs.arc_meta_limit: 1610612736
vfs.zfs.arc_meta_used: 12183379056

Note that this is 7-8X over arc_meta_limit and was all the available
RAM on the box.

This can be reproduced on several boxes (8.2-RELEASE patched to ZFS
5/28 and 9.0-RELEASE) when periodic/security/100.chksetuid runs and does
a find over all filesystems.