From owner-freebsd-fs@FreeBSD.ORG Fri Mar 4 10:05:20 2011 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 66EFB1065674 for ; Fri, 4 Mar 2011 10:05:20 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.westchester.pa.mail.comcast.net (qmta09.westchester.pa.mail.comcast.net [76.96.62.96]) by mx1.freebsd.org (Postfix) with ESMTP id 11BFC8FC1E for ; Fri, 4 Mar 2011 10:05:19 +0000 (UTC) Received: from omta09.westchester.pa.mail.comcast.net ([76.96.62.20]) by qmta09.westchester.pa.mail.comcast.net with comcast id Ey5L1g0010SCNGk59y5L7r; Fri, 04 Mar 2011 10:05:20 +0000 Received: from koitsu.dyndns.org ([98.248.33.18]) by omta09.westchester.pa.mail.comcast.net with comcast id Ey5J1g00L0PUQVN3Vy5K0q; Fri, 04 Mar 2011 10:05:19 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 6370E9B422; Fri, 4 Mar 2011 02:05:17 -0800 (PST) Date: Fri, 4 Mar 2011 02:05:17 -0800 From: Jeremy Chadwick To: =?iso-8859-1?Q?Micka=EBl_Can=E9vet?= Message-ID: <20110304100517.GA23249@icarus.home.lan> References: <1299232133.18671.3.camel@pc286.embl.fr> MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <1299232133.18671.3.camel@pc286.embl.fr> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: kmem_map too small with ZFS and 8.2-RELEASE X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Mar 2011 10:05:20 -0000 On Fri, Mar 04, 2011 at 10:48:53AM +0100, Mickaël Canévet wrote: > > I'd use vm.kmem_size="32G" (i.e. twice your RAM) and that's it. > > Should I also increase vfs.zfs.arc_max ? You should adjust vm.kmem_size, but not vm.kmem_size_max. You can adjust vfs.zfs.arc_max to basically ensure system stability. This thread is acting as evidence that there are probably edge cases where the kmem too small panic can still happen despite the limited ARC maximum defaults. For a 16GB system, I'd probably use these settings: vm.kmem_size="16384M" vfs.zfs.arc_max="13312M" I would also use these two settings: # Disable ZFS prefetching # http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html # Increases overall speed of ZFS, but when disk flushing/writes occur, # system is less responsive (due to extreme disk I/O). # NOTE: Systems with 8GB of RAM or more have prefetch enabled by # default. vfs.zfs.prefetch_disable="1" # Decrease ZFS txg timeout value from 30 (default) to 5 seconds. This # should increase throughput and decrease the "bursty" stalls that # happen during immense I/O with ZFS. # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html vfs.zfs.txg.timeout="5" The advice in the Wiki is outdated, especially for 8.2-RELEASE. Best not to follow it as of this writing. > Do you have any idea why the kernel panicked at only 8GB allocated ? I do not. A kernel developer will have to comment on that. Please attempt to reproduce the problem. If you can reproduce it reliably, this will greatly help kernel developers tracking down the source of the problem. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |