From owner-freebsd-fs@FreeBSD.ORG Fri Sep 17 06:12:18 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8E4A01065672 for ; Fri, 17 Sep 2010 06:12:18 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta05.emeryville.ca.mail.comcast.net (qmta05.emeryville.ca.mail.comcast.net [76.96.30.48]) by mx1.freebsd.org (Postfix) with ESMTP id 754FA8FC17 for ; Fri, 17 Sep 2010 06:12:18 +0000 (UTC) Received: from omta06.emeryville.ca.mail.comcast.net ([76.96.30.51]) by qmta05.emeryville.ca.mail.comcast.net with comcast id 7gf11f00216AWCUA5iCH0z; Fri, 17 Sep 2010 06:12:17 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta06.emeryville.ca.mail.comcast.net with comcast id 7iCG1f0093LrwQ28SiCHJv; Fri, 17 Sep 2010 06:12:17 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id AFE539B427; Thu, 16 Sep 2010 23:12:16 -0700 (PDT) Date: Thu, 16 Sep 2010 23:12:16 -0700 From: Jeremy Chadwick To: gil@vidals.net Message-ID: <20100917061216.GA44936@icarus.home.lan> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.20 (2009-06-14) Cc: freebsd-fs@freebsd.org Subject: Re: vm.kmem_size for stability to avoid kernel panics? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 17 Sep 2010 06:12:18 -0000 On Thu, Sep 16, 2010 at 10:39:05PM -0700, Gil Vidals wrote: > The ZFS Tuning guide says that no tuning may be necessary on servers with 2+ > GB of RAM. > > *"FreeBSD 7.2+ has improved kernel memory allocation strategy and no tuning > may be necessary on systems with more than 2 GB of RAM."* > However, is it advisable to put an upper limit for these two parameters to > ensure stability and avoid kernel panics? > > vm.kmem_size > vm.kmem_size_max It depends entirely on what OS version -- and build date -- you're using, in addition to architecture (i386 vs. amd64). Please provide more information about your system. There were changes to the underlying VM to extend vm.kmem_size_max's limit, which got committed sometime during the 7.2-STABLE (I think) cycle. However, vm.kmem_size still needs to be adjusted on both RELENG_7 and RELENG_8. You *do not* (and should not) need to adjust vm.kmem_size_max. Another ZFS-centric tunable you should adjust is vfs.zfs.arc_max -- but again, the functionality of this tunable depends exactly on what OS version and date of build you're using. The functionality in vfs.zfs.arc_max was changed from being a "high watermark" to a hard limit due to people continuing to experience "kmem map too small" panics. I think I may have posted to the list long ago about when this was changed; I don't remember the date off the top of my head. I do know that for RELENG_8, vfs.zfs.arc_max is a hard limit. Each person's hardware, environment, and workload is different, so your tuning will vary. This is the ZFS-related tuning bits we use on our amd64 RELENG_7 and RELENG_8 systems which have 4GB physical RAM installed: # Increase vm.kmem_size to allow for ZFS ARC to utilise more memory. vm.kmem_size="4096M" vfs.zfs.arc_max="3584M" # Disable ZFS prefetching # http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html # Increases overall speed of ZFS, but when disk flushing/writes occur, # system is less responsive (due to extreme disk I/O). # NOTE: 8.0-RC1 disables this by default on systems <= 4GB RAM anyway # NOTE: System has 8GB of RAM, so prefetch would be enabled by default. vfs.zfs.prefetch_disable="1" # Disable UMA (uma(9)) for ZFS; amd64 was moved to exclusively use UMA # on 2010/05/24. # http://lists.freebsd.org/pipermail/freebsd-stable/2010-June/057162.html vfs.zfs.zio.use_uma="0" # Decrease ZFS txg timeout value from 30 (default) to 5 seconds. This # should increase throughput and decrease the "bursty" stalls that # happen during immense I/O with ZFS. # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html vfs.zfs.txg.timeout="5" -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |