From owner-freebsd-fs@FreeBSD.ORG Sat Apr 7 18:03:30 2007 Return-Path: X-Original-To: freebsd-fs@FreeBSD.org Delivered-To: freebsd-fs@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id E210416A400; Sat, 7 Apr 2007 18:03:30 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from raven.bwct.de (raven.bwct.de [85.159.14.73]) by mx1.freebsd.org (Postfix) with ESMTP id 711D113C483; Sat, 7 Apr 2007 18:03:30 +0000 (UTC) (envelope-from ticso@cicely12.cicely.de) Received: from cicely5.cicely.de ([10.1.1.7]) by raven.bwct.de (8.13.4/8.13.4) with ESMTP id l37I3T2x025775; Sat, 7 Apr 2007 20:03:29 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (cicely12.cicely.de [10.1.1.14]) by cicely5.cicely.de (8.13.4/8.13.4) with ESMTP id l37I3K96091603 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Sat, 7 Apr 2007 20:03:20 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: from cicely12.cicely.de (localhost [127.0.0.1]) by cicely12.cicely.de (8.13.4/8.13.3) with ESMTP id l37I3JlT017380; Sat, 7 Apr 2007 20:03:19 +0200 (CEST) (envelope-from ticso@cicely12.cicely.de) Received: (from ticso@localhost) by cicely12.cicely.de (8.13.4/8.13.3/Submit) id l37I3J7r017379; Sat, 7 Apr 2007 20:03:19 +0200 (CEST) (envelope-from ticso) Date: Sat, 7 Apr 2007 20:03:19 +0200 From: Bernd Walter To: "Florian C. Smeets" Message-ID: <20070407180319.GH8831@cicely12.cicely.de> References: <20070406025700.GB98545@garage.freebsd.pl> <20070407025644.GC8831@cicely12.cicely.de> <20070407131353.GE63916@garage.freebsd.pl> <4617A3A6.60804@kasimir.com> <20070407165759.GG8831@cicely12.cicely.de> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20070407165759.GG8831@cicely12.cicely.de> X-Operating-System: FreeBSD cicely12.cicely.de 5.4-STABLE alpha User-Agent: Mutt/1.5.9i X-Spam-Status: No, score=-4.4 required=5.0 tests=ALL_TRUSTED=-1.8, BAYES_00=-2.599 autolearn=ham version=3.1.7 X-Spam-Checker-Version: SpamAssassin 3.1.7 (2006-10-05) on cicely12.cicely.de Cc: freebsd-fs@FreeBSD.org, freebsd-current@FreeBSD.org, Pawel Jakub Dawidek , ticso@cicely.de Subject: Re: ZFS committed to the FreeBSD base. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ticso@cicely.de List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Apr 2007 18:03:31 -0000 On Sat, Apr 07, 2007 at 06:58:00PM +0200, Bernd Walter wrote: > On Sat, Apr 07, 2007 at 03:59:02PM +0200, Florian C. Smeets wrote: > > -----BEGIN PGP SIGNED MESSAGE----- > > Hash: SHA1 > > > > Pawel Jakub Dawidek wrote: > > > On Sat, Apr 07, 2007 at 04:56:45AM +0200, Bernd Walter wrote: > > >> My sandbox just has 128MB RAM so kmem was set to ~40M. > > >> After raising kmem to 80M it survived copying the file, but paniced > > >> again while tar -xvzf the file into the same pool. > > >> vfs.zfs.vdev.cache.size is unchanged at 10M. > > > > > > 128MB RAM of suggested minimum in ZFS requirements, but it may be not > > > enough... Minimum of ARC is set to 1/8 of all memory or 64MB (whichever > > > is more). Could you locate these lines in > > > sys/contrib/opensolaris/uts/common/fs/zfs/arc.c file: > > > > > > /* set min cache to 1/32 of all memory, or 64MB, whichever is more */ > > > arc_c_min = MAX(arc_c / 4, 64<<20); > > > > > > Change 64 to eg. 32, recompile and retest? > > > > > > > Hi Pawel, > > > > i had the same problems like Bernd while trying to copy the src tree to > > a ZFS volume. I have 384MB RAM but i got the same "kmem_map: too small" > > panic. I compiled my kernel like you proposed and now i am able to copy > > anything to the volume without panic :-) > > I had increased RAM to 384 and still had a panic with default kmem > (IIRC around 100M) and even increasing kmem to 160M did help a long > time, but still produced the panic after a while. > I don't think 64M applies here as the real limit. Now with 240M kmem it looks good, but I'm still unshure: kstat.zfs.misc.arcstats.c_min: 67108864 kstat.zfs.misc.arcstats.c_max: 188743680 kstat.zfs.misc.arcstats.size: 87653376 c_max seemed to be increasing with kmem, but I did compare it with a remebered value. Should be good with: vm.kmem_size: 251658240 But top shows wired memory which is roughly twice the size of arcstats.size, so I'm still worried about kmem exhaustion if ARC runs up to c_max. Since the c_min/c_max values also influence the available RAM for other purposes as well, can we have it at least a loader.conf tuneable? Otherwise - the reboot after the panics where impressive. No long fsck times or noticed data corruption - even with NFS clients. All in all it is a great job. -- B.Walter http://www.bwct.de http://www.fizon.de bernd@bwct.de info@bwct.de support@fizon.de