From owner-freebsd-fs@FreeBSD.ORG Fri Oct 5 02:04:21 2007 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 24E2A16A421; Fri, 5 Oct 2007 02:04:21 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: from dan.emsphone.com (dan.emsphone.com [199.67.51.101]) by mx1.freebsd.org (Postfix) with ESMTP id E088F13C4C2; Fri, 5 Oct 2007 02:04:13 +0000 (UTC) (envelope-from dan@dan.emsphone.com) Received: (from dan@localhost) by dan.emsphone.com (8.14.1/8.14.1) id l9523kxN066686; Thu, 4 Oct 2007 21:03:46 -0500 (CDT) (envelope-from dan) Date: Thu, 4 Oct 2007 21:03:46 -0500 From: Dan Nelson To: Pawel Jakub Dawidek Message-ID: <20071005020346.GA36686@dan.emsphone.com> References: <20071005000046.GC92272@garage.freebsd.pl> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20071005000046.GC92272@garage.freebsd.pl> X-OS: FreeBSD 7.0-CURRENT User-Agent: Mutt/1.5.16 (2007-06-09) Cc: freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: ZFS kmem_map too small. X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 05 Oct 2007 02:04:21 -0000 In the last episode (Oct 05), Pawel Jakub Dawidek said: > We'are about to branch RELENG_7 and I'd like to start discussion with > folks that experience 'kmem_map too small' panic with the latest > HEAD. > > I'm trying hard to reproduce it and I can't, so I need to gather more > info how you are able to provoke this panic. > > What I did was to rsync 200 FreeBSD src trees from one directory to > another on the same ZFS file system. It worked fine. > > The system I'm using is i386 and the only tuning I did is bigger > kmem_map. From my /boot/loader.conf: > > vm.kmem_size=629145600 > vm.kmem_size_max=629145600 I'm running on a 1gb i386 system with vfs.zfs.arc_min="64M" vfs.zfs.arc_max="192M" vm.kmem_size_min="640M" vm.kmem_size_max="640M" and no panics. Set arc_max any higher (256M for example) and I end up panicing on things like reading a 500M mailbox in mutt, etc. Instead of playing with huge amounts of tiny files like the src tree, try with a small amount of large files like ports/distfiles, or create a couple dozen 500MB files and copy them from place to place. I'd really like to set arc_max up to 512M at least. Sort of feels like a waste having all that RAM and not being able to use it for disk cache. Is there any way to make the arc truly dynamic, being able to shrink and grow as needed just like the old buffer cache? Why does it have to live in kmem? -- Dan Nelson dnelson@allantgroup.com