From owner-freebsd-fs@FreeBSD.ORG Tue Feb 24 14:02:56 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FDDD106566C for ; Tue, 24 Feb 2009 14:02:56 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from bene1.itea.ntnu.no (bene1.itea.ntnu.no [IPv6:2001:700:300:3::56]) by mx1.freebsd.org (Postfix) with ESMTP id 78D518FC08 for ; Tue, 24 Feb 2009 14:02:55 +0000 (UTC) (envelope-from ulf.lilleengen@gmail.com) Received: from localhost (localhost [127.0.0.1]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 9C020240F4; Tue, 24 Feb 2009 15:02:53 +0100 (CET) Received: from carrot (unknown [IPv6:2001:700:300:3::184]) by bene1.itea.ntnu.no (Postfix) with ESMTP id 8954124199; Tue, 24 Feb 2009 15:01:22 +0100 (CET) Date: Tue, 24 Feb 2009 15:02:17 +0000 From: Ulf Lilleengen To: ml-ktk@netlabs.org Message-ID: <20090224150217.GA15114@carrot> References: <660f28ee8aa9c3a76b7d736e5ae3c229.squirrel@mail.netlabs.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <660f28ee8aa9c3a76b7d736e5ae3c229.squirrel@mail.netlabs.org> User-Agent: Mutt/1.5.19 (2009-01-05) X-Virus-Scanned: Debian amavisd-new at bene1.itea.ntnu.no Cc: freebsd-fs@freebsd.org Subject: Re: kernel panic while writing to a ZFS volume on iSCSI LUN X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Feb 2009 14:02:56 -0000 On Mon, Feb 23, 2009 at 05:00:06PM +0100, ml-ktk@netlabs.org wrote: > Hi, > > I am testing ZFS, iSCSI & snapshots on FreeBSD as I want to use it later > for some productive data. While doing an rsync from a maildir with lots > of small files, the box crashed. Until then there were about 3.5GB of > data transferred > > Platform: FreeBSD 7.1, i386, install from CD > Box: Xeon 3.2 something > RAM: 1.5GB > 50GB iSCSI LUN on Network Appliance Filer > > I tried to get some hints out of the dump file according to > > http://www.freebsd.org/doc/en/books/developers-handbook/kerneldebug-gdb.html > > but I noticed I had to get source first & compile the kernel on my own. > The backtrace here is done using a kernel.debug which got compiled out > of RELENG_7_1, but it was *not* the same kernel which was running at the > time of the crash (kernel was the default one, not the compiled one). > Does that make a difference for my case or is that ok like this? > > Message in /var/log/messages on reboot: > > - > Feb 20 00:01:23 chewbacca savecore: reboot after panic: > kmem_malloc(131072): kmem_map too small: 228896768 total allocated > Feb 20 00:01:23 chewbacca savecore: writing core to vmcore.0 > - > > backtrace: > http://freebsd.pastebin.com/m36e7ddd0 > > My question is if it is crashing in iSCSI or ZFS code & what I should > try to get rid of it :-) > This is a problem with ZFS due to exhaustion of the kernel memory address space (ZFS is quite hungry for memory). This can be solved by finely tuning the different limits specified here: http://wiki.freebsd.org/ZFSTuningGuide You might have to do some try and fail to get it to work, but my experience is that the problems is usually solveable if you invest enough time in the tuning. -- Ulf Lilleengen