From owner-freebsd-current@FreeBSD.ORG Mon Jan 7 01:01:59 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id F08EC16A418 for ; Mon, 7 Jan 2008 01:01:59 +0000 (UTC) (envelope-from bofh@terranova.net) Received: from tog.net (tog.net [216.89.226.5]) by mx1.freebsd.org (Postfix) with ESMTP id D0AF013C448 for ; Mon, 7 Jan 2008 01:01:59 +0000 (UTC) (envelope-from bofh@terranova.net) Received: from [216.89.228.170] (host-216-89-228-170.wireless.terranova.net [216.89.228.170]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tog.net (Postfix) with ESMTP id 2C49729B5FB for ; Sun, 6 Jan 2008 20:01:59 -0500 (EST) Message-ID: <47817A06.9070906@terranova.net> Date: Sun, 06 Jan 2008 20:01:58 -0500 From: Travis Mikalson Organization: TerraNovaNet Internet Services User-Agent: Thunderbird 1.5.0.14 (Windows/20071210) MIME-Version: 1.0 To: freebsd-current@freebsd.org References: <200801062130.40755.peter.schuller@infidyne.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: ZFS on AMD64 - any recent crashes? X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Jan 2008 01:02:00 -0000 Ivan Voras wrote: > Peter Schuller wrote: >>> In light of the recent discussion about ZFS stability, does anyone still >>> have kmem_map_too_small panics on AMD64 after tuning it (as presented in >>> http://wiki.freebsd.org/ZFSTuningGuide)? >> >> Not sure if you wanted negatives, but I have not seen such crashes on >> any of > > All reports are welcome :) > >> the three amd64 systems I'm running ZFS on (3 gb, 4 gb and 4 gb of >> RAM). The only tuning done is to increase the kmem size and arc_max >> and disabling prefetch. The increase in kmem was not to avoid crashes, >> but was to accomodate the larger arc_max chosen. > > This is somewhat a special case - you also did tuning besides kmem size. > I'm especially interested in this, hoping to gather the "combination > that works" from the reports. I can tell you what works for us. Slightly verbose and repeating things most of us already know, for the archives. Do not put your swap on ZFS. On an amd64 system with 2GB of RAM, we put the following in /boot/loader.conf: vfs.zfs.arc_max="600M" vm.kmem_size_max="1G" vm.kmem_size="1G" We seem to pretty easily panic our RELENG_7 ZFS systems if we do not set arc_max to roughly 65% or less of kmem_size_max. I do 60% to be more conservative. YMMV, but you get the idea. To panic our systems quickly when the tuning settings are bad (in other words, to trigger a kmem_map_too_small panic) we can use iozone. It usually only takes a minute or two. -- TerraNovaNet Internet Services - Key Largo, FL Voice: (305)453-4011 x101 Fax: (305)451-5991 http://www.terranova.net/ ---------------------------------------------- Life's not fair, but the root password helps.