From owner-freebsd-fs@FreeBSD.ORG Wed Apr 22 14:40:33 2009 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9ED9910657F9 for ; Wed, 22 Apr 2009 14:40:28 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from mx.egr.msu.edu (surfnturf.egr.msu.edu [35.9.37.164]) by mx1.freebsd.org (Postfix) with ESMTP id 8EE818FC15 for ; Wed, 22 Apr 2009 14:40:21 +0000 (UTC) (envelope-from mcdouga9@egr.msu.edu) Received: from localhost (localhost [127.0.0.1]) by mx.egr.msu.edu (Postfix) with ESMTP id 9AA3C71F08C for ; Wed, 22 Apr 2009 10:20:43 -0400 (EDT) X-Virus-Scanned: amavisd-new at egr.msu.edu Received: from mx.egr.msu.edu ([127.0.0.1]) by localhost (surfnturf.egr.msu.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id xml+4qgDebpE for ; Wed, 22 Apr 2009 10:20:43 -0400 (EDT) Received: from [35.9.44.65] (daemon.egr.msu.edu [35.9.44.65]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: mcdouga9) by mx.egr.msu.edu (Postfix) with ESMTPSA id 77B2571F06E for ; Wed, 22 Apr 2009 10:20:43 -0400 (EDT) Message-ID: <49EF27BB.1060100@egr.msu.edu> Date: Wed, 22 Apr 2009 10:20:43 -0400 From: Adam McDougall User-Agent: Thunderbird 2.0.0.21 (X11/20090419) MIME-Version: 1.0 CC: freebsd-fs@freebsd.org References: <49EE49D8.7000902@free.de> <20090422123020.42b756c1@ernst.jennejohn.org> In-Reply-To: <20090422123020.42b756c1@ernst.jennejohn.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: Re: FreeBSD 7.2-RC1 - ZFS related kernel panic "kmem_map too small" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Apr 2009 14:40:54 -0000 Gary Jennejohn wrote: > On Wed, 22 Apr 2009 00:34:00 +0200 > Kai Gallasch wrote: > > [snip a lot of stuff] > >> In /usr/src/UPDATING I read: >> >> [..] >> >> 20090207: >> ZFS users on amd64 machines with 4GB or more of RAM should >> reevaluate their need for setting vm.kmem_size_max and >> vm.kmem_size manually. In fact, after recent changes to the >> kernel, the default value of vm.kmem_size is larger than the >> suggested manual setting in most ZFS/FreeBSD tuning guides. >> >> So I understood this as "vm.kmem_size is set unnecessary large by >> default. You should think about decreasing it to save some RAM" >> >> On my amd64 server the default values of kmem_size are >> >> vm.kmem_size_scale: 3 >> vm.kmem_size_max: 3865468109 >> vm.kmem_size_min: 0 >> vm.kmem_size: 1201446912 >> >> Can someone give me a hint how to debug this problem further, or how to >> find some reasonable values for setting vm.kmem_size_max and >> vm.kmem_size with 16G of RAM? >> >> > > Hmm, I wonder whether this applies to 7.2-RC1. I don't know whether > the kernel changes have been committed to 7.2 or whether they were > already present when we started work on 7.2 because I haven't been > paying much attention. > > On my 8-current amd64 machine with only 4GB of RAM I see larger values > than you see with 16GB: > > sysctl vm.kmem_size_max > vm.kmem_size_max: 4509713203 > sysctl vm.kmem_size > vm.kmem_size: 1335824384 > > --- > Gary Jennejohn > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > It has been my experience that after the kmem maximums were raised to allow more than approx 1.6G kmem (a number of months ago), on some systems I still had to specifically raise the vm.kmem_size above the default otherwise I still got out of kmem panics far below the max. I suspect there was pressure for kmem and it was unable to "raise" the limit fast enough, or maybe a fragmentation problem? Additionally, depending on which host, I've found different limits to how high I can set the kmem settings on "recent" builds of 7 and 8 amd64, for example I have one 7.2 system with 4G ram and the kernel would panic if I booted with kmem=2G (1G works fine), but I have a 8.0 system with 2G ram and kmem=2G works fine. Another 8.0 system has 6G ram but we could only boot successfully with kmem=3G, not 4G.