Date: Thu, 25 Sep 2008 16:14:02 +0200 From: Bartosz Stec <admin@kkip.pl> To: freebsd-stable@freebsd.org Subject: Re: vm.kmem_size settings doesn't affect loader? Message-ID: <48DB9CAA.9060807@kkip.pl> In-Reply-To: <20080925130227.GA13497@icarus.home.lan> References: <48DB6772.1060400@kkip.pl> <20080925130227.GA13497@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
Jeremy Chadwick pisze:
> On Thu, Sep 25, 2008 at 12:26:58PM +0200, Bartosz Stec wrote:
>
>> Today I've experienced zfs-related kernel panic. Log says:
>>
>> savecore: reboot after panic: kmem_malloc(131072): kmem_map too
>> small: 327684096 total allocated
>>
>> Reported amount of memory (327684096) is wrong, because i made suggested
>> tuning in my loader.conf:
>>
>> vm.kmem_size="512M"
>> vm.kmem_size_max="512M"
>>
>> Just to be sure:
>>
>> # sysctl vm | grep kmem
>> vm.kmem_size: 536870912
>> vm.kmem_size_min: 0
>> vm.kmem_size_max: 536870912
>> vm.kmem_size_scale: 3
>>
>> Am I missing something?
>>
>
> I believe this is normal. The amount shown in "total allocated" will
> not match what you have vm.kmem_size or vm.kmem_size_max set to.
> Someone more familiar with the VM can explain why this is, but as I
> understand it, it's 100% normal.
>
Thanks for explaining. This part of tuning guide confused me:
"I was able to generate the following kernel panic in less than a
minute by copying files from a linux server connected via gigabit
crossover:
Apr 8 06:46:08 nas savecore: reboot after panic: kmem_malloc(131072): kmem_map too small: 528273408 total allocated
As you can see, this is *with* vm.kmem_size="512M"!"
I've just expected similiar numbers in my case.
> Your options are:
>
> 1) Consider increasing it from 512M to something like 1.5GB; do not
> increase it past that on RELENG_7, as there isn't support for more than
> 2GB total. For example, on a 1GB memory machine, I often recommend
> 768M. On 2GB machines, 1536M. You will need to run -CURRENT if you
> want more.
>
> 2) Tune ZFS aggressively. Start by setting vfs.zfs.arc_min="16M"
> and vfs.zfs.arc_max="64M".
>
> If your machine has some small amount of memory (768MB, 1GB, etc.),
> then you probably shouldn't be using ZFS.
>
>
Problem occured on i386 machine with 1GB of memory and 7.1-pre (3HDD,
40GB, RAIDZ1). I know that i386 is highly unrecommended for ZFS, but
it's just a home box for testing and learning purposes - I just want to
know what I'm doing and what should I expect when I decide to put ZFS on
server machines :) Currently, from posts on freebsd-fs, I conclude that
even with a gigs of kmem and using AMD64, we still can experience panic
from kmem_malloc. Manual tuning is hard for me because I'm not familiar
with BSD kernel code nor kernel memory management. I'm just an end-user
who love concepts of ZFS and wait for it to be (more) stable. Of course
I've followed tuning guide carefully.
So, for now, I will add another 512MB of memory and increse vm.kmem_size
and do more tests. Thanks once again for explaining and suggestions.
Good luck with ZFS everyone! :)
--
Bartosz Stec
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?48DB9CAA.9060807>