Date: Thu, 21 Jan 2010 17:27:31 -0500 From: Adam McDougall <mcdouga9@egr.msu.edu> To: Artem Belevich <fbsdlist@src.cx> Cc: freebsd-fs@freebsd.org Subject: Re: Repeatable ZFS "kmem map too small" panic on 8.0-STABLE Message-ID: <4B58D4D3.80009@egr.msu.edu> In-Reply-To: <ed91d4a81001211421r4f7ba7a8n1c92bfc413e5feed@mail.gmail.com> References: <4B58976E.1020402@polands.org> <4B58A069.8000802@egr.msu.edu> <4B58BD2D.30803@rcn.com> <ed91d4a81001211421r4f7ba7a8n1c92bfc413e5feed@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 01/21/10 17:21, Artem Belevich wrote: > On Thu, Jan 21, 2010 at 12:46 PM, Gary Corcoran<gcorcoran@rcn.com> wrote: >> Adam McDougall wrote: >>> >>> Put this in /boot/loader.conf: >>> vm.kmem_size="20G" >>> >>> It is intentionally higher than your amount of ram. >> >> Would you mind explaining... >> 1) why this fixes the kmem_map too small problem ? > > Because it explicitly makes kmem_map larger. > >> 2) why it should be larger than the amount of RAM, and by how much ? > > ZFS needs access to a lot of memory for ARC and it allocates/frees > memory fairly randomly. That raises two issues. > > First issue is that kernel is by default fairly conservative about its > memory needs. vm.kmem_size which limits address space for in-kernel > memory allocations is by default set to a fairly low value which works > reasonably well in most of the cases. However, for ZFS it needs to be > bumped up allow large amounts of memory to be allocated by ZFS. > > Second problem is memory fragmentation. If you set vm.kmem_size == > physical memory size, over time you may end up with a situation when > there is plenty of physical memory available, but there is no single > contiguous block of address space to map that memory into. FreeBSD > allocator is pretty good about avoiding fragmentation but you still do > need more address space than the amount of memory that could > potentially be allocated. I'd say that vm.kmem_size should be few > multiples of amounts of memory that you're planning to allocate. > > Just my $0.02 > > --Artem > Exactly what I would have said, thanks :) I'd imagine the kmem_size could be much larger still, closer to kmem_size_max, but I just picked 20G as a default for my servers that have 8G or less and I haven't seen an out of kmem panic for as long as I could raise kmem_size sufficiently high (a change was made around 6 months ago). kmem_size doesn't seem to "grow" (much?) towards kmem_size_max, it is what it is, and you need to make sure it is big enough for your needs. I have systems with just one gig and they run fine.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?4B58D4D3.80009>