Date: Sat, 5 Mar 2011 09:42:47 -0500 From: Andriy Bakay <andriy@irbisnet.com> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: "freebsd-fs@freebsd.org" <freebsd-fs@freebsd.org>, "pjd@FreeBSD.org" <pjd@FreeBSD.org> Subject: Re: kmem_map too small with ZFS and 8.2-RELEASE Message-ID: <625F7D8A-78F5-426B-B2C3-A1B4D289FD5F@irbisnet.com> In-Reply-To: <20110304143801.GA1734@icarus.home.lan> References: <1299232133.18671.3.camel@pc286.embl.fr> <4D70E7D1.5030205@egr.msu.edu> <20110304143801.GA1734@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On 2011-03-04, at 9:38, Jeremy Chadwick <freebsd@jdc.parodius.com> wrote: > On Fri, Mar 04, 2011 at 08:23:29AM -0500, Adam McDougall wrote: >> On 03/04/11 04:48, Mickaël Canévet wrote: >>>> I'd use vm.kmem_size="32G" (i.e. twice your RAM) and that's it. >>> >>> Should I also increase vfs.zfs.arc_max ? >>> >>> Do you have any idea why the kernel panicked at only 8GB allocated ? >>> >>> Thank you >> >> I believe ARC allocations in kmem can become fragmented, so when it >> is searching for a place to store a new contiguous segment of >> memory, the remaining fragmented free spaces may all be too small. >> I also set >> vm.kmem_size to about twice the amount of ram to help it avoid this >> issue. I suspect if kmem is badly fragmented then performance of >> ZFS can downgrade, so that is another reason to keep kmem bigger. > > My findings on 8.2-RELEASE indicate that doing this results in very > unexpected behaviour regarding the ARC maximum. As such, I cannot > recommend this model. > > For example, on an amd64 system with 8GB physical RAM and these two > settings in /boot/loader.conf: > > vm.kmem_size="8192M" > vfs.zfs.arc_max="6144M" > > kstat.zfs.misc.arcstats.size tops out at around 6240986896, with Wired > in top(1) showing ~6.3GB. This is expected behaviour and fits (I > think) what people expect. > > However, on the exact same system with these two settings: > > vm.kmem_size="16384M" > vfs.zfs.arc_max="6144M" > > The above ARC numbers are exactly *half* that amount. This is easily > reproducible. > > Can someone 1) justify the "2x the amount of RAM for vm.kmem_size" > setting, and 2) explain in detail the above behaviour? > > -- > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP 4BD6C0CB | > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" I guess Pawel can: vm.kmem_size=”6G” # This should be 150% of your RAM. http://blogs.freebsdish.org/pjd/2010/08/06/from-sysinstall-to-zfs-only-configuration/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?625F7D8A-78F5-426B-B2C3-A1B4D289FD5F>