Date: Fri, 28 May 2010 10:51:59 -0400 From: "Howard Leadmon" <howard@leadmon.net> To: "'Jeremy Chadwick'" <freebsd@jdc.parodius.com>, "'Andriy Gapon'" <avg@icyb.net.ua> Cc: amd64@freebsd.org, freebsd-fs@freebsd.org Subject: RE: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. Message-ID: <064701cafe75$537c1530$fa743f90$@net> In-Reply-To: <20100528134549.GA75411@icarus.home.lan> References: <060401cafe37$a411b240$ec3516c0$@net><4BFF894F.4010008@icyb.net. ua> <20100528134549.GA75411@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
> -----Original Message----- > From: Jeremy Chadwick [mailto:freebsd@jdc.parodius.com] > Sent: Friday, May 28, 2010 9:46 AM > To: Andriy Gapon > Cc: Howard Leadmon; amd64@freebsd.org; freebsd-fs@freebsd.org > Subject: Re: FreeBSD 8.1-Prerelease Panic amd64 w/ZFS.. > > On Fri, May 28, 2010 at 12:13:51PM +0300, Andriy Gapon wrote: > > on 28/05/2010 10:30 Howard Leadmon said the following: > > > I know there used to be some issues with this a while back, but > thought > > > with the 8.x FBSD servers most of this tuned itself, or then again > maybe > > > this is something different. > > > > > > > > > > > > For the first time I ever recall, I found my FreeBSD 8 server was > crashed > > > this past morning, with the following error: > > > > > > > > > > > > panic:kmem_malloc(131072):kmem_map to small: 1296826368 total > allocated > > > > > > cupid=4 > > > > O wow, this is an amd64 system (with 4G RAM) and you've got "kmem_map > too > > small". That's very very strange. > > It is? On amd64, vm.kmem_size (not vm.kmem_size_max) is what has to be > increased. I can point folks to the "official" statement from pjd@ and > some others if need be. For a very long time I questioned this because > for an even longer amount of time we were being told to increase > vm.kmem_size_max. > > vm.kmem_size_max, by default, is already huge on amd64 (~320GB or > something like that). Proof: > > vm.kmem_size_max: 329853485875 > > To the OP: you will need to increase vm.kmem_size in /boot/loader.conf > and reboot the system. "What value do I pick?" With 4GB, I would > recommend you use these two variables: > > vm.kmem_size="2048M" > vfs.zfs.arc_max="1536M" > > This will increase the available kmem, and also limit the ARC size > explicitly to nothing larger than 1.5GB. This should stabilise your > system. > > -- > | Jeremy Chadwick jdc@parodius.com | > | Parodius Networking http://www.parodius.com/ | > | UNIX Systems Administrator Mountain View, CA, USA | > | Making life hard for others since 1977. PGP: 4BD6C0CB | Thanks Jeremy, I will try your recommended settings provided above. To the other poster, as to the settings of kmem, I had nothing specific set, just whatever FBSD was using by default. In loader.conf all I had was: zfs_load="YES" vfs.root.mountfrom="zfs:tank/root" As to the setting of kmem and arc, I had the following which I will assume were defaults or auto-tunes: vfs.zfs.arc_max : 862653440 vm.kmem_size : 1380245504 vm.kmem_size_max: 329853485875 Like Jeremy said, it sure looks like vm.kmem_size_max is quite large by default. After applying Jeremy's recommendations, I now see: vfs.zfs.arc_max : 1610612736 vm.kmem_size : 2147483648 vm.kmem_size_max: 329853485875 I guess while we are all on the subject, I notice in the dmesg log the message: ZFS NOTICE: Prefetch is disabled by default if less than 4GB of RAM is present; to enable, add "vfs.zfs.prefetch_disable=0" to /boot/loader.conf. Is this anything I want to enable, that's like a big performance win, or do I just not have enough RAM to support it? Always been kinda curious about it, but so far I am liking ZFS, well outside of the machine panic.. LOL --- Howard Leadmon
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?064701cafe75$537c1530$fa743f90$>