Date: Mon, 7 Mar 2011 16:28:20 -0800 From: Jeremy Chadwick <freebsd@jdc.parodius.com> To: Martin Matuska <mm@FreeBSD.org> Cc: freebsd-fs@freebsd.org, Micka??l Can??vet <canevet@embl.fr> Subject: Re: kmem_map too small with ZFS and 8.2-RELEASE Message-ID: <20110308002820.GA26744@icarus.home.lan> In-Reply-To: <4D74AA26.10606@FreeBSD.org> References: <1299232133.18671.3.camel@pc286.embl.fr> <20110304100517.GA23249@icarus.home.lan> <4D74AA26.10606@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 07, 2011 at 10:49:26AM +0100, Martin Matuska wrote: > In 8-STABLE (amd64), starting with SVN revision 214620 > vm.kmem_size_scale defaults to 1. > > This means in 8.2-RELEASE, vm.kmem_size is automatically set to the > amout of your system RAM, > so vm.kmem_size="16G" is automatically set on a system with 16GB RAM. > > D??a 04.03.2011 11:05, Jeremy Chadwick wrote / nap??sal(a): > > On Fri, Mar 04, 2011 at 10:48:53AM +0100, Micka??l Can??vet wrote: > >>> I'd use vm.kmem_size="32G" (i.e. twice your RAM) and that's it. > >> Should I also increase vfs.zfs.arc_max ? > > You should adjust vm.kmem_size, but not vm.kmem_size_max. > > > > You can adjust vfs.zfs.arc_max to basically ensure system stability. > > This thread is acting as evidence that there are probably edge cases > > where the kmem too small panic can still happen despite the limited ARC > > maximum defaults. > > > > For a 16GB system, I'd probably use these settings: > > > > vm.kmem_size="16384M" > > vfs.zfs.arc_max="13312M" > > > > I would also use these two settings: > > > > # Disable ZFS prefetching > > # http://southbrain.com/south/2008/04/the-nightmare-comes-slowly-zfs.html > > # Increases overall speed of ZFS, but when disk flushing/writes occur, > > # system is less responsive (due to extreme disk I/O). > > # NOTE: Systems with 8GB of RAM or more have prefetch enabled by > > # default. > > vfs.zfs.prefetch_disable="1" > > > > # Decrease ZFS txg timeout value from 30 (default) to 5 seconds. This > > # should increase throughput and decrease the "bursty" stalls that > > # happen during immense I/O with ZFS. > > # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007343.html > > # http://lists.freebsd.org/pipermail/freebsd-fs/2009-December/007355.html > > vfs.zfs.txg.timeout="5" > > > > The advice in the Wiki is outdated, especially for 8.2-RELEASE. Best > > not to follow it as of this writing. > > > >> Do you have any idea why the kernel panicked at only 8GB allocated ? > > I do not. A kernel developer will have to comment on that. > > > > Please attempt to reproduce the problem. If you can reproduce it > > reliably, this will greatly help kernel developers tracking down the > > source of the problem. Thanks -- you're absolutely right. I had forgotten all about vm.kmem_size_scale. :-) So yeah, with 8.2-RELEASE onward (rather than get into individual SVN revisions I'm using 8.2-RELEASE as "a point in time"), vm.kmem_size will default to the amount of usable memory (usually slightly less than hw.physmem). Validation: $ sysctl hw.realmem hw.physmem hw.usermem hw.realmem: 9395240960 hw.physmem: 8579981312 hw.usermem: 1086521344 $ sysctl vm.kmem_size vm.kmem_size: 8303894528 As such, one only needs to tune vfs.zfs.arc_max if desired. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP 4BD6C0CB |
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20110308002820.GA26744>