Date: Sun, 22 Nov 2009 00:20:31 -0500 From: Adam McDougall <mcdouga9@egr.msu.edu> To: Jeremy Chadwick <freebsd@jdc.parodius.com> Cc: freebsd-stable@freebsd.org Subject: Re: 7.2 dies in zfs Message-ID: <20091122052030.GL1213@egr.msu.edu> In-Reply-To: <20091121193643.GA14122@icarus.home.lan> References: <m2skcajavv.wl%randy@psg.com> <m2r5ruja6v.wl%randy@psg.com> <4B066B13.1070006@freebsd.org> <m2bpiwitz7.wl%randy@psg.com> <4b07ac59.A2Afaf4X0IZlrgGU%perryh@pluto.rain.com> <57200BF94E69E54880C9BB1AF714BBCBA5722E@w2003s01.double-l.local> <20091121193643.GA14122@icarus.home.lan>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, Nov 21, 2009 at 11:36:43AM -0800, Jeremy Chadwick wrote: On Sat, Nov 21, 2009 at 08:07:40PM +0100, Johan Hendriks wrote: > Randy Bush <randy@psg.com> wrote: > > imiho, zfs can not be called production ready if it crashes if you > > do not stand on your left leg, put your right hand in the air, and > > burn some eye of newt. > > This is not a rant, but where do you read that on FreeBSD 7.2 ZFS has > been marked as production ready. > As far as i know, on FreeBSD 8.0 ZFS is called production ready. > > If you boot your system it probably tell you it is still experimental. > > Try running FreeBSD 7-Stable to get the latest ZFS version which on > FreeBSD is 13 > On 7.2 it is still at 6 (if I remember it right). RELENG_7 uses ZFS v13, RELENG_8 uses ZFS v18. RELENG_7 and RELENG_8 both, more or less, behave the same way with regards to ZFS. Both panic on kmem exhaustion. No one has answered my question as far as what's needed to stabilise ZFS on either 7.x or 8.x. I have a stable public ftp/http/rsync/cvsupd mirror that runs ZFS v13. It has been stable since mid may. I have not had a kmem panic on any of my ZFS systems for a long time, its a matter of making sure there is enough kmem at boot (not depending on kmem_size_max) and that it is big enough that fragmentation does not cause a premature allocation failure due to lack of large-enough contiguous chunk. This requires the platform to support a kmem size that is "big enough"... i386 can barely muster 1.6G and sometimes that might not be enough. I'm pretty sure all of my currently existing ZFS systems are amd64 where the kmem can now be huge. On the busy fileserver with 20 gigs of ram running FreeBSD 8.0-RC2 #21: Tue Oct 27 21:45:41 EDT 2009, I currently have: vfs.zfs.arc_max=16384M vfs.zfs.arc_min=4096M vm.kmem_size=18G The arc settings here are to try to encourage it to favor the arc cache instead of whatever else Inactive memory in 'top' contains. On other systems that are hit less hard, I simply set: vm.kmem_size="20G" I even do this on systems with much less ram, it doesn't seem to matter except it works, this is on an amd64 with only 8G. Most of my ZFS systems are 7.2-stable, some are 8.0-something. Anything with v13 is much better than v6, but 8.0 has additional fixes that have not been backported to 7 yet. I don't consider the additional fixes in 8 required for my uses yet, although I'm planning on moving forward eventually. I would consider 2G kmem a realistic minimum on a system that will see some serious disk IO (regardless of how much ram the system actually contains, as long as the kmem size can be set that big and the system not blow chunks). Hope this personal experience helps. The people who need to answer the question are those who are familiar with the code. Specifically: Kip Macy, Pawel Jakub Dawidek, and anyone else who knows the internals. Everyone else in the user community is simply guessing + going crazy trying to figure out a solution. As much as I appreciate all the work that has been done to bring ZFS to FreeBSD -- and I do mean that! -- we need answers at this point. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB | _______________________________________________ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091122052030.GL1213>