From owner-freebsd-stable@FreeBSD.ORG Sun Nov 22 09:00:08 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 20C931065672 for ; Sun, 22 Nov 2009 09:00:01 +0000 (UTC) (envelope-from svein-listmail@stillbilde.net) Received: from mail.stillbilde.net (d80.iso100.no [81.175.61.195]) by mx1.freebsd.org (Postfix) with ESMTP id 9F4BC8FC0A for ; Sun, 22 Nov 2009 09:00:00 +0000 (UTC) Received: from [192.168.4.14] (unknown [192.168.4.14]) (Authenticated sender: svein) by mail.stillbilde.net (Familien Skogens mail) with ESMTPSA id 7D5D526; Sun, 22 Nov 2009 09:59:59 +0100 (CET) Message-ID: <4B08FD93.4070409@stillbilde.net> Date: Sun, 22 Nov 2009 10:00:03 +0100 From: "Svein Skogen (listmail account)" User-Agent: Thunderbird 2.0.0.23 (Windows/20090812) MIME-Version: 1.0 To: Adam McDougall References: <4B066B13.1070006@freebsd.org> <4b07ac59.A2Afaf4X0IZlrgGU%perryh@pluto.rain.com> <57200BF94E69E54880C9BB1AF714BBCBA5722E@w2003s01.double-l.local> <20091121193643.GA14122@icarus.home.lan> <20091122052030.GL1213@egr.msu.edu> In-Reply-To: <20091122052030.GL1213@egr.msu.edu> X-Enigmail-Version: 0.96.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 8bit Cc: freebsd-stable@freebsd.org, Jeremy Chadwick Subject: Re: 7.2 dies in zfs X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 22 Nov 2009 09:00:08 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Adam McDougall wrote: > On Sat, Nov 21, 2009 at 11:36:43AM -0800, Jeremy Chadwick wrote: > > > On Sat, Nov 21, 2009 at 08:07:40PM +0100, Johan Hendriks wrote: > > Randy Bush wrote: > > > imiho, zfs can not be called production ready if it crashes if you > > > do not stand on your left leg, put your right hand in the air, and > > > burn some eye of newt. > > > > This is not a rant, but where do you read that on FreeBSD 7.2 ZFS has > > been marked as production ready. > > As far as i know, on FreeBSD 8.0 ZFS is called production ready. > > > > If you boot your system it probably tell you it is still experimental. > > > > Try running FreeBSD 7-Stable to get the latest ZFS version which on > > FreeBSD is 13 > > On 7.2 it is still at 6 (if I remember it right). > > RELENG_7 uses ZFS v13, RELENG_8 uses ZFS v18. > > RELENG_7 and RELENG_8 both, more or less, behave the same way with > regards to ZFS. Both panic on kmem exhaustion. No one has answered my > question as far as what's needed to stabilise ZFS on either 7.x or 8.x. > > I have a stable public ftp/http/rsync/cvsupd mirror that runs ZFS v13. > It has been stable since mid may. I have not had a kmem panic on any > of my ZFS systems for a long time, its a matter of making sure there is > enough kmem at boot (not depending on kmem_size_max) and that it is big enough > that fragmentation does not cause a premature allocation failure due to lack > of large-enough contiguous chunk. This requires the platform to support a > kmem size that is "big enough"... i386 can barely muster 1.6G and sometimes > that might not be enough. I'm pretty sure all of my currently existing ZFS > systems are amd64 where the kmem can now be huge. On the busy fileserver with > 20 gigs of ram running FreeBSD 8.0-RC2 #21: Tue Oct 27 21:45:41 EDT 2009, > I currently have: > vfs.zfs.arc_max=16384M > vfs.zfs.arc_min=4096M > vm.kmem_size=18G > The arc settings here are to try to encourage it to favor the arc cache > instead of whatever else Inactive memory in 'top' contains. Very interesting. For my iscsi backend (running istgt from ports), I had to change the arc_max below 128M to stop iSCSI initiators generating timeouts when the cache flushed. (This is on a system with a megaraid 8308ELP handling the disk back end, with the disks in two RAID5 arrays of four disks each, zpooled as one big pool). When I had more than 128M arc_max, zfs on regular times ate all available resources to flush to disk, leaving the istgt waiting, and iSCSI initiators timed out and had to reconnect. The iSCSI initiators are the built-in software initator in VMWare ESX 4i. //Svein - -- - --------+-------------------+------------------------------- /"\ |Svein Skogen | svein@d80.iso100.no \ / |Solberg Østli 9 | PGP Key: 0xE5E76831 X |2020 Skedsmokorset | svein@jernhuset.no / \ |Norway | PGP Key: 0xCE96CE13 | | svein@stillbilde.net ascii | | PGP Key: 0x58CD33B6 ribbon |System Admin | svein-listmail@stillbilde.net Campaign|stillbilde.net | PGP Key: 0x22D494A4 +-------------------+------------------------------- |msn messenger: | Mobile Phone: +47 907 03 575 |svein@jernhuset.no | RIPE handle: SS16503-RIPE - --------+-------------------+------------------------------- If you really are in a hurry, mail me at svein-mobile@stillbilde.net This mailbox goes directly to my cellphone and is checked even when I'm not in front of my computer. - ------------------------------------------------------------ Picture Gallery: https://gallery.stillbilde.net/v/svein/ - ------------------------------------------------------------ -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.10 (MingW32) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAksI/ZMACgkQODUnwSLUlKRr6gCfeq5dybIfp5RLOzjL04guLV25 +qgAn04SjnGG3lBRExQaMjxyKcd9Jcct =ubYi -----END PGP SIGNATURE-----