From owner-freebsd-current@FreeBSD.ORG Sun Jan 6 21:20:52 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C286516A421; Sun, 6 Jan 2008 21:20:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 7E80513C448; Sun, 6 Jan 2008 21:20:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id m06L0GB7096862; Sun, 6 Jan 2008 14:00:17 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <47814160.4050401@samsco.org> Date: Sun, 06 Jan 2008 14:00:16 -0700 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.11) Gecko/20071128 SeaMonkey/1.1.7 MIME-Version: 1.0 To: Kris Kennaway References: <20080106141157.I105@fledge.watson.org> <47810DE3.3050106@FreeBSD.org> <478119AB.8050906@FreeBSD.org> In-Reply-To: <478119AB.8050906@FreeBSD.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Sun, 06 Jan 2008 14:00:17 -0700 (MST) X-Spam-Status: No, score=-1.4 required=5.4 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-current@freebsd.org, Ivan Voras Subject: ZFS honesty X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 06 Jan 2008 21:20:52 -0000 Kris Kennaway wrote: > Ivan Voras wrote: >> Kris Kennaway wrote: >>> Ivan Voras wrote: >>>> Robert Watson wrote: >>>> >>>>> I'm not sure if anyone has mentioned this yet in the thread, but >>>>> another thing worth taking into account in considering the >>>>> stability of ZFS is whether or not Sun considers it a production >>>>> feature in Solaris. Last I heard, it was still considered an >>>>> experimental feature there as well. >>>> >>>> Last I heard, rsync didn't crash Solaris on ZFS :) >>> >>> [Citation needed] >> >> I can't provide citation about a thing that doesn't happen - you don't >> hear things like "oh and yesterday I ran rsync on my Solaris with ZFS >> and *it didn't crash*!" often. >> >> But, with some grains of salt taken, consider this Google results: >> >> * searching for "rsync crash solaris zfs": 790 results, most of them >> obviously irrelevant >> * searching for "rsync crash freebsd zfs": 10,800 results; a small >> number of the results is from this thread, some are duplicates, but >> it's a large number in any case. >> >> I feel that the number of Solaris+ZFS installations worldwide is >> larger than that of FreeBSD+ZFS and they've had ZFS longer. > > Almost all Solaris systems are 64 bit. > > Kris So, let's be honest here. ZFS is simply unreliable on FreeBSD/i386. There are things that you can do mitigate the problems, and in certain well controlled environments you might be able to make it work well enough for your needs. But as a general rule, don't expect it to work reliably, period. This is backed up by Sun's own recommendation to not run it on 32-bit Solaris. But let's also be honest about ZFS in the 64-bit world. There is ample evidence that ZFS basically wants to grow unbounded in proportion to the workload that you give it. Indeed, even Sun recommends basically throwing more RAM at most problems. Again, tuning is often needed, and I think it's fair to say that it can't be expected to work on arbitrary workloads out of the box. Now, what about the other problems that have been reported in this thread by Ivan and others? I don't think that it can be said that the only problem that ZFS has is with memory. Unfortunately, it looks like these "other" problems aren't well quantified, so I think that they are being unfairly dismissed. But at the same time, maybe these other problems are rare and unique enough that they represent very special cases that won't be encountered by most people. But it also tells me that ZFS is still immature, at least in FreeBSD. The universal need for tuning combined with the poorly understood problem reports tells me that administrators considering ZFS should expect to spend a fair amount of timing testing and tuning. Don't expect it to work out of the box for your situation. That's not to say that it's useless; there are certainly many people who can attest to it working well for them. Just be prepared to spend time and possibly money making it work, and be willing to provide good problem reports for any non-memory related problems that you encounter. Scott