From owner-freebsd-fs@FreeBSD.ORG Wed Nov 10 11:03:18 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28B89106564A for ; Wed, 10 Nov 2010 11:03:18 +0000 (UTC) (envelope-from freebsd-fs@m.gmane.org) Received: from lo.gmane.org (lo.gmane.org [80.91.229.12]) by mx1.freebsd.org (Postfix) with ESMTP id A86C58FC15 for ; Wed, 10 Nov 2010 11:03:17 +0000 (UTC) Received: from list by lo.gmane.org with local (Exim 4.69) (envelope-from ) id 1PG8SN-0005zW-Ah for freebsd-fs@freebsd.org; Wed, 10 Nov 2010 12:03:11 +0100 Received: from lara.cc.fer.hr ([161.53.72.113]) by main.gmane.org with esmtp (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 10 Nov 2010 12:03:11 +0100 Received: from ivoras by lara.cc.fer.hr with local (Gmexim 0.1 (Debian)) id 1AlnuQ-0007hv-00 for ; Wed, 10 Nov 2010 12:03:11 +0100 X-Injected-Via-Gmane: http://gmane.org/ To: freebsd-fs@freebsd.org From: Ivan Voras Date: Wed, 10 Nov 2010 12:03:00 +0100 Lines: 58 Message-ID: References: <4CD84258.6090404@llnl.gov> <4CD986DC.1070401@llnl.gov> <4CD98816.1020306@llnl.gov> Mime-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Complaints-To: usenet@dough.gmane.org X-Gmane-NNTP-Posting-Host: lara.cc.fer.hr User-Agent: Mozilla/5.0 (X11; U; FreeBSD amd64; en-US; rv:1.9.2.12) Gecko/20101102 Thunderbird/3.1.6 In-Reply-To: <4CD98816.1020306@llnl.gov> X-Enigmail-Version: 1.1.2 Subject: Re: 8.1-RELEASE: ZFS data errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Nov 2010 11:03:18 -0000 On 11/09/10 18:42, Mike Carlson wrote: >> write# gstripe label -v -s 16384 data /dev/da2 /dev/da3 /dev/da4 >> /dev/da5 /dev/da6 /dev/da7 /dev/da8 >> write# df -h >> Filesystem Size Used Avail Capacity Mounted on >> /dev/da0s1a 1.7T 22G 1.6T 1% / >> devfs 1.0K 1.0K 0B 100% /dev >> /dev/stripe/data 126T 4.0K 116T 0% /mnt >> write# fsck /mnt >> fsck: Could not determine filesystem type >> write# fsck_ufs /mnt >> ** /dev/stripe/data (NO WRITE) >> ** Last Mounted on /mnt >> ** Phase 1 - Check Blocks and Sizes >> Segmentation fault >> So, the data appears to be okay, I wanted to run through a FSCK just to >> do it but that seg faulted. Otherwise, that data looks good. Hmm, probably it tried to allocate a gazillion internal structures to check it and didn't take no for an answer. >> Question, why did you recommend using a smaller stripe size? Is that to >> ensure a sample 1GB test file gets written across ALL disk members? Yes, it's the surest way since MAXPHYS=128 KiB / 8 = 16 KiB. Well, as far as I'm concerned this probably shows that there isn't something wrong about hardware or GEOM, though more testing, like running a couple of bonnie++ rounds on the UFS on the stripe volume for a few hours, would probably be better. Btw. what bandwidth do you get from this combination (gstripe + UFS)? > Oh, I almost forgot, here is the ZFS version of that gstripe array: > > write# zpool create test01 /dev/stripe/data > write# zpool scrub > write# zpool status > pool: test01 > state: ONLINE > scrub: scrub completed after 0h0m with 0 errors on Tue Nov 9 > 09:41:34 2010 > config: > > NAME STATE READ WRITE CKSUM > test01 ONLINE 0 0 0 > stripe/data ONLINE 0 0 0 "scrub" verifies only written data, not the whole file system space (that's why it finishes so fast), so it isn't really doing any load on the array, but I agree that it looks more and more like there really is an issue in ZFS.