From owner-freebsd-fs@FreeBSD.ORG Tue Nov 9 17:42:46 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id AC0851065670 for ; Tue, 9 Nov 2010 17:42:46 +0000 (UTC) (envelope-from carlson39@llnl.gov) Received: from smtp.llnl.gov (nspiron-3.llnl.gov [128.115.41.83]) by mx1.freebsd.org (Postfix) with ESMTP id 8B1518FC1B for ; Tue, 9 Nov 2010 17:42:46 +0000 (UTC) X-Attachments: None Received: from bagua.llnl.gov (HELO [134.9.197.135]) ([134.9.197.135]) by smtp.llnl.gov with ESMTP; 09 Nov 2010 09:42:46 -0800 Message-ID: <4CD98816.1020306@llnl.gov> Date: Tue, 09 Nov 2010 09:42:46 -0800 From: Mike Carlson User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4CD84258.6090404@llnl.gov> <4CD986DC.1070401@llnl.gov> In-Reply-To: <4CD986DC.1070401@llnl.gov> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: 8.1-RELEASE: ZFS data errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Nov 2010 17:42:46 -0000 On 11/09/2010 09:37 AM, Mike Carlson wrote: > On 11/09/2010 03:23 AM, Ivan Voras wrote: >> On 11/08/10 19:32, Mike Carlson wrote: >> >>> As soon as I create the volume and write data to it, it is reported as >>> being corrupted: >>> >>> write# zpool create filevol001 da2 da3 da4 da5 da6 da7 da8 >>> However, if I create a 'raidz' volume, no errors occur: >> A very interesting problem. Can you check with some other kind of volume >> manager that striping the data doesn't cause some unusual hardware >> interaction? Can you try, as an experiment, striping them all with >> gstripe (but you'll have to use a small stripe size like 16 KiB or 8 KiB)? >> >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://BLOCKEDBLOCKEDlists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> > Sure: > > write# gstripe label -v -s 16384 data /dev/da2 /dev/da3 /dev/da4 > /dev/da5 /dev/da6 /dev/da7 /dev/da8 > Metadata value stored on /dev/da2. > Metadata value stored on /dev/da3. > Metadata value stored on /dev/da4. > Metadata value stored on /dev/da5. > Metadata value stored on /dev/da6. > Metadata value stored on /dev/da7. > Metadata value stored on /dev/da8. > Done. > write# newfs -O2 -U /dev/stripe/data > /dev/stripe/data: 133522760.0MB (273454612256 sectors) block size > 16384, fragment size 2048 > using 627697 cylinder groups of 212.72MB, 13614 blks, 6848 inodes. > with soft updates > super-block backups (for fsck -b #) at: > ... > write# mount /dev/stripe/data /mnt > write# df -h > Filesystem Size Used Avail Capacity Mounted on > /dev/da0s1a 1.7T 22G 1.6T 1% / > devfs 1.0K 1.0K 0B 100% /dev > /dev/stripe/data 126T 4.0K 116T 0% /mnt > write# cd /tmp > write# md5 /tmp/random.dat.1 > MD5 (/tmp/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36 > write# cp random.dat.1 /mnt/ > write# cp /mnt/random.dat.1 /mnt/random.dat.2 > write# cp /mnt/random.dat.1 /mnt/random.dat.3 > write# cp /mnt/random.dat.1 /mnt/random.dat.4 > write# cp /mnt/random.dat.1 /mnt/random.dat.5 > write# cp /mnt/random.dat.1 /mnt/random.dat.6 > write# md5 /mnt/* > MD5 (/mnt/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36 > MD5 (/mnt/random.dat.2) = f795fa09e1b0975c0da0ec6e49544a36 > MD5 (/mnt/random.dat.3) = f795fa09e1b0975c0da0ec6e49544a36 > MD5 (/mnt/random.dat.4) = f795fa09e1b0975c0da0ec6e49544a36 > MD5 (/mnt/random.dat.5) = f795fa09e1b0975c0da0ec6e49544a36 > MD5 (/mnt/random.dat.6) = f795fa09e1b0975c0da0ec6e49544a36 > write# fsck /mnt > fsck: Could not determine filesystem type > write# fsck_ufs /mnt > ** /dev/stripe/data (NO WRITE) > ** Last Mounted on /mnt > ** Phase 1 - Check Blocks and Sizes > Segmentation fault > > So, the data appears to be okay, I wanted to run through a FSCK just to > do it but that seg faulted. Otherwise, that data looks good. > > Question, why did you recommend using a smaller stripe size? Is that to > ensure a sample 1GB test file gets written across ALL disk members? > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://BLOCKEDlists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Oh, I almost forgot, here is the ZFS version of that gstripe array: write# zpool create test01 /dev/stripe/data write# cp /tmp/random.dat.1 /test01/ write# cp /test01/random.dat.1 /test01/random.dat.2 write# cp /test01/random.dat.1 /test01/random.dat.3 write# cp /test01/random.dat.1 /test01/random.dat.4 write# cp /test01/random.dat.1 /test01/random.dat.5 write# cp /test01/random.dat.1 /test01/random.dat.6 write# md5 /test01/* MD5 (/test01/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/test01/random.dat.2) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/test01/random.dat.3) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/test01/random.dat.4) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/test01/random.dat.5) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/test01/random.dat.6) = f795fa09e1b0975c0da0ec6e49544a36 write# zpool scrub write# zpool status pool: test01 state: ONLINE scrub: scrub completed after 0h0m with 0 errors on Tue Nov 9 09:41:34 2010 config: NAME STATE READ WRITE CKSUM test01 ONLINE 0 0 0 stripe/data ONLINE 0 0 0 errors: No known data errors Again, no errors.