From owner-freebsd-fs@FreeBSD.ORG Tue Nov 9 17:37:32 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8C40D106564A for ; Tue, 9 Nov 2010 17:37:32 +0000 (UTC) (envelope-from carlson39@llnl.gov) Received: from smtp.llnl.gov (nspiron-3.llnl.gov [128.115.41.83]) by mx1.freebsd.org (Postfix) with ESMTP id 765E78FC19 for ; Tue, 9 Nov 2010 17:37:32 +0000 (UTC) X-Attachments: None Received: from bagua.llnl.gov (HELO [134.9.197.135]) ([134.9.197.135]) by smtp.llnl.gov with ESMTP; 09 Nov 2010 09:37:31 -0800 Message-ID: <4CD986DC.1070401@llnl.gov> Date: Tue, 09 Nov 2010 09:37:32 -0800 From: Mike Carlson User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); en-US; rv:1.9.2.12) Gecko/20101027 Lightning/1.0b2 Thunderbird/3.1.6 MIME-Version: 1.0 To: freebsd-fs@freebsd.org References: <4CD84258.6090404@llnl.gov> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: Re: 8.1-RELEASE: ZFS data errors X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 09 Nov 2010 17:37:32 -0000 On 11/09/2010 03:23 AM, Ivan Voras wrote: > On 11/08/10 19:32, Mike Carlson wrote: > >> As soon as I create the volume and write data to it, it is reported as >> being corrupted: >> >> write# zpool create filevol001 da2 da3 da4 da5 da6 da7 da8 >> However, if I create a 'raidz' volume, no errors occur: > A very interesting problem. Can you check with some other kind of volume > manager that striping the data doesn't cause some unusual hardware > interaction? Can you try, as an experiment, striping them all with > gstripe (but you'll have to use a small stripe size like 16 KiB or 8 KiB)? > > _______________________________________________ > freebsd-fs@freebsd.org mailing list > http://BLOCKEDlists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > Sure: write# gstripe label -v -s 16384 data /dev/da2 /dev/da3 /dev/da4 /dev/da5 /dev/da6 /dev/da7 /dev/da8 Metadata value stored on /dev/da2. Metadata value stored on /dev/da3. Metadata value stored on /dev/da4. Metadata value stored on /dev/da5. Metadata value stored on /dev/da6. Metadata value stored on /dev/da7. Metadata value stored on /dev/da8. Done. write# newfs -O2 -U /dev/stripe/data /dev/stripe/data: 133522760.0MB (273454612256 sectors) block size 16384, fragment size 2048 using 627697 cylinder groups of 212.72MB, 13614 blks, 6848 inodes. with soft updates super-block backups (for fsck -b #) at: ... write# mount /dev/stripe/data /mnt write# df -h Filesystem Size Used Avail Capacity Mounted on /dev/da0s1a 1.7T 22G 1.6T 1% / devfs 1.0K 1.0K 0B 100% /dev /dev/stripe/data 126T 4.0K 116T 0% /mnt write# cd /tmp write# md5 /tmp/random.dat.1 MD5 (/tmp/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36 write# cp random.dat.1 /mnt/ write# cp /mnt/random.dat.1 /mnt/random.dat.2 write# cp /mnt/random.dat.1 /mnt/random.dat.3 write# cp /mnt/random.dat.1 /mnt/random.dat.4 write# cp /mnt/random.dat.1 /mnt/random.dat.5 write# cp /mnt/random.dat.1 /mnt/random.dat.6 write# md5 /mnt/* MD5 (/mnt/random.dat.1) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/mnt/random.dat.2) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/mnt/random.dat.3) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/mnt/random.dat.4) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/mnt/random.dat.5) = f795fa09e1b0975c0da0ec6e49544a36 MD5 (/mnt/random.dat.6) = f795fa09e1b0975c0da0ec6e49544a36 write# fsck /mnt fsck: Could not determine filesystem type write# fsck_ufs /mnt ** /dev/stripe/data (NO WRITE) ** Last Mounted on /mnt ** Phase 1 - Check Blocks and Sizes Segmentation fault So, the data appears to be okay, I wanted to run through a FSCK just to do it but that seg faulted. Otherwise, that data looks good. Question, why did you recommend using a smaller stripe size? Is that to ensure a sample 1GB test file gets written across ALL disk members?