From owner-freebsd-fs@FreeBSD.ORG Sun Jan 24 00:15:07 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3C05F106566B for ; Sun, 24 Jan 2010 00:15:07 +0000 (UTC) (envelope-from rincebrain@gmail.com) Received: from mail-px0-f183.google.com (mail-px0-f183.google.com [209.85.216.183]) by mx1.freebsd.org (Postfix) with ESMTP id 0AD5A8FC0A for ; Sun, 24 Jan 2010 00:15:06 +0000 (UTC) Received: by pxi13 with SMTP id 13so1662038pxi.3 for ; Sat, 23 Jan 2010 16:15:06 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:received:in-reply-to:references :date:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=pbAOg+z9kuCSEo6Jw7Gvkzc1GTxADwHZey7MzZEyp1M=; b=LFLD2SEaTBY9ajtv/5ovkGA0Reto7//tZFcJgz/LkhV3OYLyXYQXzCm5HqqyfZZFVo tnwPOzb2hs5xO7gHVphaPe8bWDG8/8xuyjARXRYjURHab1DLMhBiHqexEI6x+dWfbIv3 qdFt1E0d64EfPtDUcxpsD4DvkMNPwm6fR/dmE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; b=Rga+9cKA3J3Mf6/8Q4RM+eOnn6BGWL7d+7CaktBNEg2LHXj924QlxSJM1Xv9aVBQlZ kJyy6V/rnAZC8GIVnqC9d+v2zVsl/dJVOzWEnPh9mOUizRPIKvwrlNXq4/VXrRvpCInH CKixjxudmVKRKKJATc3Ps6gDk7Bx/5rYz5XPg= MIME-Version: 1.0 Received: by 10.114.188.8 with SMTP id l8mr3296086waf.12.1264292106477; Sat, 23 Jan 2010 16:15:06 -0800 (PST) In-Reply-To: References: <5da0588e1001222223m773648am907267235bdcf882@mail.gmail.com> <5da0588e1001230014k1b8a32f8v42046497265429ed@mail.gmail.com> <5da0588e1001231415t403f29ceq6e8dcd16edb4a28@mail.gmail.com> <5da0588e1001231541l246769eao410c5ea6ccca0de4@mail.gmail.com> Date: Sat, 23 Jan 2010 19:15:06 -0500 Message-ID: <5da0588e1001231615t37c22575uedaae938be40f530@mail.gmail.com> From: Rich To: Steven Schlansker Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Cc: freebsd-fs Subject: Re: Errors on a file on a zpool: How to remove? X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 24 Jan 2010 00:15:07 -0000 If there were no other option, certainly. I claim this is still Bad Behavior, and should be resolvable without doing something like that. - Rich On Sat, Jan 23, 2010 at 7:09 PM, Steven Schlansker wrote: > Perhaps you could create a new filesystem, say mirrors_new > Move all files that you can read > Destroy the bugged filesystem > Rename the new filesystem over the old one > Restore the missing files from backup > > Sound reasonable? > > On Jan 23, 2010, at 3:41 PM, Rich wrote: > >> I have no files named 0x0. >> >> I have a number of files which, on attempting to do anything to them >> (stat, mv, rm), EIO occurs, the checksum error number on three of the >> disks in that pool ticks up, and /var/log/messages reports what I >> reported in my initial post. (i discovered this due to FreeBSD's daily >> check-for-setuid-bits-in-strange-places find command reporting EIO on >> some files.) >> >> My original post in this thread is about how to resolve this. >> >> - Rich >> >> On Sat, Jan 23, 2010 at 6:34 PM, Wes Morgan wrot= e: >>> On Sat, 23 Jan 2010, Rich wrote: >>> >>>> On Sat, Jan 23, 2010 at 4:21 PM, Wes Morgan wr= ote: >>>>> On Sat, 23 Jan 2010, Rich wrote: >>>>> >>>>>> I already diagnosed the bad hardware - one of the two sticks of RAM >>>>>> had gone bad, and fails memtest in the other machine. >>>>>> >>>>>> =A0 pool: rigatoni >>>>>> =A0state: ONLINE >>>>>> status: One or more devices has experienced an error resulting in da= ta >>>>>> =A0 =A0 =A0 corruption. =A0Applications may be affected. >>>>>> action: Restore the file in question if possible. =A0Otherwise resto= re the >>>>>> =A0 =A0 =A0 entire pool from backup. >>>>>> =A0 =A0see: http://www.sun.com/msg/ZFS-8000-8A >>>>>> =A0scrub: scrub completed after 15h28m with 1 errors on Thu Jan 21 1= 8:09:25 2010 >>>>>> config: >>>>>> >>>>>> =A0 =A0 =A0 NAME =A0 =A0 =A0 =A0STATE =A0 =A0 READ WRITE CKSUM >>>>>> =A0 =A0 =A0 rigatoni =A0 =A0ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 =A0 1 >>>>>> =A0 =A0 =A0 =A0 da4 =A0 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 2 >>>>>> =A0 =A0 =A0 =A0 da5 =A0 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 2 >>>>>> =A0 =A0 =A0 =A0 da7 =A0 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >>>>>> =A0 =A0 =A0 =A0 da6 =A0 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 0 >>>>>> =A0 =A0 =A0 =A0 da2 =A0 =A0 =A0 ONLINE =A0 =A0 =A0 0 =A0 =A0 0 =A0 = =A0 2 >>>>>> >>>>>> errors: Permanent errors have been detected in the following files: >>>>>> >>>>>> =A0 =A0 =A0 =A0 rigatoni/mirrors:<0x0> >>>>> >>>>> Can you post your entire pool filesystem structure? That message abov= e >>>>> looks like an unreferenced block or corrupted metadata rather than an >>>>> actual file. Also, if it's part of a snapshot, you simply have to des= troy >>>>> the snapshot. >>>>> >>>>> I had a pool become corrupted due to bad memory, and all of the files= were >>>>> still able to be manipulated. The only time EIO popped up was on the >>>>> specific block that had a checksum error. >>>> >>>> # zfs list -r -t all rigatoni >>>> NAME =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0USED =A0AVAIL =A0REFER =A0MOUN= TPOINT >>>> rigatoni =A0 =A0 =A0 =A0 =A0 =A0 5.73T =A0 984G =A0 =A019K =A0/rigaton= i >>>> rigatoni/logs_bitch =A0 269M =A0 984G =A0 269M =A0/rigatoni/logs_bitch >>>> rigatoni/mirrors =A0 =A0 5.73T =A0 984G =A05.73T =A0/mirrors >>>> >>>> No snapshots here. :/ >>>> >>>> EIO only pops up on the files I mentioned above - everything else in >>>> those directories, including renaming that directory, is fine. >>> >>> I must have missed it, what files is it showing besides the <0x0> addre= ss? >>> Or do you have a file named "<0x0>"? >> >> >> >> -- >> >> Life is a yo-yo, and mankind ties knots in the string. >> _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" > > --=20 A foolish consistency is the hobgoblin of little minds. -- Ralph Waldo Emer= son