From owner-freebsd-questions@FreeBSD.ORG Sun Feb 3 21:14:30 2008 Return-Path: Delivered-To: questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 12EF116A419 for ; Sun, 3 Feb 2008 21:14:30 +0000 (UTC) (envelope-from gunther.mayer@googlemail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.173]) by mx1.freebsd.org (Postfix) with ESMTP id 9AD1E13C44B for ; Sun, 3 Feb 2008 21:14:29 +0000 (UTC) (envelope-from gunther.mayer@googlemail.com) Received: by ug-out-1314.google.com with SMTP id y2so1425056uge.37 for ; Sun, 03 Feb 2008 13:14:28 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:user-agent:mime-version:to:subject:content-type:content-transfer-encoding:from; bh=6tT20tgmp/m3AiSqV/MQHnHv7d9o71qAsdsrDVqL6mU=; b=mnhoh8rat2jvZlU3x+Xj48JVyB4GERNtGGR71X7bUtYDa4IDy7WmcTcu8jTQp0tGFdWzaHt/FVhFss5L1lH4B4/wEgjZen/JWD1HLBqD+TB6SRhJv7EDyOn4iqEENh26g/KFPxbxQztS/YPzjHn6jEkcJyQu5uLUOio7QogjJh0= DomainKey-Signature: a=rsa-sha1; c=nofws; d=googlemail.com; s=gamma; h=message-id:date:user-agent:mime-version:to:subject:content-type:content-transfer-encoding:from; b=hjcinR7CCB2IJd8kUv5eUbTpxDmGyyqQqFBBb/5tkOxzxT29cJOK1Ib6lDVOigSxm1pcrNonwrLoiBrZZAUPww30fdvglYCzA3y5W2PG2IbvNepHvWtgKmNx4DVZ5Cxk27FsAsTNrUq5cYhvIFo7j8AM24weB54EhlVJR+KJDIc= Received: by 10.66.254.19 with SMTP id b19mr1343866ugi.7.1202071706589; Sun, 03 Feb 2008 12:48:26 -0800 (PST) Received: from ?172.25.0.18? ( [41.245.183.118]) by mx.google.com with ESMTPS id u6sm18071359uge.83.2008.02.03.12.48.23 (version=TLSv1/SSLv3 cipher=RC4-MD5); Sun, 03 Feb 2008 12:48:25 -0800 (PST) Message-ID: <47A6287C.9040902@gmail.com> Date: Sun, 03 Feb 2008 22:47:56 +0200 User-Agent: Thunderbird 2.0.0.6 (X11/20071022) MIME-Version: 1.0 To: questions@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit From: Gunther Mayer Cc: Subject: automatic fsck on gmirror failure X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 03 Feb 2008 21:14:30 -0000 Hi there, I have a RAID 1 mirror implemented with gmirror and we recently had some power issues at our data centre which caused fsck to fail mysteriously. The server lost power unexpectedly, then came back up again for a minute, power died again and shortly after the next boot the following appears in my /var/log/messages Feb 2 05:20:19 myserver fsck: /dev/mirror/gm0s1f: INCORRECT BLOCK COUNT I=777684 (8 should be 0) (CORRECTED) Feb 2 05:20:19 myserver fsck: /dev/mirror/gm0s1f: CANNOT READ BLK: 12417184 Feb 2 05:20:19 myserver fsck: /dev/mirror/gm0s1f: UNEXPECTED SOFT UPDATE INCONSISTENCY; RUN fsck MANUALLY. gm0s1f is my /usr partition. This was followed by countless errors that look like Feb 2 05:20:38 myserver ad6: TIMEOUT - READ_DMA retrying (1 retry left) LBA=29096879 Feb 2 05:20:43 myserver ad6: TIMEOUT - READ_DMA retrying (0 retries left) LBA=29096879 Feb 2 05:20:48 myserver ad6: FAILURE - READ_DMA timed out LBA=29096879 Feb 2 05:20:48 myserver g_vfs_done():mirror/gm0s1f[READ(offset=6357598208, length=16384)]error = 5 and with it went any sort of remote access to the box. We had to get physical access, fsck -y and reboot for the machine to be put back into service. Now my question is: Why did fsck die on me? I thought in this day and age file system corruptions caused by power failures are repaired automatically upon reboot. Or is it possible that interrupting fsck itself caused the problem when the system went down again after the very brief uptime in between? I am really concerned about this as this caused a lot of unnecessary downtime and I really don't want this to ever happen again. I know, solving the power issues is the real solution but I want my several layers of peace of mind. Oh, I run 6.2 RELEASE. Gunther