Skip site navigation (1)Skip section navigation (2)
Date:      Sun, 20 Dec 2009 15:14:53 -0800
From:      johnea <me@johnea.net>
To:        freebsd-geom@freebsd.org
Subject:   errrors resyncronizing mirror with new disk
Message-ID:  <20091220151453.3dbe738e@johnea.net>

next in thread | raw e-mail | index | archive | help

Hello,

Recently a drive started to have READ_DMA failures and drop out of the gmirror.

While synching the replacement drive to the mirror, a WRITE error occured and terminated the sync.

A second attempt to sync completed but left the drive generating READ errors.

The third time the drive was inserted in the mirror the operation completed and no further errors have occured.

R'ingTFM and extensive scroogling have not yielded an explanation.

Is this expected behaviour? Should I suspect this drive? 
(it's brand new, but the "old" one was only 9 months)

Additionally smartctl on the new drive continues to indicate no errors.
(it did indicate each of the read errors on the old drive)

Below are the relevant syslog excerpts.

Thank You!

johnea

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

#1 The 3rd OLD disk read failure:

Dec 17 03:03:23 atom kernel: ad4: FAILURE - READ_DMA status=51<READY,DSC,ERROR> error=40<UNCORRECTABLE> LBA=9787039
Dec 17 03:03:23 atom kernel: GEOM_MIRROR: Request failed (error=5). ad4[READ(offset=5010963968, length=16384)]
Dec 17 03:03:23 atom kernel: GEOM_MIRROR: Device gm0: provider ad4 disconnected.
Dec 17 03:24:02 atom smartd[41810]: Device: /dev/ad4, 1 Currently unreadable (pending) sectors
Dec 17 03:24:02 atom smartd[41810]: Device: /dev/ad4, 1 Offline uncorrectable sectors
Dec 17 03:24:02 atom smartd[41810]: Device: /dev/ad4, ATA error count increased from 2 to 3
Dec 17 03:54:02 atom smartd[41810]: Device: /dev/ad4, 1 Currently unreadable (pending) sectors
Dec 17 03:54:02 atom smartd[41810]: Device: /dev/ad4, 1 Offline uncorrectable sectors
Dec 17 04:24:02 atom smartd[41810]: Device: /dev/ad4, 1 Currently unreadable (pending) sectors
Dec 17 04:24:02 atom smartd[41810]: Device: /dev/ad4, 1 Offline uncorrectable sectors
Last 2 lines repeating every 30 minutes...

#2 Rebooting after removing OLD disk and installing NEW disk:

Dec 17 18:00:14 atom kernel: GEOM_MIRROR: Force device gm0 start due to timeout.
Dec 17 18:00:14 atom kernel: Root mount waiting for: GMIRROR
Dec 17 18:00:14 atom kernel: GEOM_MIRROR: Device mirror/gm0 launched (1/2).
Dec 17 18:00:14 atom kernel: Trying to mount root from ufs:/dev/mirror/gm0s1a

#3 Write Error after issuing 'gmirror forget gm0; gmirror insert gm0 /dev/ad4' on NEW disk:

Dec 17 18:14:59 atom kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad4.
...
Dec 17 19:41:33 atom kernel: ad4: WARNING - WRITE_DMA48 UDMA ICRC error (retrying request) LBA=955081984
Dec 17 19:41:39 atom kernel: ad4: TIMEOUT - WRITE_DMA48 retrying (0 retries left) LBA=955081984
Dec 17 19:41:45 atom kernel: ad4: FAILURE - WRITE_DMA48 timed out LBA=955081984
Dec 17 19:41:45 atom kernel: GEOM_MIRROR: Synchronization request failed (error=5). ad4[WRITE(offset=489001975808, length=131072)]
Dec 17 19:41:45 atom kernel: GEOM_MIRROR: Device gm0: provider ad4 disconnected.
Dec 17 19:41:45 atom kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad4 stopped.

#4 Read Errors after 2nd 'gmirror forget gm0; gmirror insert gm0 /dev/ad4' on NEW disk:

Dec 17 19:56:54 atom kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad4.
Dec 17 23:16:26 atom kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad4 finished.
Dec 18 01:00:42 atom smartd[724]: Device: /dev/ad4, 4 Currently unreadable (pending) sectors
Dec 18 01:00:43 atom smartd[724]: Device: /dev/ad4, 4 Offline uncorrectable sectors
Dec 18 01:30:42 atom smartd[724]: Device: /dev/ad4, 4 Currently unreadable (pending) sectors
Dec 18 01:30:42 atom smartd[724]: Device: /dev/ad4, 4 Offline uncorrectable sectors
Dec 18 02:00:42 atom smartd[724]: Device: /dev/ad4, 8 Currently unreadable (pending) sectors
Dec 18 02:00:42 atom smartd[724]: Device: /dev/ad4, 8 Offline uncorrectable sectors
Dec 18 02:30:42 atom smartd[724]: Device: /dev/ad4, 8 Currently unreadable (pending) sectors
Dec 18 02:30:42 atom smartd[724]: Device: /dev/ad4, 8 Offline uncorrectable sectors

#5 Log entries around issuing 'gmirror remove gm0 ad4; gmirror insert gm0 ad4':

Dec 19 10:00:42 atom smartd[724]: Device: /dev/ad4, 8 Currently unreadable (pending) sectors
Dec 19 10:00:42 atom smartd[724]: Device: /dev/ad4, 8 Offline uncorrectable sectors
Dec 19 10:30:41 atom smartd[724]: Device: /dev/ad4, 8 Currently unreadable (pending) sectors
Dec 19 10:30:41 atom smartd[724]: Device: /dev/ad4, 8 Offline uncorrectable sectors
Dec 19 10:44:00 atom kernel: GEOM_MIRROR: Device gm0: provider ad4 destroyed.
Dec 19 10:44:17 atom kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad4.
Dec 19 11:00:42 atom smartd[724]: Device: /dev/ad4, 8 Currently unreadable (pending) sectors
Dec 19 11:00:42 atom smartd[724]: Device: /dev/ad4, 8 Offline uncorrectable sectors
Dec 19 14:05:24 atom kernel: GEOM_MIRROR: Device gm0: rebuilding provider ad4 finished.



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20091220151453.3dbe738e>