Date: Fri, 30 Dec 2005 11:18:06 +0100 From: "O. Hartmann" <ohartman@mail.uni-mainz.de> To: =?ISO-8859-1?Q?=22Se=E1n_C=2E_Farley=22?= <sean-freebsd@farley.org> Cc: Brian Fundakowski Feldman <green@freebsd.org>, freebsd-stable@freebsd.org Subject: Re: File system corruption with ATA RAID-1 on 6-STABLE Message-ID: <43B5095E.1060201@mail.uni-mainz.de> In-Reply-To: <20051229143004.W17588@thor.farley.org> References: <20051119112837.O15168@thor.farley.org> <20051221220315.GH35080@green.homeunix.org> <20051229143004.W17588@thor.farley.org>
next in thread | previous in thread | raw e-mail | index | archive | help
Seán C. Farley schrieb: > On Wed, 21 Dec 2005, Brian Fundakowski Feldman wrote: > >> On Sat, Nov 19, 2005 at 12:06:00PM -0600, Seán C. Farley wrote: >> >>> I installed 6.0 on a system with two SATA drives in RAID-1. 5.4 is >>> installed on a standard IDE channel. All appeared to work well until >>> sometime after updating the system to 6-STABLE, installing the Nvidia >>> driver and copying a lot of data (mainly ogg files) from my old hard >>> drive onto it. I started seeing a multitude of READ_DMA (or maybe >>> WRITE_DMA) errors in /var/log/messages. The system was beginning to >>> slow down a lot, so I rebooted. Since then I have been unable to >>> keep it up without a panic after an fsck in single-user mode. >>> >>> I realize this does not tell much, but just when ld.so.hints is being >>> removed due to being corrupt the follow panic appears: Panic: >>> ffs_valloc: dup alloc The panic may just be from a system that is too >>> corrupted. I am amazed it still boots. >> >> >> That's how it appears, yeah. >> >> What does the drive health look like? Seems most like a chipset bug >> or nvidia driver bug, but could be things are just coinciding with >> drive or drive controller failure. > > > I have since reinstalled and have been unable to reproduce the problem. > 6-STABLE as of November 28 works without any obvious problems. I have > also used the Nvidia driver v8174 and v8178. > > I do see this in messages (only one): > Dec 16 02:31:46 thor kernel: ad6: TIMEOUT - WRITE_DMA retrying (1 retry > left) LBA=195511730 > > I am also attaching the information about my SATA drives (from > smartctl). The first drive (ad4) has three errors logged. > > Seán > > > ------------------------------------------------------------------------ > > smartctl version 5.33 [i386-portbld-freebsd6.0] Copyright (C) 2002-4 Bruce Allen > Home page is http://smartmontools.sourceforge.net/ > > === START OF INFORMATION SECTION === > Device Model: ST3160827AS > Serial Number: 4MT0L7EY > Firmware Version: 3.42 > User Capacity: 160,041,885,696 bytes > Device is: Not in smartctl database [for details use: -P showall] > ATA Version is: 6 > ATA Standard is: ATA/ATAPI-6 T13 1410D revision 2 > Local Time is: Thu Dec 29 15:08:26 2005 CST > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > === START OF READ SMART DATA SECTION === > SMART overall-health self-assessment test result: PASSED > > General SMART Values: > Offline data collection status: (0x82) Offline data collection activity > was completed without error. > Auto Offline Data Collection: Enabled. > Self-test execution status: ( 0) The previous self-test routine completed > without error or no self-test has ever > been run. > Total time to complete Offline > data collection: ( 430) seconds. > Offline data collection > capabilities: (0x5b) SMART execute Offline immediate. > Auto Offline data collection on/off support. > Suspend Offline collection upon new > command. > Offline surface scan supported. > Self-test supported. > No Conveyance Self-test supported. > Selective Self-test supported. > SMART capabilities: (0x0003) Saves SMART data before entering > power-saving mode. > Supports SMART auto save timer. > Error logging capability: (0x01) Error logging supported. > No General Purpose Logging support. > Short self-test routine > recommended polling time: ( 1) minutes. > Extended self-test routine > recommended polling time: ( 94) minutes. > > SMART Attributes Data Structure revision number: 10 > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 057 046 006 Pre-fail Always - 66158687 > 3 Spin_Up_Time 0x0003 099 099 000 Pre-fail Always - 0 > 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 3 > 5 Reallocated_Sector_Ct 0x0033 099 099 036 Pre-fail Always - 46 > 7 Seek_Error_Rate 0x000f 083 060 030 Pre-fail Always - 210711557 > 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3673 > 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 > 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 3 > 194 Temperature_Celsius 0x0022 030 040 000 Old_age Always - 30 (Lifetime Min/Max 0/21) > 195 Hardware_ECC_Recovered 0x001a 057 046 000 Old_age Always - 66158687 > 197 Current_Pending_Sector 0x0012 100 099 000 Old_age Always - 0 > 198 Offline_Uncorrectable 0x0010 100 099 000 Old_age Offline - 0 > 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 > 200 Multi_Zone_Error_Rate 0x0000 100 253 000 Old_age Offline - 0 > 202 TA_Increase_Count 0x0032 100 253 000 Old_age Always - 0 > > SMART Error Log Version: 1 > ATA Error Count: 3 > CR = Command Register [HEX] > FR = Features Register [HEX] > SC = Sector Count Register [HEX] > SN = Sector Number Register [HEX] > CL = Cylinder Low Register [HEX] > CH = Cylinder High Register [HEX] > DH = Device/Head Register [HEX] > DC = Device Command Register [HEX] > ER = Error register [HEX] > ST = Status register [HEX] > Powered_Up_Time is measured from power on, and printed as > DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, > SS=sec, and sss=millisec. It "wraps" after 49.710 days. > > Error 3 occurred at disk power-on lifetime: 2904 hours (121 days + 0 hours) > When the command that caused the error occurred, the device was active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 00 1f b1 67 e8 Error: UNC at LBA = 0x0867b11f = 141013279 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 20 1f b1 67 e8 00 22:51:33.459 READ DMA > ca 00 08 eb 07 c1 e4 00 22:51:33.459 WRITE DMA > c6 00 10 00 00 00 e0 00 22:51:33.459 SET MULTIPLE MODE > ef 02 00 00 00 00 e0 00 22:51:33.459 SET FEATURES [Enable write cache] > ef aa 00 00 00 00 e0 00 22:51:33.459 SET FEATURES [Enable read look-ahead] > > Error 2 occurred at disk power-on lifetime: 2904 hours (121 days + 0 hours) > When the command that caused the error occurred, the device was active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 00 1f b1 67 e8 Error: UNC at LBA = 0x0867b11f = 141013279 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 20 1f b1 67 e8 00 22:51:33.459 READ DMA > ca 00 04 5b 04 c1 e4 00 22:51:33.459 WRITE DMA > c6 00 10 00 00 00 e0 00 22:51:33.459 SET MULTIPLE MODE > ef 02 00 00 00 00 e0 00 22:51:33.459 SET FEATURES [Enable write cache] > ef aa 00 00 00 00 e0 00 22:51:33.459 SET FEATURES [Enable read look-ahead] > > Error 1 occurred at disk power-on lifetime: 2904 hours (121 days + 0 hours) > When the command that caused the error occurred, the device was active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 00 1f b1 67 e8 Error: UNC at LBA = 0x0867b11f = 141013279 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 20 1f b1 67 e8 00 22:51:28.346 READ DMA > ca 00 0c df 07 c1 e4 00 22:51:28.345 WRITE DMA > ca 00 14 5f b9 76 e3 00 22:51:28.342 WRITE DMA > ca 00 20 df 9c 75 e3 00 22:51:28.334 WRITE DMA > ca 00 20 1f 32 ba ec 00 22:51:28.333 WRITE DMA > > SMART Self-test log structure revision number 1 > Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error > # 1 Short offline Completed without error 00% 3672 - > > SMART Selective self-test log data structure revision number 1 > SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS > 1 0 0 Not_testing > 2 0 0 Not_testing > 3 0 0 Not_testing > 4 0 0 Not_testing > 5 0 0 Not_testing > Selective self-test flags (0x0): > After scanning selected spans, do NOT read-scan remainder of disk. > If Selective self-test is pending on power-up, resume after 0 minute delay. > > > > ------------------------------------------------------------------------ > > smartctl version 5.33 [i386-portbld-freebsd6.0] Copyright (C) 2002-4 Bruce Allen > Home page is http://smartmontools.sourceforge.net/ > > === START OF INFORMATION SECTION === > Device Model: ST3160827AS > Serial Number: 4MT0LNFT > Firmware Version: 3.42 > User Capacity: 160,041,885,696 bytes > Device is: Not in smartctl database [for details use: -P showall] > ATA Version is: 6 > ATA Standard is: ATA/ATAPI-6 T13 1410D revision 2 > Local Time is: Thu Dec 29 15:08:30 2005 CST > SMART support is: Available - device has SMART capability. > SMART support is: Enabled > > === START OF READ SMART DATA SECTION === > SMART overall-health self-assessment test result: PASSED > > General SMART Values: > Offline data collection status: (0x82) Offline data collection activity > was completed without error. > Auto Offline Data Collection: Enabled. > Self-test execution status: ( 0) The previous self-test routine completed > without error or no self-test has ever > been run. > Total time to complete Offline > data collection: ( 430) seconds. > Offline data collection > capabilities: (0x5b) SMART execute Offline immediate. > Auto Offline data collection on/off support. > Suspend Offline collection upon new > command. > Offline surface scan supported. > Self-test supported. > No Conveyance Self-test supported. > Selective Self-test supported. > SMART capabilities: (0x0003) Saves SMART data before entering > power-saving mode. > Supports SMART auto save timer. > Error logging capability: (0x01) Error logging supported. > No General Purpose Logging support. > Short self-test routine > recommended polling time: ( 1) minutes. > Extended self-test routine > recommended polling time: ( 94) minutes. > > SMART Attributes Data Structure revision number: 10 > Vendor Specific SMART Attributes with Thresholds: > ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE > 1 Raw_Read_Error_Rate 0x000f 063 052 006 Pre-fail Always - 89874069 > 3 Spin_Up_Time 0x0003 099 099 000 Pre-fail Always - 0 > 4 Start_Stop_Count 0x0032 100 100 020 Old_age Always - 3 > 5 Reallocated_Sector_Ct 0x0033 100 100 036 Pre-fail Always - 0 > 7 Seek_Error_Rate 0x000f 083 060 030 Pre-fail Always - 215371066 > 9 Power_On_Hours 0x0032 096 096 000 Old_age Always - 3673 > 10 Spin_Retry_Count 0x0013 100 100 097 Pre-fail Always - 0 > 12 Power_Cycle_Count 0x0032 100 100 020 Old_age Always - 3 > 194 Temperature_Celsius 0x0022 030 040 000 Old_age Always - 30 (Lifetime Min/Max 0/20) > 195 Hardware_ECC_Recovered 0x001a 063 052 000 Old_age Always - 89874069 > 197 Current_Pending_Sector 0x0012 100 100 000 Old_age Always - 0 > 198 Offline_Uncorrectable 0x0010 100 100 000 Old_age Offline - 0 > 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 > 200 Multi_Zone_Error_Rate 0x0000 100 253 000 Old_age Offline - 0 > 202 TA_Increase_Count 0x0032 100 253 000 Old_age Always - 0 > > SMART Error Log Version: 1 > No Errors Logged > > SMART Self-test log structure revision number 1 > Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error > # 1 Short offline Completed without error 00% 3672 - > > SMART Selective self-test log data structure revision number 1 > SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS > 1 0 0 Not_testing > 2 0 0 Not_testing > 3 0 0 Not_testing > 4 0 0 Not_testing > 5 0 0 Not_testing > Selective self-test flags (0x0): > After scanning selected spans, do NOT read-scan remainder of disk. > If Selective self-test is pending on power-up, resume after 0 minute delay. > > > > ------------------------------------------------------------------------ > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" I had a similar problem with a FreeBSD 5.4. Suddenly, one of two drive, a Samsung SpinPoint SP2004C began to produce wild errors, then, after a while, everything seems to be clear. I also did a SMART self test, without revealing any obscure matter. At the end, suspecting everything except the cabling, I changed the cabling and that's it! I do not know why and how, the cable has no obvious damage (but maybe broken copper wires). It's a simple hint, but test it, it is maybe worth testing it. SATA cabling is really a worse cheap type of cabling. The idea behind is good, but the 'how they made is reality' is a horror! O.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?43B5095E.1060201>