From owner-freebsd-questions@freebsd.org Fri Dec 8 21:57:29 2017 Return-Path: Delivered-To: freebsd-questions@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A3D77E902EF for ; Fri, 8 Dec 2017 21:57:29 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost2.sentex.ca (smarthost2.sentex.ca [IPv6:2607:f3e0:80:80::2]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN "smarthost.sentex.ca", Issuer "smarthost.sentex.ca" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 81C8265A4E for ; Fri, 8 Dec 2017 21:57:29 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (lava.sentex.ca [IPv6:2607:f3e0:0:5::11]) by smarthost2.sentex.ca (8.15.2/8.15.2) with ESMTPS id vB8LvRKF013428 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO) for ; Fri, 8 Dec 2017 16:57:27 -0500 (EST) (envelope-from mike@sentex.net) Received: from [192.168.43.26] (saphire3.sentex.net [192.168.43.26]) by lava.sentex.ca (8.15.2/8.15.2) with ESMTP id vB8LvPLg046805 for ; Fri, 8 Dec 2017 16:57:25 -0500 (EST) (envelope-from mike@sentex.net) To: freebsd-questions From: Mike Tancsa Subject: zfs mirror and unequal writes Organization: Sentex Communications Message-ID: Date: Fri, 8 Dec 2017 16:57:23 -0500 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.5.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Scanned-By: MIMEDefang 2.78 X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Dec 2017 21:57:29 -0000 I was reading up on SSD life spans and decided to take a quick look at how many TBW my disks have gone through. Looking at one box I was really surprised at the uneven amount of data written 0{backup4}# smartctl -a /dev/ada0 | grep LBAs 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 21004341355 0{backup4}# smartctl -a /dev/ada1 | grep LBAs 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 413475182465 0{backup4}# I was quite surprised that there were WAY more writes to one disk that the other. Give these are zfs mirrors, one says 192TB writes and the other is 10TB ?!?!?! They were installed at the same time 1{backup4}# smartctl -a /dev/ada0 | grep Power_On 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 29445 0{backup4}# smartctl -a /dev/ada1 | grep Power_On 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 29275 0{backup4}# Any ideas what might be going on ? The Wear_Leveling_Count seems out of whack too, although I guess sort of expected ? 0{backup4}# smartctl -a /dev/ada1 | head smartctl 6.4 2015-06-04 r4109 [FreeBSD 10.4-STABLE amd64] (local build) Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Samsung based SSDs Device Model: Samsung SSD 840 PRO Series Serial Number: S1ANNSADB96894E LU WWN Device Id: 5 002538 5a0164a23 Firmware Version: DXM05B0Q User Capacity: 128,035,676,160 bytes [128 GB] 141{backup4}# smartctl -a /dev/ada0 | head smartctl 6.4 2015-06-04 r4109 [FreeBSD 10.4-STABLE amd64] (local build) Copyright (C) 2002-15, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Samsung based SSDs Device Model: Samsung SSD 840 PRO Series Serial Number: S1ANNSADB82440E LU WWN Device Id: 5 002538 5a015e935 Firmware Version: DXM05B0Q User Capacity: 128,035,676,160 bytes [128 GB] 141{backup4}# and ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 29275 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 49 177 Wear_Leveling_Count 0x0013 008 008 000 Pre-fail Always - 3332 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0 181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0 187 Uncorrectable_Error_Cnt 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 067 062 000 Old_age Always - 33 195 ECC_Error_Rate 0x001a 200 200 000 Old_age Always - 0 199 CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0 235 POR_Recovery_Count 0x0012 099 099 000 Old_age Always - 38 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 413482348777 ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 9 Power_On_Hours 0x0032 094 094 000 Old_age Always - 29445 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 51 177 Wear_Leveling_Count 0x0013 093 093 000 Pre-fail Always - 246 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0 181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0 187 Uncorrectable_Error_Cnt 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 067 062 000 Old_age Always - 33 195 ECC_Error_Rate 0x001a 200 200 000 Old_age Always - 0 199 CRC_Error_Count 0x003e 099 099 000 Old_age Always - 92 235 POR_Recovery_Count 0x0012 099 099 000 Old_age Always - 39 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 21004655923 -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/