From owner-freebsd-fs@freebsd.org Sat Aug 5 17:17:09 2017 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 769C0DBF304; Sat, 5 Aug 2017 17:17:09 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from elf.hq.norma.perm.ru (mail.norma.perm.ru [IPv6:2a00:7540:1::5]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.norma.perm.ru", Issuer "Vivat-Trade UNIX Root CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EA57880A1E; Sat, 5 Aug 2017 17:17:08 +0000 (UTC) (envelope-from emz@norma.perm.ru) Received: from [IPv6:2a02:2698:26:9b0e:d7a:6d28:b07e:5ba6] (dynamic-2a02-2698-26-0-0.perm.ertelecom.ru [IPv6:2a02:2698:26:9b0e:d7a:6d28:b07e:5ba6] (may be forged)) (authenticated bits=0) by elf.hq.norma.perm.ru (8.15.2/8.15.2) with ESMTPSA id v75HH5Od007451 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sat, 5 Aug 2017 22:17:05 +0500 (YEKT) (envelope-from emz@norma.perm.ru) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=norma.perm.ru; s=key; t=1501953426; bh=5Rj0MOK5Zp+fXqvmVRPQ3oiWuRY57fHpAHlIRc8XapQ=; h=Subject:From:To:Cc:References:Date:In-Reply-To; b=eAiV6zEnd35O/25aq3n+amRLkDFFOrjd7dP6+YqVz4GnUuTChRkd5V6x5aADeGqe9 S9B064BPGb0gFNwPFuTwjJD82jHrHPHJtxYIr17TYtTc4tOtgPifQuuQxPN7aK0ScU H1UykqzMMUpcXAbRCBeXrlaiszUfF5hZqXcYHOeI= Subject: Re: a strange and terrible saga of the cursed iSCSI ZFS SAN From: "Eugene M. Zheganin" To: freebsd-stable@FreeBSD.org Cc: freebsd-fs@freebsd.org References: <1bd10b1e-0583-6f44-297e-3147f6daddc5@norma.perm.ru> Message-ID: <1d53f489-5135-7633-fef4-35d26e4969dc@norma.perm.ru> Date: Sat, 5 Aug 2017 22:16:51 +0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <1bd10b1e-0583-6f44-297e-3147f6daddc5@norma.perm.ru> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Content-Language: en-GB X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Aug 2017 17:17:09 -0000 Hi, On 05.08.2017 22:08, Eugene M. Zheganin wrote: > > pool: userdata > state: ONLINE > status: One or more devices has experienced an error resulting in data > corruption. Applications may be affected. > action: Restore the file in question if possible. Otherwise restore the > entire pool from backup. > see: http://illumos.org/msg/ZFS-8000-8A > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > userdata ONLINE 0 0 216K > mirror-0 ONLINE 0 0 432K > gpt/userdata0 ONLINE 0 0 432K > gpt/userdata1 ONLINE 0 0 432K That would be funny, if not that sad, but while writing this message, the pool started to look like below (I just asked zpool status twice in a row, comparing to what it was): [root@san1:~]# zpool status userdata pool: userdata state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://illumos.org/msg/ZFS-8000-8A scan: none requested config: NAME STATE READ WRITE CKSUM userdata ONLINE 0 0 728K mirror-0 ONLINE 0 0 1,42M gpt/userdata0 ONLINE 0 0 1,42M gpt/userdata1 ONLINE 0 0 1,42M errors: 4 data errors, use '-v' for a list [root@san1:~]# zpool status userdata pool: userdata state: ONLINE status: One or more devices has experienced an error resulting in data corruption. Applications may be affected. action: Restore the file in question if possible. Otherwise restore the entire pool from backup. see: http://illumos.org/msg/ZFS-8000-8A scan: none requested config: NAME STATE READ WRITE CKSUM userdata ONLINE 0 0 730K mirror-0 ONLINE 0 0 1,43M gpt/userdata0 ONLINE 0 0 1,43M gpt/userdata1 ONLINE 0 0 1,43M errors: 4 data errors, use '-v' for a list So, you see, the error rate is like speed of light. And I'm not sure if the data access rate is that enormous, looks like they are increasing on their own. So may be someone have an idea on what this really means. Thanks. Eugene.