From owner-freebsd-fs@freebsd.org Sat Feb 13 16:51:19 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3115A531DE0 for ; Sat, 13 Feb 2021 16:51:19 +0000 (UTC) (envelope-from se@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DdGbW0kn4z4jkJ; Sat, 13 Feb 2021 16:51:19 +0000 (UTC) (envelope-from se@freebsd.org) Received: from Stefans-MBP-449.fritz.box (p200300cd5f0ef900f88d5cfe3b39a25c.dip0.t-ipconnect.de [IPv6:2003:cd:5f0e:f900:f88d:5cfe:3b39:a25c]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client did not present a certificate) (Authenticated sender: se/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 9EA72F63B; Sat, 13 Feb 2021 16:51:18 +0000 (UTC) (envelope-from se@freebsd.org) To: Karl Denninger , freebsd-fs@freebsd.org References: <0ca45adf-8f60-a4c3-6264-6122444a3ffd@denninger.net> <899c6b4f-2368-7ec2-4dfe-fa09fab35447@artem.ru> <20210212165216.2f613482@fabiankeil.de> <10977ffc-f806-69dd-0cef-d4fd4fc5f649@artem.ru> <2f82f113-9ca1-99a9-a433-89e3ae5edcbe@denninger.net> <2bf4f69c-9d5d-5ff9-0daa-c87515437ca3@artem.ru> <5aa03138-1dc8-5a9c-1be6-d47ed22fc0cf@denninger.net> From: Stefan Esser Subject: Re: Reading a corrupted file on ZFS Message-ID: <225e4da5-79ec-a57a-90e5-35989e6484d5@freebsd.org> Date: Sat, 13 Feb 2021 17:51:14 +0100 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <5aa03138-1dc8-5a9c-1be6-d47ed22fc0cf@denninger.net> Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="uHKKMmNIODgVzi0bd1D7Ukk51iUrmlja9" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 13 Feb 2021 16:51:19 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --uHKKMmNIODgVzi0bd1D7Ukk51iUrmlja9 Content-Type: multipart/mixed; boundary="UPHcda4tR7pyN952ebyyHhwS9xDi1jfGI"; protected-headers="v1" From: Stefan Esser To: Karl Denninger , freebsd-fs@freebsd.org Message-ID: <225e4da5-79ec-a57a-90e5-35989e6484d5@freebsd.org> Subject: Re: Reading a corrupted file on ZFS References: <0ca45adf-8f60-a4c3-6264-6122444a3ffd@denninger.net> <899c6b4f-2368-7ec2-4dfe-fa09fab35447@artem.ru> <20210212165216.2f613482@fabiankeil.de> <10977ffc-f806-69dd-0cef-d4fd4fc5f649@artem.ru> <2f82f113-9ca1-99a9-a433-89e3ae5edcbe@denninger.net> <2bf4f69c-9d5d-5ff9-0daa-c87515437ca3@artem.ru> <5aa03138-1dc8-5a9c-1be6-d47ed22fc0cf@denninger.net> In-Reply-To: <5aa03138-1dc8-5a9c-1be6-d47ed22fc0cf@denninger.net> --UPHcda4tR7pyN952ebyyHhwS9xDi1jfGI Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: quoted-printable Am 12.02.21 um 20:49 schrieb Karl Denninger: > On 2/12/2021 13:26, Artem Kuchin wrote: >> 12.02.2021 19:37, Karl Denninger =D0=BF=D0=B8=D1=88=D0=B5=D1=82: >>> On 2/12/2021 11:22, Artem Kuchin wrote: >>>> >>>> This is frustrating. why..why.. >>> >>> You created a synthetic situation that in the real world almost-never= =20 >>> exists (ONE byte modified in all copies in the same allocation block = >>> but all other data in that block is intact and recoverable.) >>> >> I could be 1 GB file with ZFS wisth block size of 1MB and with rotten = >> bits within the same 1MB of block on different disks. How i did it is = >> not important, life is unpredictable, i'm not trying to avoid=20 >> everything. The question is what to do when it happens. And currently = >> the answer is - nothing. >> > The answer to a problem that does not present itself statistically=20 > speaking in the real world is the last one you worry about.=C2=A0 Worry= about=20 > all the other ones and cover them first. True for some kinds of real world, false for others. I have a number of older 2 TB drives and use 2 of them to store DVB transport stream data, for later processing. They are not mirrored and the data format assumes that the transmission is not perfect. I have noticed that some of these files have been lost due to bad sectors, and while this would not be a problem on UFS, ZFS thinks I should not trust the file at all, despite 99,9999% of it being perfectly usable. (A fraction of the TS data is extracted from these files and stored as PS data on a ZFS RAID, where it will be safe and I never lost a file.) There are other applications that do not care for all data to be perfectly conserved (I know of some) and where using a mirrored setup configuration would just be a waste of money. Not everybody operates a data center with a large drive farm that holds data worth magnitudes more than the hardware cost (in which case the cost of redundancy is well justified). A solution could be to use UFS for files that should be readable (or recoverable with a tool) after single sector failures. But why should I have to mix UFS for such data with ZFS for the rest of the system (and I know you very well that these two do not mix too well if both are put under simultaneous high load). Therefore: There are valid reasons to not have redundancy and still not loose a large file of data for single bad block of a non-redundant (e.g striped for performance) ZPOOL/ZFS. Regards, STefan --UPHcda4tR7pyN952ebyyHhwS9xDi1jfGI-- --uHKKMmNIODgVzi0bd1D7Ukk51iUrmlja9 Content-Type: application/pgp-signature; name="OpenPGP_signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="OpenPGP_signature" -----BEGIN PGP SIGNATURE----- wsB5BAABCAAjFiEEo3HqZZwL7MgrcVMTR+u171r99UQFAmAoA4IFAwAAAAAACgkQR+u171r99USa mgf/Yp8uDF4C8g33eRB0HuRjkOiY4rkgjPgY9xPdBi9WwnIfxfeJd9s4xTrmKlSdtEXaDukC0BZ/ K8oRZmtcAOo1XidluL5dP8BIJQzhewwxIv5JcFjQlcbNBEqxy7D4YYnzIEs1tebM60BajWkY1Y8M cS3GCvxVRAgmCLMWtUG+WIV0bsrv3TosJTRWMfugaEjUzn5lbvtKDWk6YEAMj2NMtqaF6zWT+KsX PClwuyiX9waAzR2YAQMMywJnQWrzZt0Ape5gW0ZdyUno1Y6asCe8ApsHSxNED2z3O2lGh/6h5Iml Ipp9BehNJ9LLPNqlhRPGOrMlJ15E4/UThJyfTVKm0g== =SjJL -----END PGP SIGNATURE----- --uHKKMmNIODgVzi0bd1D7Ukk51iUrmlja9--