From owner-freebsd-fs@freebsd.org Tue Jun 30 01:42:03 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 3F5C535A996 for ; Tue, 30 Jun 2020 01:42:03 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 49wnCb0gvsz3SYx for ; Tue, 30 Jun 2020 01:42:02 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 5A28888C5 for ; Tue, 30 Jun 2020 01:42:02 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 5A28888C5 From: Allan Jude Subject: Re: ZSTD Project Weekly Status Update To: freebsd-fs References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Message-ID: <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> Date: Mon, 29 Jun 2020 21:42:01 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.9.0 MIME-Version: 1.0 In-Reply-To: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="612zfeGV38IdUQknGJ724Lxcls0v44tr2" X-Rspamd-Queue-Id: 49wnCb0gvsz3SYx X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US]; local_wl_from(0.00)[freebsd.org] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 30 Jun 2020 01:42:03 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --612zfeGV38IdUQknGJ724Lxcls0v44tr2 Content-Type: multipart/mixed; boundary="AUlwrInjLHPj4CRv5GJltZ10ewp1oFUWZ"; protected-headers="v1" From: Allan Jude To: freebsd-fs Message-ID: <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> Subject: Re: ZSTD Project Weekly Status Update References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> In-Reply-To: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> --AUlwrInjLHPj4CRv5GJltZ10ewp1oFUWZ Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable In my continued effort to finish the integration of ZSTD into ZFS, here is my second weekly status report: There is still a number of existing feedback items and known issues that need to be addressed. I am trying to work through those now. https://github.com/openzfs/zfs/commit/622b7e1e50ab667a6af1685245a2f5a8d5e= 9bff3 - Addressed an issue where the user could manually set the hidden compress_level property, causing incorrect operation. The property is not marked with the PROP_READONLY flag because it requires PROP_INHERIT - It has been pointed out again recently that setting the compress=3Dzstd= property on a dataset, but not actually writing any blocks, does not set the zstd feature flag to 'active'. If this pool is then exported, and imported using an older version of ZFS that does not know of zstd, it will trigger an ASSERT() when the value of the compression property enum is out-of-range. The plan is to fully activate the feature when the property is set, but the details of how (and where) to do still still need to be worked out. - I am still working on the issue of inheritance for both the compress and the hidden compress_level properties. If you create a child dataset with the compress property set to zstd-13, it works as expected. But if you `zfs inherit compress` on that dataset, the output of `zfs get compress,compress_level` changes from: zof/inherit/zstd-10/zstd-13 compression zstd-13 local zof/inherit/zstd-10/zstd-13 compress_level 13 local to: zof/inherit/zstd-10/zstd-13 compression zstd-13 inherited from zof/inherit/zstd-10 zof/inherit/zstd-10/zstd-13 compress_level 13 local This is due to the fact that both the parent, and the child, actually have compress=3D16 (zstd), and the zstd-10 or zstd-13 is determined by combining the hidden compress_level property. The expected behaviour in this case would be that the compression type (and therefore the compress_level) would get reset to the value from the parent. There is a related problem when the child's compression setting is set to lz4 (or any other type that doesn't use a level). This project was sponsored by the FreeBSD Foundation. --=20 Allan Jude --AUlwrInjLHPj4CRv5GJltZ10ewp1oFUWZ-- --612zfeGV38IdUQknGJ724Lxcls0v44tr2 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJe+phpAAoJEBmVNT4SmAt+U98P/0RI9o/VfCUVPNs0vIjpmwxw edpO5chNAcbyISVXws6/e7rM6EVwxZeNfNRpJbIEkuIP/JcPxAwCu+FXWcrSZlLt psShyzIGJD4S2ZI53ZUnksa3KEO266pu3H79hiELBzhlvPbVfJBp8kQ/O0adxXnF MI637R/ykEgUqKEjBaZaod83Si3+JOy+PRCHNTccHeRY+x2NHAAyGYdYl7LocyaA MzJvQDiPdJS/EB/xusiToNFDkeOFw5kPgVc8siuHi6uAmFsHp3fbETTBOZP37A0d PmFPyXFtTeoZbX+q/ipsKNpX0xLKd6kuor8wAfJoFuWM1czlV733JqIkaNxoqI1w MgKEPoXXaQhR9cANaLgOiaCMLS7XdCGUMQ+8spyhpEVgKY/zK5DM1Hz6cluuAuAs UOWK2oB5erk6tEKtCdue0n1Qax97mZ8GNL3yKpOHBXY+MwVKbVj5q9z4NBdSxASl 1uakIsqy4ZV8Ric4vdHVZyxRfJH+0rzmqphARHAopDdmK3pFpdVDDHqBWPHGq4U6 VMkwUqZX3COTsrm/+Ux9Ygb4CZalfq4btP1YVvfrexH9eGyG1DfPCAfLGCeu0xWK 6FUb5Oh2vh4F9gcI6vAjoLC6T1/V9zKmYPBsjAF6por//bsOT/YHVAMccjzjcX+m wdbZ+Hj6kt9XHK3xi/SJ =vtmX -----END PGP SIGNATURE----- --612zfeGV38IdUQknGJ724Lxcls0v44tr2--