Date: Tue, 25 Aug 2020 22:22:33 -0400 From: Allan Jude <allanjude@freebsd.org> To: status-updates@freebsdfoundation.org, freebsd-fs <freebsd-fs@freebsd.org>, openzfs-developer <developer@open-zfs.org> Subject: Re: ZSTD Project Weekly Status Update Message-ID: <ce721076-962a-ddf4-6886-0eafbbb418b1@freebsd.org> In-Reply-To: <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> <bebcc0bb-7590-a04b-09ae-fa04e22d27dc@freebsd.org> <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> <9d77cb73-c8e8-cca0-b4b8-28e6790268d6@freebsd.org> <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> <db71835b-9bb7-2722-fd02-194b97f1564e@freebsd.org> <e9597d9b-88e0-334f-d266-6cbbaf746855@freebsd.org> <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe Content-Type: multipart/mixed; boundary="wv7SQW4dBTHm9wc9shJIOMU1u6Rdt0du3"; protected-headers="v1" From: Allan Jude <allanjude@freebsd.org> To: status-updates@freebsdfoundation.org, freebsd-fs <freebsd-fs@freebsd.org>, openzfs-developer <developer@open-zfs.org> Message-ID: <ce721076-962a-ddf4-6886-0eafbbb418b1@freebsd.org> Subject: Re: ZSTD Project Weekly Status Update References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> <bebcc0bb-7590-a04b-09ae-fa04e22d27dc@freebsd.org> <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> <9d77cb73-c8e8-cca0-b4b8-28e6790268d6@freebsd.org> <327f4b10-9727-331e-2dc9-641dad96dd2a@freebsd.org> <db71835b-9bb7-2722-fd02-194b97f1564e@freebsd.org> <e9597d9b-88e0-334f-d266-6cbbaf746855@freebsd.org> <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> In-Reply-To: <738e1ca9-05b6-bc1f-468c-b5eee03643ab@freebsd.org> --wv7SQW4dBTHm9wc9shJIOMU1u6Rdt0du3 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable This is the tenth weekly status report on my FreeBSD Foundation sponsored project to complete the integration of ZSTD compression into OpenZFS. Late last week the main pull request was merged, and ZSTD support is now part of OpenZFS's trunk branch. Last night, OpenZFS with ZSTD was imported into FreeBSD's -current branch= =2E I am continuing to work on a number of things related to ZSTD, including future-proofing support (so upgrading ZSTD won't cause problems with features like nopwrite), and improving the integration of ZSTD into FreeBSD, including enabling support for booting from ZSTD compressed datasets, and improving the performance of ZSTD on FreeBSD. I'll also be adding some additional tests to make sure we detect any issues when we do look at updating ZSTD. Additionally, I am working on a bunch of documentation around using ZSTD in ZFS. For my benchmarking of ZSTD, I have been using a zfs recv of a stream in a file on a tmpfs, and recording how long it takes to receive and sync the data. The test data is a copy of the FreeBSD 12.1 source code, since that is easily reproducible. Does anyone have experience or a better suggestion on how to get the most consistent and repeatable results when benchmarking like this? On 2020-08-18 18:51, Allan Jude wrote: > This is the ninth weekly status report on my FreeBSD Foundation > sponsored project to complete the integration of ZSTD compression into > OpenZFS. >=20 > https://github.com/openzfs/zfs/pull/10693 - The L2ARC fixes (for when > compressed ARC is disabled) have been merged. >=20 > https://github.com/openzfs/zfs/pull/10278/ - A number of other cleanups= > and fixes for the ZSTD have been integrated and squashed, and it looks > like the completed ZSTD feature will be merged very soon. >=20 > This included a bunch of fixes for makefiles and runfiles to hook the > tests I added up to the ZFS test suite so they are run properly. >=20 > It looks like this will mean that the ZSTD feature will be included in > OpenZFS 2.0. Thanks for everyone who has tested, reviewed, or > contributed to this effort, especially those who kept it alive while I > was working on other things. >=20 > Post-merge, the remaining work is to develop future-proofing around ZST= D > so that we will be able to more seamlessly upgrade to newer releases of= > ZSTD. Recompression of the same input resulting in the same on-disk > checksum is the main concern, as without this upgrading the compression= > algorithm will break features like nop-write. >=20 > This project is sponsored by the FreeBSD Foundation. >=20 --=20 Allan Jude --wv7SQW4dBTHm9wc9shJIOMU1u6Rdt0du3-- --oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfRcdsAAoJEBmVNT4SmAt+tpoQAN1PAjBf3b9hIyhRZ0foAiSP 6Yz7KeNNwvUHb7tlYrEpKIMiYftOzqVRM1hzkn1fZvjcbET0avAXFP7tlfKGADiK VVOZJuT4QcF3FQazG7eeOG1Qe6kQNXmqqMsV7kipJtHAdGGy3pjS701K6K2wg7ka J1vLcyDkMk+w++IVQlrQwI/WocActnaraeBLqk8aj8SX0tsj4NpU9BEaSslKeiZU 873MyiY/QVfITahuKjoKXq7Hj6cBQPW5KGVZ9LBlXeYXp9usIZZ+i8XR9xHFpXa6 dSVL79o4p4nTlNghZ9wLKfkGtN+Isd7atCQkdHZK9I9Xdb6enlYw8nkpVsvheFWP vOw+cE25fTAYAaXKa7X5UTV0+Ns9H6by5/fBlblqpkE4ZJK+dERfiQCRaYdti02U blH3MbfC8vFk485HMKK2M43/IJyjNrbca+NUIpI4ry3ipxTE3MsFJYf5iZNpSzcC RlF9QJpFJWkI6ewjYxijNtw/74441H+rUSFQ00GKTlLiJUKzy+sLg4rPjRjgku1W HJ1ffF/ZROiL+sH2KaPSUXukTiNJ8DMMmLvvb1nBoJ1cjPJUIV3NbedsmyQ8Z8VK gCz1mrjASM4GfFwaWG3gW4FWGVXxREwJpU2YqlSY0DZkDjWXhq5X77F0LG5d/Eli 8vbIHl4QCJafj4+FIvSX =MofC -----END PGP SIGNATURE----- --oCsWTS9EN6xchYXf4VnVcjAQoSXZYLuGe--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ce721076-962a-ddf4-6886-0eafbbb418b1>