Date: Fri, 13 Dec 2024 21:47:53 +0000 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Ed Maste <emaste@freebsd.org> Cc: FreeBSD Current <freebsd-current@freebsd.org> Subject: Re: Switching release media dist sets to .tzst (tar + zstd)? Message-ID: <iqguuu433aybga26gcvssmenxk24a6mhfkk5amxfbz62fqz2ic@4ddtkar7zvci> In-Reply-To: <CAPyFy2AXee_Y2xei176VAcW6peC0x=waxfPap%2B3uGG2o%2Bynv-A@mail.gmail.com> References: <CAPyFy2AXee_Y2xei176VAcW6peC0x=waxfPap%2B3uGG2o%2Bynv-A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--jruhgstu4ull3b45 Content-Type: text/plain; protected-headers=v1; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable Subject: Re: Switching release media dist sets to .tzst (tar + zstd)? MIME-Version: 1.0 Hey Ed, Thanks for providing the opportunity to discuss this before landing it. On Fri, Dec 13, 2024 at 04:15:45PM -0500, Ed Maste wrote: > I have been reviewing parts of the release artifact build process, > including ISO and memstick images, and came across the distribution > sets (e.g., base.txz, src.txz) used by the installer to populate new > file systems. I=E2=80=99d like to discuss switching these to .tzst (tar + > zstd) compression. >=20 > While I haven=E2=80=99t yet conducted detailed benchmarks comparing zstd = and > xz specifically for this use case, here are some initial > considerations: >=20 > Pros of zstd: > - Faster compression and decompression speeds. > - Aligns with the compression method used for FreeBSD packages. >=20 > Cons of zstd: > - Somewhat larger compressed file sizes. > - Requires updates to tools that interact with distribution sets. > - May have limited availability on some other operating systems (?). The tool for updating HardenedBSD installs (and the tool used to build the update artifacts) would be impacted. It wouldn't be too difficult to update the tools (hbsd-update and hbsd-update-build). However, if the switch zstd is not done at the same time for all supported branches (main and stable/14), we would need to have hbsd-update reference different archives between different branches--zstd for main and xz for stable/14. I would prefer not to have to include branch-specific code in a generic system updater utility. >=20 > I have a review open to demonstrate the extent of the change in the > build system & installer: https://reviews.freebsd.org/D48042 One thought might be to make the choice of compression method dynamic. Folks could then choose what makes sense for them. FreeBSD could make the switch to zstd while downstreams could still use xz (should they so choose.) HardenedBSD would likely stay on xz until it makes sense to follow its upstream. >=20 > It might be that this is not worth pursuing, as dist sets will most > likely go away with the migration to pkgbase, but I would like to > discuss and make an explicit decision. We can separately consider > compression on the release media images themselves. >=20 > Feedback Requested: >=20 > Is there support for this idea? Are there objections to pursuing this? > Are there other factors I should consider, especially compatibility conce= rns? For reference, hbsd-update can be found at [1] and hbsd-update-build can be found at [2]. [1]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/blob/hardened/current= /master/usr.sbin/hbsd-update/hbsd-update?ref_type=3Dheads [2]: https://git.hardenedbsd.org/hardenedbsd/HardenedBSD/-/blob/hardened/current= /master/usr.sbin/hbsd-update/hbsd-update-build?ref_type=3Dheads Thanks, --=20 Shawn Webb Cofounder / Security Engineer HardenedBSD Tor-ified Signal: +1 303-901-1600 / shawn_webb_opsec.50 https://git.hardenedbsd.org/hardenedbsd/pubkeys/-/raw/master/Shawn_Webb/03A= 4CBEBB82EA5A67D9F3853FF2E67A277F8E1FA.pub.asc --jruhgstu4ull3b45 Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEA6TL67gupaZ9nzhT/y5nonf44foFAmdcq4EACgkQ/y5nonf4 4fps4hAAgkCULKlwU/+oOoN0qQOzTYrYzyQEizkYR3ZWCiKMfjgqgEeCG9ihhv1+ tHqkRCaO5rJDiFJviHG6UfhCaZvD2UhpAnDHwJu2ZPXRi746EU5VCHQndk+y19pL eeoZelqqb0KCDIfHZw+PNVozhec57ngP2VjCvRXtTjdXHNYufTkyIXe+R8VLmyEX 2Qh54Yy9cWKXrfcQNiCgJYWg2ZPZKghrZDEavTD/ss0Dwh3eETfwysb6x1YoxJfT XClYfNZvqng9TLi1PRabSSRinfw6sJJKTupN9kw1KHAR5CBxlf0cwDMp1d7mSQMt 3b4hAeVjLRtlUqqT/tnu1AohpP3XuIUWVyvLiLhWYinHcusm+QZJ5Z7oAMfIZnga NIsqeITbaHt4vMUxZ7aQht8Co4ZNDuI/KvZzmOAFIEyfvSI7+GoumwpdJAr8gkzI pYMEfbHaFCMGjrDhtkj6pyYFfvCiU9eOv0e6lItqmBKEgSUTHE2C6WayzsTlkp5A Y5/GiJfVycz522GFwoLXyzeKIn4Cojrd3vVQMihwTzsTaSltg8pl0KD+EAw7Ge2i Q2aUCa0jVABvD+5TQde+QOlLZpSYTv4f/0g8EkxuGO1EZNnulXp5udbWT3HVoLeX jggbK8y/9OGfL4gr0ujxp1gL+HSTvd19tHbf/dbpLKExPdkT2VU= =iIAH -----END PGP SIGNATURE----- --jruhgstu4ull3b45--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?iqguuu433aybga26gcvssmenxk24a6mhfkk5amxfbz62fqz2ic>