Date: Wed, 19 Dec 2018 11:30:33 -0500 From: Shawn Webb <shawn.webb@hardenedbsd.org> To: Matthew Macy <mmacy@freebsd.org> Cc: freebsd-fs <freebsd-fs@freebsd.org>, freebsd-current <freebsd-current@freebsd.org> Subject: Re: The future of ZFS in FreeBSD Message-ID: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> In-Reply-To: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com> References: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--rlmvscpkcfr6pslq Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On Tue, Dec 18, 2018 at 10:49:48PM -0800, Matthew Macy wrote: > The sources for FreeBSD's ZFS support are currently taken directly > from Illumos with local ifdefs to support the peculiarities of FreeBSD > where the Solaris Portability Layer (SPL) shims fall short. FreeBSD > has regularly pulled changes from Illumos and tried to push back any > bug fixes and new features done in the context of FreeBSD. In the past > few years the vast majority of new development in ZFS has taken place > in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced > that they will be moving to ZoL > https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means > that there will be little to no net new development of Illumos. While > working through the git history of ZoL I have also discovered that > many races and locking bugs have been fixed in ZoL and never made it > back to Illumos and thus FreeBSD. This state of affairs has led to a > general agreement among the stakeholders that I have spoken to that it > makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf > has graciously encouraged me to add FreeBSD support directly to ZoL > https://github.com/zfsonfreebsd/ZoF so that we might all have a single > shared code base. >=20 > A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port > Before it can be committed some additional functionality needs to be > added to the FreeBSD opencrypto framework. These can be found at > https://reviews.freebsd.org/D18520 >=20 > This port will provide FreeBSD users with multi modifier protection, > project quotas, encrypted datasets, allocation classes, vectorized > raidz, vectorized checksums, and various command line improvements. >=20 > Before ZoF can be merged back in to ZoL several steps need to be taken: > - Integrate FreeBSD support into ZoL CI > - Have most of the ZFS test suite passing > - Complete additional QA testing at iX >=20 > We at iX Systems need to port ZoL's EC2 CI scripts to work with > FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes. > Being integrated in to their CI will mean that, among other things, > most integration issues will be caught before a PR is merged upstream > rather than many months later when it is MFVed into FreeBSD. I???m > hoping to submit the PR to ZoL some time in January. >=20 > This port will make it easy for end users on a range of releases to > run the latest version of ZFS. Nonetheless, transitioning away from an > Illumos based ZFS is not likely to be entirely seamless. The > stakeholders I???ve spoken to all agree that this is the best path > forward but some degree of effort needs to be made to accommodate > downstream consumers. The current plan is to import ZoF and unhook the > older Illumos based sources from the build on April 15th or two months > after iX systems QA deems ZoF stable - which ever comes later. The > Illumos based sources will be removed some time later - but well > before 13. This will give users a 3 month period during which both the > port and legacy Illumos based ZFS will be available to users. Pools > should interoperate between ZoF and legacy provided the user does not > enable any features available only in ZoF. We will try to accommodate > any downstream consumers in the event that they need that date pushed > back. We ask that any downstream consumers who are particularly > sensitive to changes start testing the port when it is formally > announced and report back any issues they have. I will do my best to > ensure that this message is communicated to all those who it may > concern. However, I can???t ensure that everyone reads these lists. That > is the responsibility of -CURRENT users. Hey Matt, Thank you for your detailed and informative post. It really helps downstream consumers of FreeBSD. I'm curious what this means for OpenZFS. I was under the impression that OpenZFS was the upstream for all the ZFS implementations (sans Oracle). Thanks, --=20 Shawn Webb Cofounder and Security Engineer HardenedBSD Tor-ified Signal: +1 443-546-8752 Tor+XMPP+OTR: lattera@is.a.hacker.sx GPG Key ID: 0x6A84658F52456EEE GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89 3D9E 6A84 658F 5245 6EEE --rlmvscpkcfr6pslq Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAlwaciUACgkQaoRlj1JF bu7GnxAAkKRQBJ1ypOD0J+FxS/K5fqXH+mBD2DW66/iU0EPcAsTZFPj5nhsCuhLL znBjvCH4nnMu1vYoTwn0Yeism1ioYz6EGxfdxVfyz/X+cOv1Bf9tGiKwvm5U8TvM AK2YIaRHL8lWLeQzMapJx5xm1PVPIo6FPyVOVZSCP0JetfJ96ahTPOOWcOozk8h6 vYurzIXnsA7IBxEy22/h3h1ZyJ4IxRFW81XAGsdQPdhWA2V2eBGka9UGisk5cgmZ vxRnZH/8RBjDaGNnzrpZBlND7Ukyu1PZeJmdF01wInB7ruTfDmdTYeQK2VMfmHOP feU70wyFI7AhVPQ2SbM1fUDVvwkG0kS1Q+qIX6X5EHqbOER9g0cUbphEYIvFpMQD ICr2m2LpeJGem1HMCtZb/j+rChRJtvKytIk60QC+lXDzhRhiR9irM7pfCNiJ1RW2 xCLhjEkCNtMYVw/5KpdqmN91GNBnp7BdhzKSKSW6xiB5CECWzEv9qexQlmxT7OzS 0bsg2EdYLc4C4YkL2fw2UF2y5oYUMI9niS7n0rRZpMbD8eTK6yhAlhHEBWDvhDUT c/ITYnduUf0yW39n9uQNQgYdauxnj8yJwK0eIQNFAIMgBxDZAgm+s0KmxAyCNg/P c8uSzZBPFdpHCBbeSybGCBEpABn9EULervJJ8KEFBeoaoNSqMvg= =IVGV -----END PGP SIGNATURE----- --rlmvscpkcfr6pslq--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20181219163033.jwm7opiwmdhbk6p3>