Skip site navigation (1)Skip section navigation (2)
Date:      Wed, 19 Dec 2018 11:30:33 -0500
From:      Shawn Webb <shawn.webb@hardenedbsd.org>
To:        Matthew Macy <mmacy@freebsd.org>
Cc:        freebsd-fs <freebsd-fs@freebsd.org>, freebsd-current <freebsd-current@freebsd.org>
Subject:   Re: The future of ZFS in FreeBSD
Message-ID:  <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd>
In-Reply-To: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com>
References:  <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com>

next in thread | previous in thread | raw e-mail | index | archive | help

--rlmvscpkcfr6pslq
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Tue, Dec 18, 2018 at 10:49:48PM -0800, Matthew Macy wrote:
> The sources for FreeBSD's ZFS support are currently taken directly
> from Illumos with local ifdefs to support the peculiarities of FreeBSD
> where the Solaris Portability Layer (SPL) shims fall short. FreeBSD
> has regularly pulled changes from Illumos and tried to push back any
> bug fixes and new features done in the context of FreeBSD. In the past
> few years the vast majority of new development in ZFS has taken place
> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced
> that they will be moving to ZoL
> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means
> that there will be little to no net new development of Illumos. While
> working through the git history of ZoL I have also discovered that
> many races and locking bugs have been fixed in ZoL and never made it
> back to Illumos and thus FreeBSD. This state of affairs has led to a
> general agreement among the stakeholders that I have spoken to that it
> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf
> has graciously encouraged me to add FreeBSD support directly to ZoL
> https://github.com/zfsonfreebsd/ZoF so that we might all have a single
> shared code base.
>=20
> A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port
> Before it can be committed some additional functionality needs to be
> added to the FreeBSD opencrypto framework. These can be found at
> https://reviews.freebsd.org/D18520
>=20
> This port will provide FreeBSD users with multi modifier protection,
> project quotas, encrypted datasets, allocation classes, vectorized
> raidz, vectorized checksums, and various command line improvements.
>=20
> Before ZoF can be merged back in to ZoL several steps need to be taken:
> - Integrate FreeBSD support into ZoL CI
> - Have most of the ZFS test suite passing
> - Complete additional QA testing at iX
>=20
> We at iX Systems need to port ZoL's EC2 CI scripts to work with
> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes.
> Being integrated in to their CI will mean that, among other things,
> most integration issues will be caught before a PR is merged upstream
> rather than many months later when it is MFVed into FreeBSD. I???m
> hoping to submit the PR to ZoL some time in January.
>=20
> This port will make it easy for end users on a range of releases to
> run the latest version of ZFS. Nonetheless, transitioning away from an
> Illumos based ZFS is not likely to be entirely seamless. The
> stakeholders I???ve spoken to all agree that this is the best path
> forward but some degree of effort needs to be made to accommodate
> downstream consumers. The current plan is to import ZoF and unhook the
> older Illumos based sources from the build on April 15th or two months
> after iX systems QA deems ZoF stable - which ever comes later. The
> Illumos based sources will be removed some time later - but well
> before 13. This will give users a 3 month period during which both the
> port and legacy Illumos based ZFS will be available to users. Pools
> should interoperate between ZoF and legacy provided the user does not
> enable any features available only in ZoF. We will try to accommodate
> any downstream consumers in the event that they need that date pushed
> back. We ask that any downstream consumers who are particularly
> sensitive to changes start testing the port when it is formally
> announced and report back any issues they have. I will do my best to
> ensure that this message is communicated to all those who it may
> concern. However, I can???t ensure that everyone reads these lists. That
> is the responsibility of -CURRENT users.

Hey Matt,

Thank you for your detailed and informative post. It really helps
downstream consumers of FreeBSD.

I'm curious what this means for OpenZFS. I was under the impression
that OpenZFS was the upstream for all the ZFS implementations (sans
Oracle).

Thanks,

--=20
Shawn Webb
Cofounder and Security Engineer
HardenedBSD

Tor-ified Signal:    +1 443-546-8752
Tor+XMPP+OTR:        lattera@is.a.hacker.sx
GPG Key ID:          0x6A84658F52456EEE
GPG Key Fingerprint: 2ABA B6BD EF6A F486 BE89  3D9E 6A84 658F 5245 6EEE

--rlmvscpkcfr6pslq
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAEBCAAdFiEEKrq2ve9q9Ia+iT2eaoRlj1JFbu4FAlwaciUACgkQaoRlj1JF
bu7GnxAAkKRQBJ1ypOD0J+FxS/K5fqXH+mBD2DW66/iU0EPcAsTZFPj5nhsCuhLL
znBjvCH4nnMu1vYoTwn0Yeism1ioYz6EGxfdxVfyz/X+cOv1Bf9tGiKwvm5U8TvM
AK2YIaRHL8lWLeQzMapJx5xm1PVPIo6FPyVOVZSCP0JetfJ96ahTPOOWcOozk8h6
vYurzIXnsA7IBxEy22/h3h1ZyJ4IxRFW81XAGsdQPdhWA2V2eBGka9UGisk5cgmZ
vxRnZH/8RBjDaGNnzrpZBlND7Ukyu1PZeJmdF01wInB7ruTfDmdTYeQK2VMfmHOP
feU70wyFI7AhVPQ2SbM1fUDVvwkG0kS1Q+qIX6X5EHqbOER9g0cUbphEYIvFpMQD
ICr2m2LpeJGem1HMCtZb/j+rChRJtvKytIk60QC+lXDzhRhiR9irM7pfCNiJ1RW2
xCLhjEkCNtMYVw/5KpdqmN91GNBnp7BdhzKSKSW6xiB5CECWzEv9qexQlmxT7OzS
0bsg2EdYLc4C4YkL2fw2UF2y5oYUMI9niS7n0rRZpMbD8eTK6yhAlhHEBWDvhDUT
c/ITYnduUf0yW39n9uQNQgYdauxnj8yJwK0eIQNFAIMgBxDZAgm+s0KmxAyCNg/P
c8uSzZBPFdpHCBbeSybGCBEpABn9EULervJJ8KEFBeoaoNSqMvg=
=IVGV
-----END PGP SIGNATURE-----

--rlmvscpkcfr6pslq--



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20181219163033.jwm7opiwmdhbk6p3>