Date: Wed, 19 Dec 2018 13:32:43 -0500 From: Allan Jude <allanjude@freebsd.org> To: freebsd-current@freebsd.org Subject: Re: The future of ZFS in FreeBSD Message-ID: <a64a41fd-f287-cf9d-cd4c-9e55c21c2f00@freebsd.org> In-Reply-To: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> References: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com> <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd>
next in thread | previous in thread | raw e-mail | index | archive | help
This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz Content-Type: multipart/mixed; boundary="QmybEsV62TKI9AojVqn4pAhaNTclfMKfA"; protected-headers="v1" From: Allan Jude <allanjude@freebsd.org> To: freebsd-current@freebsd.org Message-ID: <a64a41fd-f287-cf9d-cd4c-9e55c21c2f00@freebsd.org> Subject: Re: The future of ZFS in FreeBSD References: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com> <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> In-Reply-To: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> --QmybEsV62TKI9AojVqn4pAhaNTclfMKfA Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018-12-19 11:30, Shawn Webb wrote: > On Tue, Dec 18, 2018 at 10:49:48PM -0800, Matthew Macy wrote: >> The sources for FreeBSD's ZFS support are currently taken directly >> from Illumos with local ifdefs to support the peculiarities of FreeBSD= >> where the Solaris Portability Layer (SPL) shims fall short. FreeBSD >> has regularly pulled changes from Illumos and tried to push back any >> bug fixes and new features done in the context of FreeBSD. In the past= >> few years the vast majority of new development in ZFS has taken place >> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced= >> that they will be moving to ZoL >> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means >> that there will be little to no net new development of Illumos. While >> working through the git history of ZoL I have also discovered that >> many races and locking bugs have been fixed in ZoL and never made it >> back to Illumos and thus FreeBSD. This state of affairs has led to a >> general agreement among the stakeholders that I have spoken to that it= >> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf >> has graciously encouraged me to add FreeBSD support directly to ZoL >> https://github.com/zfsonfreebsd/ZoF so that we might all have a single= >> shared code base. >> >> A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port >> Before it can be committed some additional functionality needs to be >> added to the FreeBSD opencrypto framework. These can be found at >> https://reviews.freebsd.org/D18520 >> >> This port will provide FreeBSD users with multi modifier protection, >> project quotas, encrypted datasets, allocation classes, vectorized >> raidz, vectorized checksums, and various command line improvements. >> >> Before ZoF can be merged back in to ZoL several steps need to be taken= : >> - Integrate FreeBSD support into ZoL CI >> - Have most of the ZFS test suite passing >> - Complete additional QA testing at iX >> >> We at iX Systems need to port ZoL's EC2 CI scripts to work with >> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes. >> Being integrated in to their CI will mean that, among other things, >> most integration issues will be caught before a PR is merged upstream >> rather than many months later when it is MFVed into FreeBSD. I???m >> hoping to submit the PR to ZoL some time in January. >> >> This port will make it easy for end users on a range of releases to >> run the latest version of ZFS. Nonetheless, transitioning away from an= >> Illumos based ZFS is not likely to be entirely seamless. The >> stakeholders I???ve spoken to all agree that this is the best path >> forward but some degree of effort needs to be made to accommodate >> downstream consumers. The current plan is to import ZoF and unhook the= >> older Illumos based sources from the build on April 15th or two months= >> after iX systems QA deems ZoF stable - which ever comes later. The >> Illumos based sources will be removed some time later - but well >> before 13. This will give users a 3 month period during which both the= >> port and legacy Illumos based ZFS will be available to users. Pools >> should interoperate between ZoF and legacy provided the user does not >> enable any features available only in ZoF. We will try to accommodate >> any downstream consumers in the event that they need that date pushed >> back. We ask that any downstream consumers who are particularly >> sensitive to changes start testing the port when it is formally >> announced and report back any issues they have. I will do my best to >> ensure that this message is communicated to all those who it may >> concern. However, I can???t ensure that everyone reads these lists. Th= at >> is the responsibility of -CURRENT users. >=20 > Hey Matt, >=20 > Thank you for your detailed and informative post. It really helps > downstream consumers of FreeBSD. >=20 > I'm curious what this means for OpenZFS. I was under the impression > that OpenZFS was the upstream for all the ZFS implementations (sans > Oracle). >=20 > Thanks, >=20 The original goal of the OpenZFS project was to have a single OS agnostic repo where all of the common code lives, to try to make it easier to have all of the implementations be relatively in sync with each other, and have feature parity and import compatibility. For the last 5 years, this goal is been out of reach, because no one could justify the effort and expense of maintaining the 'one true repo' when no one would actually be using or benefiting from that repo directly= =2E Today, the OpenZFS repo is just a fork of the illumos-gate repo, but where pull requests are accepted, and where previous Delphix employees would deal with the process of trying to upstream patches to illumos. This process has not worked well recently, as things have gotten stuck waiting for 'merge advocates' in illumos. The ZoF effort will move us closer to the goal of a common repo. The general idea will be to have 'OS Dependent' and 'OS Independent' code, similar to the MI/MD split we do in other kernel code. So the upstream repo will contain OpenZFS, and directories for linux and freebsd. We will leverage the CI work that has already been done, and this will mean that all proposed changes will have to pass CI on FreeBSD as well as Linux before they are merged. Even illumos will begin pulling changes from the ZoL repo, but their timelines are much longer, and more vague than will be acceptable for FreeBSD. There is a monthly ZFS call with leaders from all of the projects (FreeBSD, illumos, ZoL, ZFS on OSX, ZFS on Windows, various vendors), where the need for this change, and some other parts of the plan are discussed: https://www.youtube.com/channel/UC0IK6Y4Go2KtRueHDiQcxow/videos The way that things have evolved over the last few years means it would be much more difficult to just import changes from ZoL. ZoL was originally far behind the OpenZFS repo, but as they were catching up, they were also fixing bugs and adding features. So there is not really a common point in the source we could start from to import ZoL commit by commit. My recent attempts to import a single ZoL feature were also fraught with issues, as the original feature was imported before the zpool import rewrite, but many improvements and bug fixes were done after the rewrite. So the code just doesn't apply cleanly to FreeBSD, since it already has the zpool import rewrite applied. Of paramount importance will be the stability and reliability of the FreeBSD implementation of ZFS. However, for the future, we are going to need to make this switch, so it is better to start sooner rather than later. The biggest thing to remember is that this is still OpenZFS, and still run by the same developers as it has been. We are just commonizing on the repo that has the most features integrated into it. --=20 Allan Jude --QmybEsV62TKI9AojVqn4pAhaNTclfMKfA-- --LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJcGo7NAAoJEBmVNT4SmAt+PcwP/07O/M31ku52rnTNzPL0La7d QqkvegszPXE5vbmZq7ObKEgd6vyXjTj8zUjfe7QAmx5RiXB3A52O4Uqy/TwOU2XW QSYE6/3X+Pa4Qj7XlqmIackYAnP88mTQjDMu8bCK+SDuEwVBT+pUbpEhgdqDIBO6 fgJD7v5uZXWztfjEKzdZslDyWqm1vYvfekVxhynNbfap212QIeDzOsqp817XO41f 6rR6Ou17jnQq0rqK0mcJPxC/msAgjr6yuqqt/4di4ov9vIMU8EHF84vCXPNWzOm4 Nwa7VguehsSEqAYtHhS60jii73uIgXDfsmKXJYe8y2JurP4lBaVfY9HN7R2q/ahi baGvsQ1gbec6xNIOzVaPryeLhRmhMBmhl9VmpKERXb+P0fR4ZRe2oxOXufJKI05i MJwN3V4bR1N5RgGb3ejj6Kmp6UostQocwU2wpSEVb4zypGsjLea73SxeXddcKNsP C5hN0+anPNnaTGb+vQVNthFttPPPLrRecwAYbiTkiYZMLY9R+wKZv3rv6r+q2kdH HWBm1TctmRQuhAFMhfBvpt0NRUClCh4iTVWwhW5mXNR7Nke/AYnpjM4GvtvqCFsD h44zJCy1p9kqgkNzHq53jJneB5i6mrBC5JpCYPmTycus7B11Qw75wA6O4kuLdgsP c97QnhC3zwdl3UXEugun =UsHM -----END PGP SIGNATURE----- --LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?a64a41fd-f287-cf9d-cd4c-9e55c21c2f00>