From owner-freebsd-current@freebsd.org Wed Dec 19 18:32:49 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 180E3133FCFA for ; Wed, 19 Dec 2018 18:32:49 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from mx1.scaleengine.net (mx1.scaleengine.net [209.51.186.6]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 29C478E263 for ; Wed, 19 Dec 2018 18:32:48 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by mx1.scaleengine.net (Postfix) with ESMTPSA id 86F1E1FF82 for ; Wed, 19 Dec 2018 18:32:46 +0000 (UTC) To: freebsd-current@freebsd.org References: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> From: Allan Jude Openpgp: preference=signencrypt Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= mQINBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABtCJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+iQI/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6a5Ag0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAGJAiUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: The future of ZFS in FreeBSD Message-ID: Date: Wed, 19 Dec 2018 13:32:43 -0500 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.3.3 MIME-Version: 1.0 In-Reply-To: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz" X-Rspamd-Queue-Id: 29C478E263 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; NEURAL_HAM_SHORT(-0.99)[-0.985,0]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US]; NEURAL_HAM_LONG(-1.00)[-0.999,0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2018 18:32:49 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz Content-Type: multipart/mixed; boundary="QmybEsV62TKI9AojVqn4pAhaNTclfMKfA"; protected-headers="v1" From: Allan Jude To: freebsd-current@freebsd.org Message-ID: Subject: Re: The future of ZFS in FreeBSD References: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> In-Reply-To: <20181219163033.jwm7opiwmdhbk6p3@mutt-hbsd> --QmybEsV62TKI9AojVqn4pAhaNTclfMKfA Content-Type: text/plain; charset=windows-1252 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2018-12-19 11:30, Shawn Webb wrote: > On Tue, Dec 18, 2018 at 10:49:48PM -0800, Matthew Macy wrote: >> The sources for FreeBSD's ZFS support are currently taken directly >> from Illumos with local ifdefs to support the peculiarities of FreeBSD= >> where the Solaris Portability Layer (SPL) shims fall short. FreeBSD >> has regularly pulled changes from Illumos and tried to push back any >> bug fixes and new features done in the context of FreeBSD. In the past= >> few years the vast majority of new development in ZFS has taken place >> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced= >> that they will be moving to ZoL >> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means >> that there will be little to no net new development of Illumos. While >> working through the git history of ZoL I have also discovered that >> many races and locking bugs have been fixed in ZoL and never made it >> back to Illumos and thus FreeBSD. This state of affairs has led to a >> general agreement among the stakeholders that I have spoken to that it= >> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf >> has graciously encouraged me to add FreeBSD support directly to ZoL >> https://github.com/zfsonfreebsd/ZoF so that we might all have a single= >> shared code base. >> >> A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port >> Before it can be committed some additional functionality needs to be >> added to the FreeBSD opencrypto framework. These can be found at >> https://reviews.freebsd.org/D18520 >> >> This port will provide FreeBSD users with multi modifier protection, >> project quotas, encrypted datasets, allocation classes, vectorized >> raidz, vectorized checksums, and various command line improvements. >> >> Before ZoF can be merged back in to ZoL several steps need to be taken= : >> - Integrate FreeBSD support into ZoL CI >> - Have most of the ZFS test suite passing >> - Complete additional QA testing at iX >> >> We at iX Systems need to port ZoL's EC2 CI scripts to work with >> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes. >> Being integrated in to their CI will mean that, among other things, >> most integration issues will be caught before a PR is merged upstream >> rather than many months later when it is MFVed into FreeBSD. I???m >> hoping to submit the PR to ZoL some time in January. >> >> This port will make it easy for end users on a range of releases to >> run the latest version of ZFS. Nonetheless, transitioning away from an= >> Illumos based ZFS is not likely to be entirely seamless. The >> stakeholders I???ve spoken to all agree that this is the best path >> forward but some degree of effort needs to be made to accommodate >> downstream consumers. The current plan is to import ZoF and unhook the= >> older Illumos based sources from the build on April 15th or two months= >> after iX systems QA deems ZoF stable - which ever comes later. The >> Illumos based sources will be removed some time later - but well >> before 13. This will give users a 3 month period during which both the= >> port and legacy Illumos based ZFS will be available to users. Pools >> should interoperate between ZoF and legacy provided the user does not >> enable any features available only in ZoF. We will try to accommodate >> any downstream consumers in the event that they need that date pushed >> back. We ask that any downstream consumers who are particularly >> sensitive to changes start testing the port when it is formally >> announced and report back any issues they have. I will do my best to >> ensure that this message is communicated to all those who it may >> concern. However, I can???t ensure that everyone reads these lists. Th= at >> is the responsibility of -CURRENT users. >=20 > Hey Matt, >=20 > Thank you for your detailed and informative post. It really helps > downstream consumers of FreeBSD. >=20 > I'm curious what this means for OpenZFS. I was under the impression > that OpenZFS was the upstream for all the ZFS implementations (sans > Oracle). >=20 > Thanks, >=20 The original goal of the OpenZFS project was to have a single OS agnostic repo where all of the common code lives, to try to make it easier to have all of the implementations be relatively in sync with each other, and have feature parity and import compatibility. For the last 5 years, this goal is been out of reach, because no one could justify the effort and expense of maintaining the 'one true repo' when no one would actually be using or benefiting from that repo directly= =2E Today, the OpenZFS repo is just a fork of the illumos-gate repo, but where pull requests are accepted, and where previous Delphix employees would deal with the process of trying to upstream patches to illumos. This process has not worked well recently, as things have gotten stuck waiting for 'merge advocates' in illumos. The ZoF effort will move us closer to the goal of a common repo. The general idea will be to have 'OS Dependent' and 'OS Independent' code, similar to the MI/MD split we do in other kernel code. So the upstream repo will contain OpenZFS, and directories for linux and freebsd. We will leverage the CI work that has already been done, and this will mean that all proposed changes will have to pass CI on FreeBSD as well as Linux before they are merged. Even illumos will begin pulling changes from the ZoL repo, but their timelines are much longer, and more vague than will be acceptable for FreeBSD. There is a monthly ZFS call with leaders from all of the projects (FreeBSD, illumos, ZoL, ZFS on OSX, ZFS on Windows, various vendors), where the need for this change, and some other parts of the plan are discussed: https://www.youtube.com/channel/UC0IK6Y4Go2KtRueHDiQcxow/videos The way that things have evolved over the last few years means it would be much more difficult to just import changes from ZoL. ZoL was originally far behind the OpenZFS repo, but as they were catching up, they were also fixing bugs and adding features. So there is not really a common point in the source we could start from to import ZoL commit by commit. My recent attempts to import a single ZoL feature were also fraught with issues, as the original feature was imported before the zpool import rewrite, but many improvements and bug fixes were done after the rewrite. So the code just doesn't apply cleanly to FreeBSD, since it already has the zpool import rewrite applied. Of paramount importance will be the stability and reliability of the FreeBSD implementation of ZFS. However, for the future, we are going to need to make this switch, so it is better to start sooner rather than later. The biggest thing to remember is that this is still OpenZFS, and still run by the same developers as it has been. We are just commonizing on the repo that has the most features integrated into it. --=20 Allan Jude --QmybEsV62TKI9AojVqn4pAhaNTclfMKfA-- --LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJcGo7NAAoJEBmVNT4SmAt+PcwP/07O/M31ku52rnTNzPL0La7d QqkvegszPXE5vbmZq7ObKEgd6vyXjTj8zUjfe7QAmx5RiXB3A52O4Uqy/TwOU2XW QSYE6/3X+Pa4Qj7XlqmIackYAnP88mTQjDMu8bCK+SDuEwVBT+pUbpEhgdqDIBO6 fgJD7v5uZXWztfjEKzdZslDyWqm1vYvfekVxhynNbfap212QIeDzOsqp817XO41f 6rR6Ou17jnQq0rqK0mcJPxC/msAgjr6yuqqt/4di4ov9vIMU8EHF84vCXPNWzOm4 Nwa7VguehsSEqAYtHhS60jii73uIgXDfsmKXJYe8y2JurP4lBaVfY9HN7R2q/ahi baGvsQ1gbec6xNIOzVaPryeLhRmhMBmhl9VmpKERXb+P0fR4ZRe2oxOXufJKI05i MJwN3V4bR1N5RgGb3ejj6Kmp6UostQocwU2wpSEVb4zypGsjLea73SxeXddcKNsP C5hN0+anPNnaTGb+vQVNthFttPPPLrRecwAYbiTkiYZMLY9R+wKZv3rv6r+q2kdH HWBm1TctmRQuhAFMhfBvpt0NRUClCh4iTVWwhW5mXNR7Nke/AYnpjM4GvtvqCFsD h44zJCy1p9kqgkNzHq53jJneB5i6mrBC5JpCYPmTycus7B11Qw75wA6O4kuLdgsP c97QnhC3zwdl3UXEugun =UsHM -----END PGP SIGNATURE----- --LZXkpUdvpWGjuMdklzWDpCX07UHw6frMz--