Date: Thu, 20 Dec 2018 11:03:59 +0000 From: Bob Bishop <rb@gid.co.uk> To: Matthew Macy <mmacy@freebsd.org> Cc: Steven Hartland <killing@multiplay.co.uk>, freebsd-current <freebsd-current@freebsd.org>, freebsd-fs <freebsd-fs@freebsd.org> Subject: Re: The future of ZFS in FreeBSD Message-ID: <B527B90E-9C5E-4DC5-8DE5-35A85221BA64@gid.co.uk> In-Reply-To: <CAPrugNpU2nnyuxetwvgeHXVNKRiDMk3uO-0OSBZ5nK55PKpY8A@mail.gmail.com> References: <CAPrugNriggEMMnLTZtf6xNQNYajBYNMnGdN96-ejDYQonoOhgw@mail.gmail.com> <CAHEMsqbhdcy1GKznVhxKAqCabYHvm_G9oxQR6eQrJ372215E0w@mail.gmail.com> <CAPrugNpr1YCo%2Br%2BCCts-uEfdU-nOF6-g15eqODJa_nKia_PwWw@mail.gmail.com> <CAHEMsqaJERWM_HpD4ru8Jj6QhVEb_CoOotzH1oPuYa95AoqWeA@mail.gmail.com> <CAPrugNpU2nnyuxetwvgeHXVNKRiDMk3uO-0OSBZ5nK55PKpY8A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Hi, > On 19 Dec 2018, at 23:16, Matthew Macy <mmacy@freebsd.org> wrote: >=20 > On Wed, Dec 19, 2018 at 15:11 Steven Hartland = <killing@multiplay.co.uk> > wrote: >=20 >> Sorry been off for a few weeks so must have missed that, please do = prod me >> on again if you don=E2=80=99t see any response to anything not just = this. Like many >> others I get so may emails across so many lists it=E2=80=99s more = than likely I >> just missed it. >>=20 >> That said would you say that with the right support we can make = progress >> on the this prior to the port? I have to ask as the alternative = version has >> been on the cusp for many years now so it=E2=80=99s feels more like a = distant >> memory than something that may happen, no disrespect to anyone = involved, as >> I know all too well how hard it can be to get something like this = over the >> line, especially when people have competing priorities. >>=20 >=20 > I am hoping that it's sufficiently important to FreeBSD ZFS developers = that > they'll give the PR the attention it needs so that it can be merged = before > summer. My understanding is that it's mostly suffered from neglect. = TRIM is > most important to FreeBSD and it already had its own implementation. >=20 > https://github.com/zfsonlinux/zfs/pull/5925 Please correct me if I=E2=80=99m wrong but this looks a lot less mature = than FreeBSD=E2=80=99s existing TRIM support for ZFS which we=E2=80=99ve = had in production for six years. What is the rationale here? I=E2=80=99m concerned that it looks like an = opportunity for mighty regressions. > I forwarded you the private communication again as well. >=20 > -M >=20 >=20 >>=20 >> On Wed, 19 Dec 2018 at 22:52, Matthew Macy <mmacy@freebsd.org> wrote: >>=20 >>>=20 >>>=20 >>> On Wed, Dec 19, 2018 at 14:47 Steven Hartland = <killing@multiplay.co.uk> >>> wrote: >>>=20 >>>> Thanks for the write up most appreciated. One of the more meaty >>>> differences is that FreeBSD ZFS still has the only merged and = production >>>> ready TRIM support so my question would be are their any plans to = address >>>> this before creating the new port as going back to a world without = TRIM >>>> support wouldn=E2=80=99t be something I=E2=80=99d look forward to. >>>>=20 >>>=20 >>> Well, then please follow up on the request I CC'd you on a week ago >>> asking that you engage on the deadlist based TRIM PR. That's a = better >>> forum for discussing these details than lamenting in public lists. >>>=20 >>> Thanks. >>>=20 >>> -M >>>=20 >>>=20 >>>=20 >>>=20 >>>> On Wed, 19 Dec 2018 at 06:51, Matthew Macy <mmacy@freebsd.org> = wrote: >>>>=20 >>>>> The sources for FreeBSD's ZFS support are currently taken directly >>>>> from Illumos with local ifdefs to support the peculiarities of = FreeBSD >>>>> where the Solaris Portability Layer (SPL) shims fall short. = FreeBSD >>>>> has regularly pulled changes from Illumos and tried to push back = any >>>>> bug fixes and new features done in the context of FreeBSD. In the = past >>>>> few years the vast majority of new development in ZFS has taken = place >>>>> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix = announced >>>>> that they will be moving to ZoL >>>>> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift = means >>>>> that there will be little to no net new development of Illumos. = While >>>>> working through the git history of ZoL I have also discovered that >>>>> many races and locking bugs have been fixed in ZoL and never made = it >>>>> back to Illumos and thus FreeBSD. This state of affairs has led to = a >>>>> general agreement among the stakeholders that I have spoken to = that it >>>>> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf >>>>> has graciously encouraged me to add FreeBSD support directly to = ZoL >>>>> https://github.com/zfsonfreebsd/ZoF so that we might all have a = single >>>>> shared code base. >>>>>=20 >>>>> A port for ZoF can be found at = https://github.com/miwi-fbsd/zof-port >>>>> Before it can be committed some additional functionality needs to = be >>>>> added to the FreeBSD opencrypto framework. These can be found at >>>>> https://reviews.freebsd.org/D18520 >>>>>=20 >>>>> This port will provide FreeBSD users with multi modifier = protection, >>>>> project quotas, encrypted datasets, allocation classes, vectorized >>>>> raidz, vectorized checksums, and various command line = improvements. >>>>>=20 >>>>> Before ZoF can be merged back in to ZoL several steps need to be = taken: >>>>> - Integrate FreeBSD support into ZoL CI >>>>> - Have most of the ZFS test suite passing >>>>> - Complete additional QA testing at iX >>>>>=20 >>>>> We at iX Systems need to port ZoL's EC2 CI scripts to work with >>>>> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) = passes. >>>>> Being integrated in to their CI will mean that, among other = things, >>>>> most integration issues will be caught before a PR is merged = upstream >>>>> rather than many months later when it is MFVed into FreeBSD. I=E2=80= =99m >>>>> hoping to submit the PR to ZoL some time in January. >>>>>=20 >>>>> This port will make it easy for end users on a range of releases = to >>>>> run the latest version of ZFS. Nonetheless, transitioning away = from an >>>>> Illumos based ZFS is not likely to be entirely seamless. The >>>>> stakeholders I=E2=80=99ve spoken to all agree that this is the = best path >>>>> forward but some degree of effort needs to be made to accommodate >>>>> downstream consumers. The current plan is to import ZoF and unhook = the >>>>> older Illumos based sources from the build on April 15th or two = months >>>>> after iX systems QA deems ZoF stable - which ever comes later. The >>>>> Illumos based sources will be removed some time later - but well >>>>> before 13. This will give users a 3 month period during which both = the >>>>> port and legacy Illumos based ZFS will be available to users. = Pools >>>>> should interoperate between ZoF and legacy provided the user does = not >>>>> enable any features available only in ZoF. We will try to = accommodate >>>>> any downstream consumers in the event that they need that date = pushed >>>>> back. We ask that any downstream consumers who are particularly >>>>> sensitive to changes start testing the port when it is formally >>>>> announced and report back any issues they have. I will do my best = to >>>>> ensure that this message is communicated to all those who it may >>>>> concern. However, I can=E2=80=99t ensure that everyone reads these = lists. That >>>>> is the responsibility of -CURRENT users. >>>>>=20 >>>>> -M >>>>>=20 >>>> _______________________________________________ >>>>> freebsd-fs@freebsd.org mailing list >>>>> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >>>>> To unsubscribe, send any mail to = "freebsd-fs-unsubscribe@freebsd.org" >>>>>=20 >>>>=20 > _______________________________________________ > freebsd-current@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" -- Bob Bishop rb@gid.co.uk
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?B527B90E-9C5E-4DC5-8DE5-35A85221BA64>