From owner-freebsd-current@freebsd.org Wed Dec 19 22:52:30 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 49E50134C10C; Wed, 19 Dec 2018 22:52:30 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id E199A74C53; Wed, 19 Dec 2018 22:52:29 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-io1-f43.google.com (mail-io1-f43.google.com [209.85.166.43]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id AB4795E3C; Wed, 19 Dec 2018 22:52:29 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-io1-f43.google.com with SMTP id l14so16928294ioj.5; Wed, 19 Dec 2018 14:52:29 -0800 (PST) X-Gm-Message-State: AA+aEWYcclioekmN1H96Whvq+gldXirvD0Sa+KZic86Te1jc4/tSbJwF f38HRUv2hUzT9G1fYOGysXoeeynHNyDm2acWnCc= X-Google-Smtp-Source: AFSGD/VstMykI/2JblKobGbwAbThi/E5wXJtdfND8/+aJ+XCT4RFjTq3poqgS9+ZTP6DefoC+JlTovszjnU/ji8jXcw= X-Received: by 2002:a6b:8f8d:: with SMTP id r135mr19621676iod.5.1545259949011; Wed, 19 Dec 2018 14:52:29 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Macy Date: Wed, 19 Dec 2018 14:52:17 -0800 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: The future of ZFS in FreeBSD To: Steven Hartland Cc: freebsd-current , freebsd-fs X-Rspamd-Queue-Id: E199A74C53 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.98 / 15.00]; local_wl_from(0.00)[freebsd.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.98)[-0.985,0]; NEURAL_HAM_LONG(-1.00)[-0.999,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2018 22:52:30 -0000 On Wed, Dec 19, 2018 at 14:47 Steven Hartland wrote: > Thanks for the write up most appreciated. One of the more meaty > differences is that FreeBSD ZFS still has the only merged and production > ready TRIM support so my question would be are their any plans to address > this before creating the new port as going back to a world without TRIM > support wouldn=E2=80=99t be something I=E2=80=99d look forward to. > Well, then please follow up on the request I CC'd you on a week ago asking that you engage on the deadlist based TRIM PR. That's a better forum for discussing these details than lamenting in public lists. Thanks. -M > On Wed, 19 Dec 2018 at 06:51, Matthew Macy wrote: > >> The sources for FreeBSD's ZFS support are currently taken directly >> from Illumos with local ifdefs to support the peculiarities of FreeBSD >> where the Solaris Portability Layer (SPL) shims fall short. FreeBSD >> has regularly pulled changes from Illumos and tried to push back any >> bug fixes and new features done in the context of FreeBSD. In the past >> few years the vast majority of new development in ZFS has taken place >> in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced >> that they will be moving to ZoL >> https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means >> that there will be little to no net new development of Illumos. While >> working through the git history of ZoL I have also discovered that >> many races and locking bugs have been fixed in ZoL and never made it >> back to Illumos and thus FreeBSD. This state of affairs has led to a >> general agreement among the stakeholders that I have spoken to that it >> makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf >> has graciously encouraged me to add FreeBSD support directly to ZoL >> https://github.com/zfsonfreebsd/ZoF so that we might all have a single >> shared code base. >> >> A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port >> Before it can be committed some additional functionality needs to be >> added to the FreeBSD opencrypto framework. These can be found at >> https://reviews.freebsd.org/D18520 >> >> This port will provide FreeBSD users with multi modifier protection, >> project quotas, encrypted datasets, allocation classes, vectorized >> raidz, vectorized checksums, and various command line improvements. >> >> Before ZoF can be merged back in to ZoL several steps need to be taken: >> - Integrate FreeBSD support into ZoL CI >> - Have most of the ZFS test suite passing >> - Complete additional QA testing at iX >> >> We at iX Systems need to port ZoL's EC2 CI scripts to work with >> FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes. >> Being integrated in to their CI will mean that, among other things, >> most integration issues will be caught before a PR is merged upstream >> rather than many months later when it is MFVed into FreeBSD. I=E2=80=99m >> hoping to submit the PR to ZoL some time in January. >> >> This port will make it easy for end users on a range of releases to >> run the latest version of ZFS. Nonetheless, transitioning away from an >> Illumos based ZFS is not likely to be entirely seamless. The >> stakeholders I=E2=80=99ve spoken to all agree that this is the best path >> forward but some degree of effort needs to be made to accommodate >> downstream consumers. The current plan is to import ZoF and unhook the >> older Illumos based sources from the build on April 15th or two months >> after iX systems QA deems ZoF stable - which ever comes later. The >> Illumos based sources will be removed some time later - but well >> before 13. This will give users a 3 month period during which both the >> port and legacy Illumos based ZFS will be available to users. Pools >> should interoperate between ZoF and legacy provided the user does not >> enable any features available only in ZoF. We will try to accommodate >> any downstream consumers in the event that they need that date pushed >> back. We ask that any downstream consumers who are particularly >> sensitive to changes start testing the port when it is formally >> announced and report back any issues they have. I will do my best to >> ensure that this message is communicated to all those who it may >> concern. However, I can=E2=80=99t ensure that everyone reads these lists= . That >> is the responsibility of -CURRENT users. >> >> -M >> > _______________________________________________ >> freebsd-fs@freebsd.org mailing list >> https://lists.freebsd.org/mailman/listinfo/freebsd-fs >> To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >> >