From owner-freebsd-current@freebsd.org Wed Dec 19 22:46:59 2018 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 933F0134B9EB for ; Wed, 19 Dec 2018 22:46:59 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: from mail-lf1-x136.google.com (mail-lf1-x136.google.com [IPv6:2a00:1450:4864:20::136]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "Google Internet Authority G3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id ABAD8742DB for ; Wed, 19 Dec 2018 22:46:57 +0000 (UTC) (envelope-from steven@multiplay.co.uk) Received: by mail-lf1-x136.google.com with SMTP id i26so16288308lfc.0 for ; Wed, 19 Dec 2018 14:46:57 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=multiplay-co-uk.20150623.gappssmtp.com; s=20150623; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=j4JVyF3Ru+/bxAAy2sknhbV8VkaTEa8gvgJAlGIhRL8=; b=q7/EAzxt6gSGQzz0WsX3SXkmyEoRUFGQdq0d2NJvlwA3ZSFUYKDQC2eB4OGdonIuEW 1YJs7SKbQOiSkMRrIHyS+oAazfqM7kmiMoD64vutd0o3M9Ou9+b6SXxlc5Ssd7HZBgmr sXUiReTlaT2qtZblSelPRq0Ec01mw0XukI0Ho7/FjYmr2im+ZGePkFzDUbJHjH7xzgv/ KSQ9PNkJYU0eJObhNQaFDAn/EdIhE6JsSdF6JQa3zzk8xquqyaVF6do2Jv10RJv5EhLp NWIggirYie97EEFTjtrSphUcolnSazw0Dj9qV6P8xDiw9NZXWfYzFOaIj68MEAzmUF86 Czig== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=j4JVyF3Ru+/bxAAy2sknhbV8VkaTEa8gvgJAlGIhRL8=; b=Zekv3BbQ/Mvn+A73bpbJbSZZQQXeEgGYh/blXN/B1JasBOqyY8GWSMtYP3wYy24GHT fu3LVSK9/vD2tK3AEm//CCM3fuGsvM3DCSeap//NGPM9IPt3KpZgVmUgyDxnOW6j5rYw xO9wbJL6VZAIGd6Ay0QxirQHGQ9gey54e5kAcogkjQ56CQI0u1HXoOcgE26wp9zqTROc WIsj4uoschukC8FVmhmd88r39/BN59c3H6AdUeK8XqzIly1N0UxMyLrzc8tkFQAq7VS/ +3YSM64P2RSoYn4IxFNYD/01mWQgb9s0cVBwKlp199Bjoke0XJPAflSVGmQt4pVeG6Xl V+IQ== X-Gm-Message-State: AA+aEWYu74z4DsaAOtDtZ//tNBwdll+WtDpw0P+TV2kwFB7zOeqC+VD3 JrEKOXgjhXTVU+ASBticodVaRDGwqF1k5GQUG8hVZA== X-Google-Smtp-Source: AFSGD/WQD/RE+5tEzWyD025ciYbkObwvlK2dicWiBxl6Xcxv07Pb0jIn/BVxO9WqgLe2tXKCJLdb5c2CAiCdMjKnitI= X-Received: by 2002:a19:2755:: with SMTP id n82mr12762571lfn.94.1545259615325; Wed, 19 Dec 2018 14:46:55 -0800 (PST) MIME-Version: 1.0 References: In-Reply-To: From: Steven Hartland Date: Wed, 19 Dec 2018 22:46:44 +0000 Message-ID: Subject: Re: The future of ZFS in FreeBSD To: Matthew Macy Cc: freebsd-current , freebsd-fs X-Rspamd-Queue-Id: ABAD8742DB X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=multiplay-co-uk.20150623.gappssmtp.com header.s=20150623 header.b=q7/EAzxt; spf=pass (mx1.freebsd.org: domain of steven@multiplay.co.uk designates 2a00:1450:4864:20::136 as permitted sender) smtp.mailfrom=steven@multiplay.co.uk X-Spamd-Result: default: False [-3.72 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.997,0]; R_DKIM_ALLOW(-0.20)[multiplay-co-uk.20150623.gappssmtp.com:s=20150623]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; NEURAL_HAM_LONG(-0.99)[-0.992,0]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; DMARC_NA(0.00)[multiplay.co.uk]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[multiplay-co-uk.20150623.gappssmtp.com:+]; MX_GOOD(-0.01)[ASPMX.L.GOOGLE.COM,ALT2.ASPMX.L.GOOGLE.COM,ALT1.ASPMX.L.GOOGLE.COM,ASPMX2.GOOGLEMAIL.COM,ASPMX3.GOOGLEMAIL.COM]; RCVD_IN_DNSWL_NONE(0.00)[6.3.1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.2.0.0.4.6.8.4.0.5.4.1.0.0.a.2.list.dnswl.org : 127.0.5.0]; NEURAL_HAM_SHORT(-0.90)[-0.904,0]; IP_SCORE(-0.62)[ipnet: 2a00:1450::/32(-1.60), asn: 15169(-1.40), country: US(-0.08)]; FORGED_SENDER(0.30)[killing@multiplay.co.uk,steven@multiplay.co.uk]; MIME_TRACE(0.00)[0:+,1:+]; RCVD_TLS_LAST(0.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; FROM_NEQ_ENVFROM(0.00)[killing@multiplay.co.uk,steven@multiplay.co.uk]; RCVD_COUNT_TWO(0.00)[2] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 19 Dec 2018 22:46:59 -0000 Thanks for the write up most appreciated. One of the more meaty differences is that FreeBSD ZFS still has the only merged and production ready TRIM support so my question would be are their any plans to address this before creating the new port as going back to a world without TRIM support wouldn=E2=80=99t be something I=E2=80=99d look forward to. On Wed, 19 Dec 2018 at 06:51, Matthew Macy wrote: > The sources for FreeBSD's ZFS support are currently taken directly > from Illumos with local ifdefs to support the peculiarities of FreeBSD > where the Solaris Portability Layer (SPL) shims fall short. FreeBSD > has regularly pulled changes from Illumos and tried to push back any > bug fixes and new features done in the context of FreeBSD. In the past > few years the vast majority of new development in ZFS has taken place > in DelphixOS and zfsonlinux (ZoL). Earlier this year Delphix announced > that they will be moving to ZoL > https://www.delphix.com/blog/kickoff-future-eko-2018 This shift means > that there will be little to no net new development of Illumos. While > working through the git history of ZoL I have also discovered that > many races and locking bugs have been fixed in ZoL and never made it > back to Illumos and thus FreeBSD. This state of affairs has led to a > general agreement among the stakeholders that I have spoken to that it > makes sense to rebase FreeBSD's ZFS on ZoL. Brian Behlendorf > has graciously encouraged me to add FreeBSD support directly to ZoL > https://github.com/zfsonfreebsd/ZoF so that we might all have a single > shared code base. > > A port for ZoF can be found at https://github.com/miwi-fbsd/zof-port > Before it can be committed some additional functionality needs to be > added to the FreeBSD opencrypto framework. These can be found at > https://reviews.freebsd.org/D18520 > > This port will provide FreeBSD users with multi modifier protection, > project quotas, encrypted datasets, allocation classes, vectorized > raidz, vectorized checksums, and various command line improvements. > > Before ZoF can be merged back in to ZoL several steps need to be taken: > - Integrate FreeBSD support into ZoL CI > - Have most of the ZFS test suite passing > - Complete additional QA testing at iX > > We at iX Systems need to port ZoL's EC2 CI scripts to work with > FreeBSD and make sure that most of the ZFS Test Suite (ZTS) passes. > Being integrated in to their CI will mean that, among other things, > most integration issues will be caught before a PR is merged upstream > rather than many months later when it is MFVed into FreeBSD. I=E2=80=99m > hoping to submit the PR to ZoL some time in January. > > This port will make it easy for end users on a range of releases to > run the latest version of ZFS. Nonetheless, transitioning away from an > Illumos based ZFS is not likely to be entirely seamless. The > stakeholders I=E2=80=99ve spoken to all agree that this is the best path > forward but some degree of effort needs to be made to accommodate > downstream consumers. The current plan is to import ZoF and unhook the > older Illumos based sources from the build on April 15th or two months > after iX systems QA deems ZoF stable - which ever comes later. The > Illumos based sources will be removed some time later - but well > before 13. This will give users a 3 month period during which both the > port and legacy Illumos based ZFS will be available to users. Pools > should interoperate between ZoF and legacy provided the user does not > enable any features available only in ZoF. We will try to accommodate > any downstream consumers in the event that they need that date pushed > back. We ask that any downstream consumers who are particularly > sensitive to changes start testing the port when it is formally > announced and report back any issues they have. I will do my best to > ensure that this message is communicated to all those who it may > concern. However, I can=E2=80=99t ensure that everyone reads these lists.= That > is the responsibility of -CURRENT users. > > -M > _______________________________________________ > freebsd-fs@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-fs > To unsubscribe, send any mail to "freebsd-fs-unsubscribe@freebsd.org" >