From owner-freebsd-fs@freebsd.org Wed Mar 27 16:06:29 2019 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id EF0321553D20 for ; Wed, 27 Mar 2019 16:06:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: from mail-vs1-xe41.google.com (mail-vs1-xe41.google.com [IPv6:2607:f8b0:4864:20::e41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 530D982629 for ; Wed, 27 Mar 2019 16:06:28 +0000 (UTC) (envelope-from matthew.ahrens@delphix.com) Received: by mail-vs1-xe41.google.com with SMTP id a190so8064150vsd.0 for ; Wed, 27 Mar 2019 09:06:28 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=delphix.com; s=google; h=mime-version:references:in-reply-to:from:date:message-id:subject:to; bh=OvaVcE7PjXmeRhEWZrYBwxgsAlMgfGKbLjjK5/DMtIE=; b=d7UaS4XS4QePX88XAOKkvB4eqjawEkOxcvYgNVsY3P0Jtrxlrl1DkxTY3GVWfow9fv 07M7V4cq+rPmTeuOwHndWqs3KQKFWqdfhpgFYzTVbHXLtEw2LruQKqZHOJxGVAAuWNZf smfy7maJTfrxlMnUlsXA6uk/XgHJcAnbl4pL4= X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to; bh=OvaVcE7PjXmeRhEWZrYBwxgsAlMgfGKbLjjK5/DMtIE=; b=ExKykaY0AvQvvIw8oOIBND1RaCPG6lKNz0E7mEojeGLoBT4nzzTMADo9LHuTVEeS44 GXzSxtiUweJqSnVkDG9sDxWcU/LHAxpIQzA4IMCt74HM3fyzckuiGEYvAJHQuHZmCWHH hxTTe63+22WKyJpT4CdSvW0DzBoju3miR5Yo5NTiHPoscthzttqLd982QLQDuBNNiKqK mO1/uGtLXdkr1CxRSFng0rLUz9JrgXrSOfxGMOpiod/VTcHaPNRMpW4B5knJWWMbHgQh JG8f6IDW92OdEvUdajB+t21c67uIC86RH4cUEeRZp/9K6Fz7DJUKcE/Gs6POYisw6X2T 5BCw== X-Gm-Message-State: APjAAAVEDIlW+jkK6GFbUvhbJjCmRkdH8QDhlMuP6yf3dBq9OAJ3TYob Y4P18jZ4WLdQeCML3OkV0ozNEa12yhkdFinSrTUcKw== X-Google-Smtp-Source: APXvYqxydGHy4u3Ev3ngv4invBzORcRR5OoXlQr8fNUm7xs1YTcg+2nAUdnaUVvvbkaYzAEwBfeu2H8He9dW4dxvFV0= X-Received: by 2002:a67:ee91:: with SMTP id n17mr22848607vsp.56.1553702787350; Wed, 27 Mar 2019 09:06:27 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Matthew Ahrens Date: Wed, 27 Mar 2019 09:06:15 -0700 Message-ID: Subject: Re: March OpenZFS Leadership Meeting To: developer , illumos-zfs , zfs-devel@list.zfsonlinux.org, zfs-devel@freebsd.org, freebsd-fs , zfs-discuss X-Rspamd-Queue-Id: 530D982629 X-Spamd-Bar: ------ Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-6.98 / 15.00]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; NEURAL_HAM_SHORT(-0.98)[-0.978,0]; REPLY(-4.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 27 Mar 2019 16:06:29 -0000 At yesterday's meeting we discussed: - log spacemap review - DRAID summit - default pool features - filesystem GUIDs. Video recording available: https://www.youtube.com/watch?v=3DtVhf4ZxfB1Y The next meeting will be April 23rd at 11AM Pacific time (2 hours earlier than usual). Detailed notes, thanks to Karyn: - Log spacemap is out for review (Serapheim) - Open PR for this work, including performance results - For people who want to port this feature to another platform, Serapheim also has a list of other changes required - DRAID summit (Richard Elling) - This work has been progressing, and it is at the point where it would be great to get people together in a conference room (physically and virtually) for a day to hash it all out. We would do this in the May/= June timeframe. - Please raise your hand (click yes in the participants window) if you want to participate. - Working on finding the venue. - Yes votes: Joyent, Mark Maybee, BrianB, Don Brady, Matt Ahrens, Tom Caputti. Please reach out to Richard if you are interested in participating. - Default pool features (Josh P) - Summary of the proposal - Portable, current, and what was available in 20XX + Tier 1 platforms - Tier 1 platforms: FreeBSD (11.X), ZoL (0.7.X), illumos-gate (from 1 year ago) - zpool status or upgrade would use the previously selected portability setting - Questions raised: - Ability to define the feature sets at runtime, so you can add new definitions on your own - Sef: Disagrees with the editable file. My own suggested addition would be the ability to query an existing pool against portability list. e.g., "How portable is this pool?" I don=E2=80=99t know = if that=E2=80=99s useful outside my head. - Tier 1 platform: specific ZoL version rather than a specific OS platform - Root pools: grub doesn=E2=80=99t support the same features as the = Tier 1 platforms. - Matt=E2=80=99s take: - User-configurable features may or may not be useful, but we can hold off and decide on that later. - ZoL: Which distros are shipping their own versions and how important are they? - Grub: We discussed this before, and we decided it was too hard and we were going to defer the conversation until later - Do we want to bias toward portability or the latest feature set? - Each distro ships different versions of ZoL: - RHEL has a separate repo for ZoL (0.7) - Debian ships 0.6.x - Matt: If you aren=E2=80=99t shipping your own version of ZoL= , you aren=E2=80=99t a Tier 1 platform? - Since portability isn=E2=80=99t turned on by default, maybe it=E2= =80=99s fine - You could say that only RHEL & SLES (+Ubuntu) are the platforms that matter since they are large, supported releases. Focus on main distros and release versions - Ubuntu LTS will be Tier 1 - Christian: Heard a rumor that Canonical is working on an installer that will include ZoL. Is that happening? What will that mean for portability? - Canonical are working on it, but we don=E2=80=99t have all the = details. - Jorgen: Considering that "upgrading" is one-way - you need to be conservative. At the moment, users want to dual boot Windows and Linux, and don't care they are missing "userobj quota" as it's not exactly going to make the pool faster. - JoshP: Boot pool is not nearly as important as data pools - No volunteers to lead the implementation effort. - Suggestion to have the discussion on the mailing list to make sure people are included. - Matt pointed out that the person leading the implementation gets to decide the details beyond what=E2=80=99s in the core requiremen= ts. (subject to code review) - Feature idea: GUIDs for filesystems that are invariant to zfs send | recv (Christian) - `guid` is publicly documented since https://github.com/zfsonlinux/zfs/pull/6102 - The `guid` property for snapshots is invariant to zfs send | recv - I use `guid` in zrepl to build diffs of the snapshot lists between =E2=80=9Cthe same=E2=80=9D filesystem on the sending & receiving side= . - However, filesystem identity is currently derived from the dataset path. The `guid` for a sent-recvd filesystem is different on the receiving side, hence not invariant. - Filesystem identity across pools / machines is required to reliably track renames & destroys of filesystems for purposes of replication. - Question: Do we want such a cross-pool `guid` for filesystems _in_ ZFS? - Previous work & ideas in that area? - How does a receiving pool handle an (unlikely) collision of GUIDs? - Do we have this problem for snapshots already? - Alternative: implement it as a user-property. - JoshC: How does -R work with this? - Matt: It is in userland, and it does work. When you send, it has the from guid and you send it to the new guid. You don=E2=80=99t necessarily know the name if it has been renamed, but you can find the snapshot that has the same guid as the from guid that the send stream is describing. You= can receive the same snapshot in one pool. The problem is solvable with the solution today. - Christian: Exhaustive search is good, but it can get tricky if there is divergence and deleted ... Wants reliable detection mecha= nism. - No clear conclusion on this, and Christian also doesn=E2=80=99t have = time to implement this on his own at this point. - Matt is open to someone implementing a guid for zrepl / other ZFS-specific functions. - For pjd=E2=80=99s need: Exclude these sub-filesystems or metada= ta. Use redacted send/receive (+ more stuff) On Tue, Mar 26, 2019 at 9:29 AM Matthew Ahrens wrote: > The next OpenZFS Leadership meeting will be held today, March 26, 1pm-2pm > Pacific time. > > Everyone is welcome to attend and participate, and we will try to keep th= e > meeting on agenda and on time. The meetings will be held online via Zoom= , > and recorded and posted to the website and YouTube after the meeting. > > The agenda for the meeting will be a discussion of the projects listed in > the agenda doc. > > For more information and details on how to attend, as well as notes and > video from the previous meeting, please see the agenda document: > > > https://docs.google.com/document/d/1w2jv2XVYFmBVvG1EGf-9A5HBVsjAYoLIFZAnW= HhV-BM/edit > > --matt >