From owner-freebsd-fs@freebsd.org Fri Jan 22 02:27:06 2021 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 978004E796B for ; Fri, 22 Jan 2021 02:27:06 +0000 (UTC) (envelope-from jdelisle@gmail.com) Received: from mail-pf1-x42a.google.com (mail-pf1-x42a.google.com [IPv6:2607:f8b0:4864:20::42a]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4DMNST5SMxz3mw5; Fri, 22 Jan 2021 02:27:05 +0000 (UTC) (envelope-from jdelisle@gmail.com) Received: by mail-pf1-x42a.google.com with SMTP id o20so2747221pfu.0; Thu, 21 Jan 2021 18:27:05 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=7ZbXOBi6dpx31AfIMOmZ+aOHbdBW//QoF0F7QllFbLE=; b=N5oKebgHPjXPMSdVdFY8CkxkC8bGsozQL7S7d1it5yxjqx1z2bUjfK217PppwkKRkT ggl8tOQK3OWdtfy7KMOaF4WqRjwjBIJ+4+QBy+pQsy4qtrMn27X5vymrs9w2NJnp565J tFJq5K4O3fcL7LS+Q+lTv4H6YiNcPGNdO9wbVbGsszX/TpGPOkjbr4ye1npI9//Ksmpc MbZMNkFR+Dg7dZnoWQn1FXjMxFVpYWd5WRKI7l9AgBqxeM8uKO7npH9HHgMVs2jlUZ5b 0DCRULCa7Gp49U1Cxf+8+KqEx6LV2sp7oiOIKc4NWWvsHC+kQkkjgAbeuqxPfR37LBQh 0dGw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=7ZbXOBi6dpx31AfIMOmZ+aOHbdBW//QoF0F7QllFbLE=; b=b8TpdIk53j+QLRm6eTzuMtHgFJCeVlNvNUocDV+4lhzSWXCUvOwerbVfOpvj7fVFxj 7fja7BpI+FKiYn7qeXnGCbxLk6+utGMkzy98mWmNRbL6MWB4RtZjuBhs/2Q+TX83BpkH 7qti0xKd/SZPRQfuWMSzzox3LNqmTRZQsTn0TVg/DbjMih1vr2Uh2IeBk39fECcxGNBC a7f9gUxPk1Cj0QprCaJnZVdVNZPmYhhWba2B0r48eVdrY848BI8K1byMOcM2eDv12/SQ U+PPtoiWwwbOLiF4DijlH2q2HtO3KBZ9r2jXKV4soYki5lvA/HTK48YPrfLnHtUx2upG tkwA== X-Gm-Message-State: AOAM531vPP+BynIz1tVP/JFPOgSp4urMDmx9xjSkUT9/JVumwPPvrctT zWSvPXb65FC6jYcfiJSQQ5ECQ03BB5yDOu6c4YwY3vTrZnI= X-Google-Smtp-Source: ABdhPJxs56nEiZEdVTGQmrQ+e9nL7eSjBjvDvIKsuUFywMlvXYvFiqg12cHHcom696p/m4m2mE/b7xKUVwLrIWKVSag= X-Received: by 2002:a63:fe13:: with SMTP id p19mr2382759pgh.119.1611282423892; Thu, 21 Jan 2021 18:27:03 -0800 (PST) MIME-Version: 1.0 References: <92af08f3-342c-c98c-702f-54ee48bb089f@FreeBSD.org> <63109762-27df-7677-e71a-b70572af12c8@FreeBSD.org> In-Reply-To: <63109762-27df-7677-e71a-b70572af12c8@FreeBSD.org> From: John Delisle Date: Thu, 21 Jan 2021 20:26:51 -0600 Message-ID: Subject: Re: zpool remove not working for metadata special devices To: Andriy Gapon Cc: freebsd-fs X-Rspamd-Queue-Id: 4DMNST5SMxz3mw5 X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=N5oKebgH; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of jdelisle@gmail.com designates 2607:f8b0:4864:20::42a as permitted sender) smtp.mailfrom=jdelisle@gmail.com X-Spamd-Result: default: False [-2.20 / 15.00]; ARC_NA(0.00)[]; RBL_DBL_DONT_QUERY_IPS(0.00)[2607:f8b0:4864:20::42a:from]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; RCVD_TLS_ALL(0.00)[]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2607:f8b0:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; TO_MATCH_ENVRCPT_ALL(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000]; NEURAL_SPAM_SHORT(0.80)[0.795]; SPAMHAUS_ZRD(0.00)[2607:f8b0:4864:20::42a:from:127.0.2.255]; NEURAL_HAM_MEDIUM(-1.00)[-1.000]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2607:f8b0:4864:20::42a:from]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2607:f8b0::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-fs]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" X-Content-Filtered-By: Mailman/MimeDel 2.1.34 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Jan 2021 02:27:06 -0000 Thank you Andriy for your insight into the issue. I can confirm that after a reboot, everything works fine! Interestingly, after creating the pool and rebooting, I can add and remove special mirrors without needing to reboot in-between operations. Thanks again! Looking forward to trying this in a production setting. On Mon, Jan 18, 2021 at 7:26 AM Andriy Gapon wrote: > On 2021-01-18 14:53, Andriy Gapon wrote: > > It looks like there is an issue with recording of spa_min_ashift > > internal parameter when a special device is first added. > > This OpenZFS change explicitly addresses a couple of tangential issues > (special + ashift) but it also fixes the issue that you reported: > > > https://github.com/openzfs/zfs/commit/b3f4436d37bb72ffd924132aa4ef2d743762011f > > > -- > Andriy Gapon >