From owner-freebsd-fs@freebsd.org Mon Jul 13 10:02:36 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 85D7735ED02 for ; Mon, 13 Jul 2020 10:02:36 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B4zj75PrPz3SC8; Mon, 13 Jul 2020 10:02:35 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (m.cksoft.de [IPv6:2001:67c:24f8:2003::25:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id D5B8C1EA9C3; Mon, 13 Jul 2020 12:02:25 +0200 (CEST) Received: from amavisfra1.cksoft.de (unknown [IPv6:2001:67c:24f8:2003::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 7709663027; Mon, 13 Jul 2020 12:02:25 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.35.42]) by amavisfra1.cksoft.de (amavisfra1.cksoft.de [192.168.35.41]) (amavisd-new, port 10051) with ESMTP id QO58x7N4yS80; Mon, 13 Jul 2020 12:02:22 +0200 (CEST) Received: from nocfra1.cksoft.de (nocfra1.cksoft.de [IPv6:2001:67c:24f8:2001::53:1]) by m.cksoft.de (Postfix) with ESMTP id 86BE563026; Mon, 13 Jul 2020 12:02:24 +0200 (CEST) Received: by nocfra1.cksoft.de (Postfix, from userid 1000) id 7B31613E9E; Mon, 13 Jul 2020 12:02:24 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by nocfra1.cksoft.de (Postfix) with ESMTP id 76A5113D6F; Mon, 13 Jul 2020 12:02:24 +0200 (CEST) Date: Mon, 13 Jul 2020 12:02:24 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@nocfra1.cksoft.de Reply-To: Christian Kratzer To: Allan Jude cc: freebsd-fs@freebsd.org Subject: Re: gptzfsboot targeting wrong vdev In-Reply-To: <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> Message-ID: References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> User-Agent: Alpine 2.22 (BSF 395 2020-01-19) X-NCC-RegID: de.cksoft X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 X-Rspamd-Queue-Id: 4B4zj75PrPz3SC8 X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of ck-lists@cksoft.de designates 2001:67c:24f8:1::25:1 as permitted sender) smtp.mailfrom=ck-lists@cksoft.de X-Spamd-Result: default: False [-1.33 / 15.00]; HAS_REPLYTO(0.00)[ck@cksoft.de]; RCVD_VIA_SMTP_AUTH(0.00)[]; RCVD_COUNT_FIVE(0.00)[6]; ARC_NA(0.00)[]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; REPLYTO_DN_EQ_FROM_DN(0.00)[]; MIME_GOOD(-0.10)[multipart/mixed,text/plain]; REPLYTO_DOM_EQ_FROM_DOM(0.00)[]; DMARC_NA(0.00)[cksoft.de]; R_SPF_ALLOW(-0.20)[+mx]; NEURAL_HAM_LONG(-0.98)[-0.978]; NEURAL_HAM_MEDIUM(-0.93)[-0.925]; NEURAL_HAM_SHORT(-0.12)[-0.122]; CTYPE_MIXED_BOGUS(1.00)[]; RCPT_COUNT_TWO(0.00)[2]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+]; ASN(0.00)[asn:57407, ipnet:2001:67c:24f8::/48, country:DE]; RCVD_TLS_LAST(0.00)[] Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8BIT X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 10:02:36 -0000 Hi, On Tue, 7 Jul 2020, Allan Jude wrote: > On 2020-07-07 02:21, Andriy Gapon wrote: >> On 06/07/2020 16:10, Christian Kratzer wrote: >>> When booting from ada0 I get following: >>> >>>     ZFS: i/o error - all block copies unavailable >>>     ZFS: can't read MOS of pool zp1 >>>     gptzfsboot: failed to mount default pool zp1 >>> >>>     FreeBSD/x86 boot >>> > > So, just to be clear, at this point you have not loaded the boot loader > yet. You are in the bootstrap (gptzfsboot), and it is unable to load the > loader. Sorry for the delay in answering. I got around to some testing again. Thanks for clarifying this. It does make a lot more sense now. > I think it just looks at the first 'freebsd-zfs' type'd partition. > However, if zp1 is GELI encrypted, it shouldn't be able to even tell the > name of the pool. That is what I have also been confused about. It should not have access to the zpool.cache and and should have no way of seeing zp1 which is in geli encrypted da0p1 and da1p2. The loader.conf contains the keys to those which come visbile when the kernel starts. So I was pretty confused on where it was getting the name zp1 from which to my undestanding should not be visible anywhere. > You might try changing the partition type of the boots you are not > booting from, to 'freebsd-vinum' or something other than 'freebsd-zfs' > so that gptzfsboot only sees 1 'freebsd-zfs' to boot from. Before trying that I tried setting the bootme property on the root partitions on ada0 and ada1. That did not help either so I proceeded to set all but the zroot partition type to freebsd-vinum root@zfs1:/home/ck # gpart show => 40 7814037088 da0 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da1 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da2 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da3 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da4 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da5 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da6 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 7814037088 da7 GPT (3.6T) 40 4056 - free - (2.0M) 4096 7814029312 1 freebsd-vinum (3.6T) 7814033408 3720 - free - (1.8M) => 40 468862048 ada0 GPT (224G) 40 1024 1 freebsd-boot (512K) 1064 134217728 2 freebsd-swap (64G) 134218792 33554432 3 freebsd-vinum (16G) 167773224 33554432 4 freebsd-vinum (16G) 201327656 267534424 5 freebsd-zfs [bootme] (128G) 468862080 8 - free - (4.0K) => 40 468862048 ada1 GPT (224G) 40 1024 1 freebsd-boot (512K) 1064 134217728 2 freebsd-swap (64G) 134218792 33554432 3 freebsd-vinum (16G) 167773224 33554432 4 freebsd-vinum (16G) 201327656 267534424 5 freebsd-zfs [bootme] (128G) 468862080 8 - free - (4.0K) That helped. The system now finally boots from the mirror on ada0p5 and ada1p5. Unclear why setting the bootme property did not help on its own. Thanks Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-fs@freebsd.org Mon Jul 13 17:27:20 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2E90C36B038; Mon, 13 Jul 2020 17:27:20 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B59ZJ0TtSz4G6Z; Mon, 13 Jul 2020 17:27:20 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: from mail-lf1-f41.google.com (mail-lf1-f41.google.com [209.85.167.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: mmacy) by smtp.freebsd.org (Postfix) with ESMTPSA id E14BB2FC09; Mon, 13 Jul 2020 17:27:19 +0000 (UTC) (envelope-from mmacy@freebsd.org) Received: by mail-lf1-f41.google.com with SMTP id g139so9540624lfd.10; Mon, 13 Jul 2020 10:27:19 -0700 (PDT) X-Gm-Message-State: AOAM530CBzn5362d8ZkxsvvELYsKWsBX50PZOGery4Z1yoPOYy3iPums r8PZV4bAbh+nVKWCVFSIc5Gxd/2qIfFZmtxim7g= X-Google-Smtp-Source: ABdhPJx27xPiexmBFm2IppULhgxJHDxbmT+4h4zYmbzee1ZEKh1k00d4Va3rPc4oxES60biSZQaQlJa/0PKLXz5muvM= X-Received: by 2002:a19:a8c:: with SMTP id 134mr139172lfk.128.1594661238368; Mon, 13 Jul 2020 10:27:18 -0700 (PDT) MIME-Version: 1.0 From: Matthew Macy Date: Mon, 13 Jul 2020 10:27:07 -0700 X-Gmail-Original-Message-ID: Message-ID: Subject: CFT for vendor openzfs - week 2 reminder To: freebsd-current , freebsd-fs , freebsd-hackers@freebsd.org Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 17:27:20 -0000 On Wednesday, July 8th I issued the initial call for testing for the update to HEAD to vendored openzfs. We'd like to give users roughly a month to test before merging. The current *tentative* merge date is August 10th. I hope it's not terribly controversial to point out that it really rests with users of non amd64 platforms to test to avoid any unpleasant surprises the next time they update their trees following the merge. ========================================================== NB: Do NOT zpool upgrade unless you are willing to live without the ability to ever rollback to the legacy zfs kmod. Checkout updated HEAD: % git clone https://github.com/mattmacy/networking.git -b projects/openzfs_vendor freebsd Checkout updated openzfs in to sys/contrib: % git clone https://github.com/zfsonfreebsd/ZoF.git -b projects/openzfs_vendor freebsd/sys/contrib/openzfs Build world and kernel with whatever your usual configuration is. Where possible the openzfs kmod is backward compatible with the cmd utils in HEAD so common operations work with existing tools and the new kmod. In the projects/openzfs_vendor branch of ZoF ozfs libraries are backward compatible with the zfs kmod in HEAD. Although ideally one would test this in a separate boot environment, the interoperability should allow one to rollback without too much difficulty. Thanks in advance for your time. -M From owner-freebsd-fs@freebsd.org Mon Jul 13 17:33:24 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 992F736B8DE; Mon, 13 Jul 2020 17:33:24 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from smtp.freebsd.org (smtp.freebsd.org [96.47.72.83]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B59jJ3cfSz4Gtp; Mon, 13 Jul 2020 17:33:24 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: from mail-vs1-f44.google.com (mail-vs1-f44.google.com [209.85.217.44]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) (Authenticated sender: kevans) by smtp.freebsd.org (Postfix) with ESMTPSA id 5A2D52FC9B; Mon, 13 Jul 2020 17:33:24 +0000 (UTC) (envelope-from kevans@freebsd.org) Received: by mail-vs1-f44.google.com with SMTP id q15so7025564vso.9; Mon, 13 Jul 2020 10:33:24 -0700 (PDT) X-Gm-Message-State: AOAM5306R1o2NxPKOb91irwSUY966oxcqPWslD1LZMpN04EmTXZjlaCp YD/FHKAVYlqhov/cjbR+7yHF6HoLeNmrv9oWSuE= X-Google-Smtp-Source: ABdhPJxLP+ZFemGV2UDvUVqZph6aaR1h0y2+4eQBMZda/iTy0C7aXY85dvUWuPef+Co/tB6BIKkmK6n0QeQRd+VSg+g= X-Received: by 2002:a67:8a4a:: with SMTP id m71mr376954vsd.59.1594661603722; Mon, 13 Jul 2020 10:33:23 -0700 (PDT) MIME-Version: 1.0 References: In-Reply-To: From: Kyle Evans Date: Mon, 13 Jul 2020 12:33:11 -0500 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: CFT for vendor openzfs - week 2 reminder To: Matthew Macy Cc: freebsd-current , freebsd-fs , "freebsd-hackers@freebsd.org" Content-Type: text/plain; charset="UTF-8" X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 17:33:24 -0000 On Mon, Jul 13, 2020 at 12:27 PM Matthew Macy wrote: > > On Wednesday, July 8th I issued the initial call for testing for the > update to HEAD to vendored openzfs. We'd like to give users roughly a > month to test before merging. The current *tentative* merge date is > August 10th. I hope it's not terribly controversial to point out that > it really rests with users of non amd64 platforms to test to avoid any > unpleasant surprises the next time they update their trees following > the merge. > I've had no problems with this on a couple amd64 systems; I did note that my loader.conf's needed a good s/vfs.zfs.arc_max/vfs.zfs.arc.max/ but I'm told a compat sysctl is on the TODO list to ease the transition. Thanks, Kyle Evans From owner-freebsd-fs@freebsd.org Mon Jul 13 17:36:48 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id A802D36BA70 for ; Mon, 13 Jul 2020 17:36:48 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B59nD3jZ8z4Hcv for ; Mon, 13 Jul 2020 17:36:48 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id AD3E11FE7B; Mon, 13 Jul 2020 17:36:47 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net AD3E11FE7B Subject: Re: gptzfsboot targeting wrong vdev To: Christian Kratzer Cc: freebsd-fs@freebsd.org References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> From: Allan Jude Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Message-ID: Date: Mon, 13 Jul 2020 13:36:46 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P" X-Rspamd-Queue-Id: 4B59nD3jZ8z4Hcv X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; local_wl_from(0.00)[freebsd.org]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 17:36:48 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P Content-Type: multipart/mixed; boundary="ADmQm5kpvKza7Iik7dVjnz5Jx8pwakhwM"; protected-headers="v1" From: Allan Jude To: Christian Kratzer Cc: freebsd-fs@freebsd.org Message-ID: Subject: Re: gptzfsboot targeting wrong vdev References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> In-Reply-To: --ADmQm5kpvKza7Iik7dVjnz5Jx8pwakhwM Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2020-07-13 06:02, Christian Kratzer wrote: > Hi, >=20 > On Tue, 7 Jul 2020, Allan Jude wrote: >=20 >> On 2020-07-07 02:21, Andriy Gapon wrote: >>> On 06/07/2020 16:10, Christian Kratzer wrote: > >>>> When booting from ada0 I get following: >>>> >>>> =C2=A0=C2=A0=C2=A0=C2=A0ZFS: i/o error - all block copies unavailabl= e >>>> =C2=A0=C2=A0=C2=A0=C2=A0ZFS: can't read MOS of pool zp1 >>>> =C2=A0=C2=A0=C2=A0=C2=A0gptzfsboot: failed to mount default pool zp1= >>>> >>>> =C2=A0=C2=A0=C2=A0=C2=A0FreeBSD/x86 boot >>>> >> >> So, just to be clear, at this point you have not loaded the boot loade= r >> yet. You are in the bootstrap (gptzfsboot), and it is unable to load t= he >> loader. >=20 >=20 > Sorry for the delay in answering.=C2=A0 I got around to some testing ag= ain. >=20 > Thanks for clarifying this. It does make a lot more sense now. >=20 >=20 >> I think it just looks at the first 'freebsd-zfs' type'd partition. >> However, if zp1 is GELI encrypted, it shouldn't be able to even tell t= he >> name of the pool. >=20 > That is what I have also been confused about.=C2=A0 It should not have = access > to the zpool.cache and and should have no way of seeing zp1 which is > in geli encrypted da0p1 and da1p2. >=20 > The loader.conf contains the keys to those which come visbile when the > kernel starts. >=20 > So I was pretty confused on where it was getting the name zp1 from whic= h > to my undestanding should not be visible anywhere. >=20 >> You might try changing the partition type of the boots you are not >> booting from, to 'freebsd-vinum' or something other than 'freebsd-zfs'= >> so that gptzfsboot only sees 1 'freebsd-zfs' to boot from. >=20 >=20 > Before trying that I tried setting the bootme property on the root > partitions > on ada0 and ada1. >=20 > That did not help either so I proceeded to set all but the zroot > partition type to freebsd-vinum >=20 > =C2=A0=C2=A0=C2=A0=C2=A0root@zfs1:/home/ck # gpart show > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da0=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da1=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da2=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da3=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da4=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da5=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da6=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 40=C2=A0 7814037088=C2=A0 da7=C2=A0 GPT=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0 4056=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - fr= ee -=C2=A0 (2.0M) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 4096=C2=A0 7814029312=C2=A0=C2= =A0=C2=A0 1=C2=A0 freebsd-vinum=C2=A0 (3.6T) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 7814033408=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0 3720=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (1.8= M) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0= 468862048=C2=A0 ada0=C2=A0 GPT=C2=A0 (224G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 1024=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (= 512K) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1064=C2=A0= 134217728=C2=A0=C2=A0=C2=A0=C2=A0 2=C2=A0 freebsd-swap=C2=A0 (64G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 134218792=C2=A0=C2=A0 33554432=C2=A0=C2=A0= =C2=A0=C2=A0 3=C2=A0 freebsd-vinum=C2=A0 (16G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 167773224=C2=A0=C2=A0 33554432=C2=A0=C2=A0= =C2=A0=C2=A0 4=C2=A0 freebsd-vinum=C2=A0 (16G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 201327656=C2=A0 267534424=C2=A0=C2=A0=C2= =A0=C2=A0 5=C2=A0 freebsd-zfs=C2=A0 [bootme]=C2=A0 (128G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 468862080=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free = -=C2=A0 (4.0K) >=20 > =C2=A0=C2=A0=C2=A0=C2=A0=3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0= 468862048=C2=A0 ada1=C2=A0 GPT=C2=A0 (224G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 1024=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (= 512K) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1064=C2=A0= 134217728=C2=A0=C2=A0=C2=A0=C2=A0 2=C2=A0 freebsd-swap=C2=A0 (64G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 134218792=C2=A0=C2=A0 33554432=C2=A0=C2=A0= =C2=A0=C2=A0 3=C2=A0 freebsd-vinum=C2=A0 (16G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 167773224=C2=A0=C2=A0 33554432=C2=A0=C2=A0= =C2=A0=C2=A0 4=C2=A0 freebsd-vinum=C2=A0 (16G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 201327656=C2=A0 267534424=C2=A0=C2=A0=C2= =A0=C2=A0 5=C2=A0 freebsd-zfs=C2=A0 [bootme]=C2=A0 (128G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 468862080=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free = -=C2=A0 (4.0K) >=20 > That helped.=C2=A0 The system now finally boots from the mirror on ada0= p5 and > ada1p5. >=20 >=20 > Unclear why setting the bootme property did not help on its own. >=20 >=20 > Thanks Christian >=20 bootme is only used by gptboot, for UFS. You might want to example your disks with 'zdb -l /dev/ada0' (for each disk, and each partition). It seems there is something not encrypted somewhere. It may be an old label from a pool that has been destroyed, and a new pool with the same name was later created or something. --=20 Allan Jude --ADmQm5kpvKza7Iik7dVjnz5Jx8pwakhwM-- --Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfDJuuAAoJEBmVNT4SmAt+VUAQAJo6R4/1QGtnigzZgZADqqsp 8LD/jBN3CsKmWIOfOLOb777YIlctU12ltxWujGdaPSztpV43mKzfJ4RTrBzVvJzh KS13naw0jvLxnUrzMsSV55wC9SY2NI0WtSD0ZgKxLeU+YCx+ru7wsex3Ph6eQziL 3qKdInKtjaUU1cp8XteU9zZXS1E1TJj6BHYncTQp2+/fgAUKgAC9Kw61N1vG0P34 d1yAoGa+9yg5KbaceIMDsotHE1KF5unZAofHt1RH43wIJAOcVV5UsImcPF0HEgK+ dB7jZT2k55Pm01YVlhSoVXYBMu5LCZnCiVnDMpXzbqdsSYwamYDv52H0p3dseof6 JVxQNz35ao4FdlkJ+yVxp85HnB9xh+FNsyO4ofZiUxekjyexu3AvDRcF50dJXCL1 HWThCQf0knXSV2aysbCmo8qezDcyZbyd4RgvqnJcJPFt1uJKUcQj4EpXmjIJFOT3 DM9xqcse7cBTEAwdp2693In+F5dAA53HXSa9rub8Z3lNQzXY/RKAI8Eag9ojBK3f N79JA97bhOgizk1N2zIlQjxIbZouDtWk/wP9bsesaBXQ5FydJwlEvMEUsUHdTz7u rBMkZK7Dq7XOfbOgQoJCc/g5eEICrWdO+2r/Mfok9a+hj7Mzu5HRuI1Y3Rn4tfte YPFHEt5rErRTe4vyzVyQ =2YHU -----END PGP SIGNATURE----- --Upx5m2iDaQcpDAsFa9BiJ76FZdTCNyK6P-- From owner-freebsd-fs@freebsd.org Mon Jul 13 18:02:58 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C0BE936C4F5 for ; Mon, 13 Jul 2020 18:02:58 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5BMQ2lPwz4Jy8; Mon, 13 Jul 2020 18:02:58 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (m.cksoft.de [195.88.109.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id 889DF1EAAF1; Mon, 13 Jul 2020 20:02:55 +0200 (CEST) Received: from amavisfra1.cksoft.de (unknown [IPv6:2001:67c:24f8:2003::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 3432E63027; Mon, 13 Jul 2020 20:02:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.35.42]) by amavisfra1.cksoft.de (amavisfra1.cksoft.de [192.168.35.41]) (amavisd-new, port 10051) with ESMTP id yr1viylq0lqm; Mon, 13 Jul 2020 20:02:52 +0200 (CEST) Received: from nocfra1.cksoft.de (nocfra1.cksoft.de [IPv6:2001:67c:24f8:2001::53:1]) by m.cksoft.de (Postfix) with ESMTP id EE44C63026; Mon, 13 Jul 2020 20:02:53 +0200 (CEST) Received: by nocfra1.cksoft.de (Postfix, from userid 1000) id 0132313EBA; Mon, 13 Jul 2020 20:02:53 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by nocfra1.cksoft.de (Postfix) with ESMTP id EF41313EB7; Mon, 13 Jul 2020 20:02:53 +0200 (CEST) Date: Mon, 13 Jul 2020 20:02:53 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@nocfra1.cksoft.de Reply-To: Christian Kratzer To: Allan Jude cc: freebsd-fs@freebsd.org Subject: Re: gptzfsboot targeting wrong vdev In-Reply-To: Message-ID: References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> User-Agent: Alpine 2.22 (BSF 395 2020-01-19) X-NCC-RegID: de.cksoft X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4B5BMQ2lPwz4Jy8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:57407, ipnet:2001:67c:24f8::/48, country:DE] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 18:02:58 -0000 Hi, On Mon, 13 Jul 2020, Allan Jude wrote: > bootme is only used by gptboot, for UFS. ok. > You might want to example your disks with 'zdb -l /dev/ada0' (for each > disk, and each partition). It seems there is something not encrypted > somewhere. It may be an old label from a pool that has been destroyed, > and a new pool with the same name was later created or something. let me provide the partitions up front root@zfs1:/home/ck # gpart show /dev/ada0 => 40 468862048 ada0 GPT (224G) 40 1024 1 freebsd-boot (512K) 1064 134217728 2 freebsd-swap (64G) 134218792 33554432 3 freebsd-vinum (16G) 167773224 33554432 4 freebsd-vinum (16G) 201327656 267534424 5 freebsd-zfs [bootme] (128G) 468862080 8 - free - (4.0K) root@zfs1:/home/ck # ada1 is essentially the same. These are the two m2 ssd I have zroot und zil on. The two freebsd-vinum partitions used to be freebsd-zfs Partitions 3 and 4 are used for the log device and these happen to have the names I was getting confused about root@zfs1:/home/ck # zdb -l /dev/ada0p3 | head -10 ------------------------------------ LABEL 0 ------------------------------------ version: 5000 name: 'zp1' state: 0 txg: 37392274 pool_guid: 6725164972097052508 hostid: 694021601 hostname: 'zfs1' root@zfs1:/home/ck # root@zfs1:/home/ck # zdb -l /dev/ada0p4 | head -10 ------------------------------------ LABEL 0 ------------------------------------ version: 5000 name: 'zp2' state: 0 txg: 21037796 pool_guid: 16088850666393843588 hostid: 694021601 hostname: 'zfs1' root@zfs1:/home/ck # It is just a name and not a label. gptzfsboot was trying to tell me it cannot boot from ada0p3 which has the name zp1. My two other pools on the sas hba happen to be called zp1 and zp2. For some reason when I setup the system I moved stuff around and ended up with two zlog partitions first and the zroot last. So now everything makes sense again. The freebsd-vinum hack solves this for me and I have gained some more experience. In the future I will ensure to have the first partition after boot to be the zroot. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-fs@freebsd.org Mon Jul 13 18:50:24 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 6F88C36D54B for ; Mon, 13 Jul 2020 18:50:24 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5CQ8236qz4MLs for ; Mon, 13 Jul 2020 18:50:24 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 8FDB320213; Mon, 13 Jul 2020 18:50:23 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 8FDB320213 Subject: Re: gptzfsboot targeting wrong vdev To: Christian Kratzer Cc: freebsd-fs@freebsd.org References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> From: Allan Jude Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Message-ID: Date: Mon, 13 Jul 2020 14:50:22 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="mo73Kw2L6DA2XfwVvLDFd9PwCBQuLoHR6" X-Rspamd-Queue-Id: 4B5CQ8236qz4MLs X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US]; local_wl_from(0.00)[freebsd.org] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 18:50:24 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --mo73Kw2L6DA2XfwVvLDFd9PwCBQuLoHR6 Content-Type: multipart/mixed; boundary="LON8G8603DFyG9bh0AbOza5oSTxIWhgd9"; protected-headers="v1" From: Allan Jude To: Christian Kratzer Cc: freebsd-fs@freebsd.org Message-ID: Subject: Re: gptzfsboot targeting wrong vdev References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> In-Reply-To: --LON8G8603DFyG9bh0AbOza5oSTxIWhgd9 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable On 2020-07-13 14:02, Christian Kratzer wrote: > Hi, >=20 > On Mon, 13 Jul 2020, Allan Jude wrote: >> bootme is only used by gptboot, for UFS. >=20 > ok. >=20 >> You might want to example your disks with 'zdb -l /dev/ada0' (for each= >> disk, and each partition). It seems there is something not encrypted >> somewhere. It may be an old label from a pool that has been destroyed,= >> and a new pool with the same name was later created or something. >=20 > let me provide the partitions up front >=20 > root@zfs1:/home/ck # gpart show /dev/ada0 > =3D>=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0 468862048=C2=A0 ada0=C2= =A0 GPT=C2=A0 (224G) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 40=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0 1024=C2=A0=C2=A0=C2=A0=C2=A0 1=C2=A0 freebsd-boot=C2=A0 (= 512K) > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 1064=C2=A0 134217728=C2=A0=C2=A0=C2= =A0=C2=A0 2=C2=A0 freebsd-swap=C2=A0 (64G) > =C2=A0 134218792=C2=A0=C2=A0 33554432=C2=A0=C2=A0=C2=A0=C2=A0 3=C2=A0 f= reebsd-vinum=C2=A0 (16G) > =C2=A0 167773224=C2=A0=C2=A0 33554432=C2=A0=C2=A0=C2=A0=C2=A0 4=C2=A0 f= reebsd-vinum=C2=A0 (16G) > =C2=A0 201327656=C2=A0 267534424=C2=A0=C2=A0=C2=A0=C2=A0 5=C2=A0 freebs= d-zfs=C2=A0 [bootme]=C2=A0 (128G) > =C2=A0 468862080=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 = 8=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 - free -=C2=A0 (4.0K) >=20 > root@zfs1:/home/ck # >=20 > ada1 is essentially the same. >=20 > These are the two m2 ssd I have zroot und zil on. >=20 > The two freebsd-vinum partitions used to be freebsd-zfs >=20 > Partitions 3 and 4 are used for the log device and these happen to have= > the names I was getting confused about >=20 > root@zfs1:/home/ck # zdb -l /dev/ada0p3 | head -10 > ------------------------------------ > LABEL 0 > ------------------------------------ > =C2=A0=C2=A0=C2=A0 version: 5000 > =C2=A0=C2=A0=C2=A0 name: 'zp1' > =C2=A0=C2=A0=C2=A0 state: 0 > =C2=A0=C2=A0=C2=A0 txg: 37392274 > =C2=A0=C2=A0=C2=A0 pool_guid: 6725164972097052508 > =C2=A0=C2=A0=C2=A0 hostid: 694021601 > =C2=A0=C2=A0=C2=A0 hostname: 'zfs1' > root@zfs1:/home/ck # >=20 >=20 >=20 > root@zfs1:/home/ck # zdb -l /dev/ada0p4 | head -10 > ------------------------------------ > LABEL 0 > ------------------------------------ > =C2=A0=C2=A0=C2=A0 version: 5000 > =C2=A0=C2=A0=C2=A0 name: 'zp2' > =C2=A0=C2=A0=C2=A0 state: 0 > =C2=A0=C2=A0=C2=A0 txg: 21037796 > =C2=A0=C2=A0=C2=A0 pool_guid: 16088850666393843588 > =C2=A0=C2=A0=C2=A0 hostid: 694021601 > =C2=A0=C2=A0=C2=A0 hostname: 'zfs1' > root@zfs1:/home/ck # >=20 >=20 > It is just a name and not a label.=C2=A0 gptzfsboot was trying to tell = me it > cannot boot from ada0p3 which has the name zp1. >=20 > My two other pools on the sas hba happen to be called zp1 and zp2. >=20 > For some reason when I setup the system I moved stuff around and ended > up with two zlog partitions first and the zroot last. >=20 > So now everything makes sense again. >=20 > The freebsd-vinum hack solves this for me and I have gained some more > experience. >=20 > In the future I will ensure=C2=A0 to have the first partition after boo= t to > be the zroot. >=20 > Greetings > Christian >=20 So are your SLOG devices not encrypted? That seems like an oversight, since any synchronous writes will be written to the SLOG first. --=20 Allan Jude --LON8G8603DFyG9bh0AbOza5oSTxIWhgd9-- --mo73Kw2L6DA2XfwVvLDFd9PwCBQuLoHR6 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfDKzuAAoJEBmVNT4SmAt+V2wQAK0IVVRpnDjnhkRKt9mkuET0 KHD5MzhRVYyokSX+ASlPi44f6bay5928xKoMGi3Yp3DOyM/LX90+jSXD/dqbPsHg NR6+qXopfGmmn2jQVyaooIMhfAhtRUmeTsKsXvVBekHP5D6+Lk4Vs3rYOQtTsPeJ 8LFK5C8CQ3l0p1payAWVYGqbZgfmUH2QoJ9Pt5hAYjMWUKNGzv00c87XkGUxd/Vy s6niqf8lDIlMqVAL4so/JPOjR1WBM8DtXda/h3u+xSamN27N0qxnipkdvNvsaP92 xfBvwB7dmy8C6Cn1f7B0H5U2XZJPy8iSJl/TkRH0jxXxuAEHcFzuzOCHaa0crsM6 lr8AMqa+VRT+FVSXXDND6CX0nzgmOTg1bBUu+f/1FxL/fEbi8xb8UPEIOLgo3iho JHdvxw7aVUESVDs+7iob6PhUGLujfXIx/QN4wRUvsoFd5A7B8WERLL8xxco7CXcm XB1yHwLl54qcye8IX+CNzEAAG1YdKtOTb9N4RSd/+AuHcP5R2tDyeDmP/WACsqVu 36WswAoIrFJsBYeTXahYANF6Eiu6oaLjFJCuxoSvCQKT12ckNwZGDUoU//JfO1TP IdD+pixmAb0PI30G+QSAEp5WeUUU+aandgC/UjTqo9h03PVOBF4OUqxOzPoR9CGs /tCaImYtf5+hz8BnUebz =tOEt -----END PGP SIGNATURE----- --mo73Kw2L6DA2XfwVvLDFd9PwCBQuLoHR6-- From owner-freebsd-fs@freebsd.org Tue Jul 14 04:59:57 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E87B354FF5; Tue, 14 Jul 2020 04:59:57 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [IPv6:2001:19f0:300:2185:123::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5SxR2KtZz4Km6; Tue, 14 Jul 2020 04:59:55 +0000 (UTC) (envelope-from jamie@catflap.org) Received: from donotpassgo.dyslexicfish.net (donotpassgo.dyslexicfish.net [104.207.135.49]) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5) with ESMTP id 06E4xmGs062844 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Tue, 14 Jul 2020 05:59:48 +0100 (BST) (envelope-from jamie@donotpassgo.dyslexicfish.net) Received: (from jamie@localhost) by donotpassgo.dyslexicfish.net (8.14.5/8.14.5/Submit) id 06E4xm79062842; Tue, 14 Jul 2020 05:59:48 +0100 (BST) (envelope-from jamie) From: Jamie Landeg-Jones Message-Id: <202007140459.06E4xm79062842@donotpassgo.dyslexicfish.net> Date: Tue, 14 Jul 2020 05:59:48 +0100 Organization: Dyslexic Fish To: mmacy@freebsd.org, freebsd-hackers@freebsd.org, freebsd-fs@freebsd.org, freebsd-current@freebsd.org Subject: Re: CFT for vendor openzfs - week 2 reminder References: In-Reply-To: User-Agent: Heirloom mailx 12.4 7/29/08 MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.2.7 (donotpassgo.dyslexicfish.net [104.207.135.49]); Tue, 14 Jul 2020 05:59:48 +0100 (BST) X-Rspamd-Queue-Id: 4B5SxR2KtZz4Km6 X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=catflap.org; spf=pass (mx1.freebsd.org: domain of jamie@catflap.org designates 2001:19f0:300:2185:123::1 as permitted sender) smtp.mailfrom=jamie@catflap.org X-Spamd-Result: default: False [-3.01 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-0.98)[-0.980]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx:dyslexicfish.net]; MIME_GOOD(-0.10)[text/plain]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-0.98)[-0.983]; HAS_ORG_HEADER(0.00)[]; NEURAL_HAM_SHORT(-0.24)[-0.243]; DMARC_POLICY_ALLOW(-0.50)[catflap.org,none]; FROM_EQ_ENVFROM(0.00)[]; RCVD_TLS_LAST(0.00)[]; R_DKIM_NA(0.00)[]; ASN(0.00)[asn:20473, ipnet:2001:19f0::/38, country:US]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jul 2020 04:59:57 -0000 Can anyone using the new vendor openzfs let us know if it fixes the "mmp_thread_enter" bug recently MFC'ed to STABLE? https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=247829 Cheers From owner-freebsd-fs@freebsd.org Tue Jul 14 05:34:35 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E431F3566F5 for ; Tue, 14 Jul 2020 05:34:35 +0000 (UTC) (envelope-from agapon@gmail.com) Received: from mail-lf1-f49.google.com (mail-lf1-f49.google.com [209.85.167.49]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5TjQ2bZPz4S4w; Tue, 14 Jul 2020 05:34:34 +0000 (UTC) (envelope-from agapon@gmail.com) Received: by mail-lf1-f49.google.com with SMTP id m26so10643399lfo.13; Mon, 13 Jul 2020 22:34:34 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:openpgp:autocrypt :message-id:date:user-agent:mime-version:in-reply-to :content-language:content-transfer-encoding; bh=5fyAm+zxv1R/wr4ZEofsgBNKlk9aigXg4TCQP417eIc=; b=Y/z/bM5nZJgyckOZRgyPOke7t0iDrx2TelQK4NcIi3cNszoKMC/MEW70yDsHSUA817 swkk62k3bEsBSGjZAngxh7TElBJT6rYWYGFktJc+zZturVk7X25KRd0vkzWte5iBGxYv r2Ox8IvPyXUhUIwXjTWDhZpfX64O8eiFL4vcljP+nCGZb2BZUqt8YHdOKgDTDYzD1pp8 DtFW4f5UXK9om1FJ9WvYpPut5M/CyeFsBmwTV1LSKxhaUX2/yW1ZuzDE/HtlF/2oagDA Mj9RvRIvo4U/8191biAUn8QRLTjw+xpGXWlz5vjsZplF0BPGtsFXuCtKkSgOENXy8+nu P8dw== X-Gm-Message-State: AOAM531g2aYZP4dy7SeT10cHaBzFPhg9MjOIGaS8eOoNm86xDzPf9cVx 4cxFijcwXU9ikMsSkTdXbmIpBvnCgL0= X-Google-Smtp-Source: ABdhPJxyhVecyG2wn1p3JV0/7fhfq2SbSBE691vfnmBiT5n9aX9kQouY/hJHCYTfpggsmLrLLK8FUQ== X-Received: by 2002:a05:6512:202a:: with SMTP id s10mr1313149lfs.12.1594704872182; Mon, 13 Jul 2020 22:34:32 -0700 (PDT) Received: from [192.168.0.88] (east.meadow.volia.net. [93.72.151.96]) by smtp.googlemail.com with ESMTPSA id 23sm5068512lff.91.2020.07.13.22.34.30 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Mon, 13 Jul 2020 22:34:31 -0700 (PDT) Subject: Re: gptzfsboot targeting wrong vdev To: Christian Kratzer , Allan Jude Cc: freebsd-fs@freebsd.org, Toomas Soome References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> From: Andriy Gapon Openpgp: preference=signencrypt Autocrypt: addr=avg@FreeBSD.org; prefer-encrypt=mutual; keydata= mQINBFm4LIgBEADNB/3lT7f15UKeQ52xCFQx/GqHkSxEdVyLFZTmY3KyNPQGBtyvVyBfprJ7 mAeXZWfhat6cKNRAGZcL5EmewdQuUfQfBdYmKjbw3a9GFDsDNuhDA2QwFt8BmkiVMRYyvI7l N0eVzszWCUgdc3qqM6qqcgBaqsVmJluwpvwp4ZBXmch5BgDDDb1MPO8AZ2QZfIQmplkj8Y6Z AiNMknkmgaekIINSJX8IzRzKD5WwMsin70psE8dpL/iBsA2cpJGzWMObVTtCxeDKlBCNqM1i gTXta1ukdUT7JgLEFZk9ceYQQMJJtUwzWu1UHfZn0Fs29HTqawfWPSZVbulbrnu5q55R4PlQ /xURkWQUTyDpqUvb4JK371zhepXiXDwrrpnyyZABm3SFLkk2bHlheeKU6Yql4pcmSVym1AS4 dV8y0oHAfdlSCF6tpOPf2+K9nW1CFA8b/tw4oJBTtfZ1kxXOMdyZU5fiG7xb1qDgpQKgHUX8 7Rd2T1UVLVeuhYlXNw2F+a2ucY+cMoqz3LtpksUiBppJhw099gEXehcN2JbUZ2TueJdt1FdS ztnZmsHUXLxrRBtGwqnFL7GSd6snpGIKuuL305iaOGODbb9c7ne1JqBbkw1wh8ci6vvwGlzx rexzimRaBzJxlkjNfMx8WpCvYebGMydNoeEtkWldtjTNVsUAtQARAQABtB5BbmRyaXkgR2Fw b24gPGF2Z0BGcmVlQlNELm9yZz6JAlQEEwEIAD4WIQS+LEO7ngQnXA4Bjr538m7TUc1yjwUC WbgsiAIbIwUJBaOagAULCQgHAgYVCAkKCwIEFgIDAQIeAQIXgAAKCRB38m7TUc1yj+JAEACV l9AK/nOWAt/9cufV2fRj0hdOqB1aCshtSrwHk/exXsDa4/FkmegxXQGY+3GWX3deIyesbVRL rYdtdK0dqJyT1SBqXK1h3/at9rxr9GQA6KWOxTjUFURsU7ok/6SIlm8uLRPNKO+yq0GDjgaO LzN+xykuBA0FlhQAXJnpZLcVfPJdWv7sSHGedL5ln8P8rxR+XnmsA5TUaaPcbhTB+mG+iKFj GghASDSfGqLWFPBlX/fpXikBDZ1gvOr8nyMY9nXhgfXpq3B6QCRYKPy58ChrZ5weeJZ29b7/ QdEO8NFNWHjSD9meiLdWQaqo9Y7uUxN3wySc/YUZxtS0bhAd8zJdNPsJYG8sXgKjeBQMVGuT eCAJFEYJqbwWvIXMfVWop4+O4xB+z2YE3jAbG/9tB/GSnQdVSj3G8MS80iLS58frnt+RSEw/ psahrfh0dh6SFHttE049xYiC+cM8J27Aaf0i9RflyITq57NuJm+AHJoU9SQUkIF0nc6lfA+o JRiyRlHZHKoRQkIg4aiKaZSWjQYRl5Txl0IZUP1dSWMX4s3XTMurC/pnja45dge/4ESOtJ9R 8XuIWg45Oq6MeIWdjKddGhRj3OohsltKgkEU3eLKYtB6qRTQypHHUawCXz88uYt5e3w4V16H lCpSTZV/EVHnNe45FVBlvK7k7HFfDDkryLkCDQRZuCyIARAAlq0slcsVboY/+IUJdcbEiJRW be9HKVz4SUchq0z9MZPX/0dcnvz/gkyYA+OuM78dNS7Mbby5dTvOqfpLJfCuhaNYOhlE0wY+ 1T6Tf1f4c/uA3U/YiadukQ3+6TJuYGAdRZD5EqYFIkreARTVWg87N9g0fT9BEqLw9lJtEGDY EWUE7L++B8o4uu3LQFEYxcrb4K/WKmgtmFcm77s0IKDrfcX4doV92QTIpLiRxcOmCC/OCYuO jB1oaaqXQzZrCutXRK0L5XN1Y1PYjIrEzHMIXmCDlLYnpFkK+itlXwlE2ZQxkfMruCWdQXye syl2fynAe8hvp7Mms9qU2r2K9EcJiR5N1t1C2/kTKNUhcRv7Yd/vwusK7BqJbhlng5ZgRx0m WxdntU/JLEntz3QBsBsWM9Y9wf2V4tLv6/DuDBta781RsCB/UrU2zNuOEkSixlUiHxw1dccI 6CVlaWkkJBxmHX22GdDFrcjvwMNIbbyfQLuBq6IOh8nvu9vuItup7qemDG3Ms6TVwA7BD3j+ 3fGprtyW8Fd/RR2bW2+LWkMrqHffAr6Y6V3h5kd2G9Q8ZWpEJk+LG6Mk3fhZhmCnHhDu6CwN MeUvxXDVO+fqc3JjFm5OxhmfVeJKrbCEUJyM8ESWLoNHLqjywdZga4Q7P12g8DUQ1mRxYg/L HgZY3zfKOqcAEQEAAYkCPAQYAQgAJhYhBL4sQ7ueBCdcDgGOvnfybtNRzXKPBQJZuCyIAhsM BQkFo5qAAAoJEHfybtNRzXKPBVwQAKfFy9P7N3OsLDMB56A4Kf+ZT+d5cIx0Yiaf4n6w7m3i ImHHHk9FIetI4Xe54a2IXh4Bq5UkAGY0667eIs+Z1Ea6I2i27Sdo7DxGwq09Qnm/Y65ADvXs 3aBvokCcm7FsM1wky395m8xUos1681oV5oxgqeRI8/76qy0hD9WR65UW+HQgZRIcIjSel9vR XDaD2HLGPTTGr7u4v00UeTMs6qvPsa2PJagogrKY8RXdFtXvweQFz78NbXhluwix2Tb9ETPk LIpDrtzV73CaE2aqBG/KrboXT2C67BgFtnk7T7Y7iKq4/XvEdDWscz2wws91BOXuMMd4c/c4 OmGW9m3RBLufFrOag1q5yUS9QbFfyqL6dftJP3Zq/xe+mr7sbWbhPVCQFrH3r26mpmy841ym dwQnNcsbIGiBASBSKksOvIDYKa2Wy8htPmWFTEOPRpFXdGQ27awcjjnB42nngyCK5ukZDHi6 w0qK5DNQQCkiweevCIC6wc3p67jl1EMFY5+z+zdTPb3h7LeVnGqW0qBQl99vVFgzLxchKcl0 R/paSFgwqXCZhAKMuUHncJuynDOP7z5LirUeFI8qsBAJi1rXpQoLJTVcW72swZ42IdPiboqx NbTMiNOiE36GqMcTPfKylCbF45JNX4nF9ElM0E+Y8gi4cizJYBRr2FBJgay0b9Cp Message-ID: Date: Tue, 14 Jul 2020 08:34:30 +0300 User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:60.0) Gecko/20100101 Firefox/60.0 Thunderbird/60.9.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 4B5TjQ2bZPz4S4w X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=none; spf=pass (mx1.freebsd.org: domain of agapon@gmail.com designates 209.85.167.49 as permitted sender) smtp.mailfrom=agapon@gmail.com X-Spamd-Result: default: False [-1.74 / 15.00]; MIME_TRACE(0.00)[0:+]; RCVD_VIA_SMTP_AUTH(0.00)[]; MID_RHS_MATCH_FROM(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[4]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+ip4:209.85.128.0/17]; MIME_GOOD(-0.10)[text/plain]; DMARC_NA(0.00)[FreeBSD.org]; ARC_NA(0.00)[]; NEURAL_SPAM_SHORT(0.05)[0.046]; NEURAL_HAM_LONG(-0.82)[-0.821]; RCVD_COUNT_THREE(0.00)[3]; TO_MATCH_ENVRCPT_SOME(0.00)[]; RCVD_IN_DNSWL_NONE(0.00)[209.85.167.49:from]; NEURAL_HAM_MEDIUM(-0.97)[-0.967]; FORGED_SENDER(0.30)[avg@FreeBSD.org,agapon@gmail.com]; RWL_MAILSPIKE_POSSIBLE(0.00)[209.85.167.49:from]; R_DKIM_NA(0.00)[]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:209.85.128.0/17, country:US]; RCVD_TLS_ALL(0.00)[]; FROM_NEQ_ENVFROM(0.00)[avg@FreeBSD.org,agapon@gmail.com]; RECEIVED_SPAMHAUS_PBL(0.00)[93.72.151.96:received] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jul 2020 05:34:36 -0000 On 13/07/2020 21:02, Christian Kratzer wrote: > In the future I will ensure  to have the first partition after boot to be the > zroot. By the way, I've been considering changing our ZFS boot code so that it does not treat a pool as a candidate unless it has bootfs explicitly set (non-default value). I think that that would be a reasonable change. And not hard to do. But never got around to doing it. -- Andriy Gapon From owner-freebsd-fs@freebsd.org Tue Jul 14 18:14:07 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C206136B364 for ; Tue, 14 Jul 2020 18:14:07 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5pYq1WsLz4KD1; Tue, 14 Jul 2020 18:14:06 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (m.cksoft.de [IPv6:2001:67c:24f8:2003::25:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id 7A4C71E9EB6; Tue, 14 Jul 2020 20:13:58 +0200 (CEST) Received: from amavisfra1.cksoft.de (unknown [IPv6:2001:67c:24f8:2003::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 2AD1C63027; Tue, 14 Jul 2020 20:13:58 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([IPv6:2001:67c:24f8:2003::25:3]) by amavisfra1.cksoft.de (amavisfra1.cksoft.de [IPv6:2001:67c:24f8:2003::25:a1]) (amavisd-new, port 10051) with ESMTP id eQ8LuSygdGiT; Tue, 14 Jul 2020 20:13:52 +0200 (CEST) Received: from nocfra1.cksoft.de (nocfra1.cksoft.de [IPv6:2001:67c:24f8:2001::53:1]) by m.cksoft.de (Postfix) with ESMTP id 5348C63026; Tue, 14 Jul 2020 20:13:54 +0200 (CEST) Received: by nocfra1.cksoft.de (Postfix, from userid 1000) id 6303813ECA; Tue, 14 Jul 2020 20:13:54 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by nocfra1.cksoft.de (Postfix) with ESMTP id 5E58B13E4A; Tue, 14 Jul 2020 20:13:54 +0200 (CEST) Date: Tue, 14 Jul 2020 20:13:54 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@nocfra1.cksoft.de Reply-To: Christian Kratzer To: Allan Jude cc: freebsd-fs@freebsd.org Subject: Re: gptzfsboot targeting wrong vdev In-Reply-To: Message-ID: References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> User-Agent: Alpine 2.22 (BSF 395 2020-01-19) X-NCC-RegID: de.cksoft X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4B5pYq1WsLz4KD1 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:57407, ipnet:2001:67c:24f8::/48, country:DE]; REPLY(-4.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Jul 2020 18:14:07 -0000 Hi, On Mon, 13 Jul 2020, Allan Jude wrote: > So are your SLOG devices not encrypted? That seems like an oversight, > since any synchronous writes will be written to the SLOG first. yes the slog devices are not encrypted. And /boot/keys contains the keys. All of the above are on the ada0, ada1 m2 ssd devices. The main threat scenario I am protecting against is disposal of end of live or broken disks in the main pool. If I wanted to protect ada0/1 I would need to fall back to booting from a separate external devices again that also includes the keys. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/ From owner-freebsd-fs@freebsd.org Wed Jul 15 02:26:15 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 2ED45356EAC for ; Wed, 15 Jul 2020 02:26:15 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from tor1-11.mx.scaleengine.net (tor1-11.mx.scaleengine.net [209.51.186.6]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 4B61Td1gW6z3YKt for ; Wed, 15 Jul 2020 02:26:12 +0000 (UTC) (envelope-from allanjude@freebsd.org) Received: from [10.1.1.2] (Seawolf.HML3.ScaleEngine.net [209.51.186.28]) (Authenticated sender: allanjude.freebsd@scaleengine.com) by tor1-11.mx.scaleengine.net (Postfix) with ESMTPSA id 5686A24C48; Wed, 15 Jul 2020 02:26:06 +0000 (UTC) DKIM-Filter: OpenDKIM Filter v2.10.3 tor1-11.mx.scaleengine.net 5686A24C48 From: Allan Jude To: status-updates@freebsdfoundation.org, freebsd-fs , openzfs-developer References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> Autocrypt: addr=allanjude@freebsd.org; prefer-encrypt=mutual; keydata= xsFNBFVwZcYBEADwrZDH0xe0ZVjc9ORCc6PcBLwS/RTXA6NkvpD6ea02pZ8lPOVgteuuugFc D34LdDbiWr+479vfrKBh+Y38GL0oZ0/13j10tIlDMHSa5BU0y6ACtnhupFvVlQ57+XaJAb/q 7qkfSiuxVwQ3FY3PL3cl1RrIP5eGHLA9hu4eVbu+FOX/q/XVKz49HaeIaxzo2Q54572VzIo6 C28McX9m65UL5fXMUGJDDLCItLmehZlHsQQ+uBxvODLFpVV2lUgDR/0rDa0B9zHZX8jY8qQ7 ZdCSy7CwClXI054CkXZCaBzgxYh/CotdI8ezmaw7NLs5vWNTxaDEFXaFMQtMVhvqQBpHkfOD 7rjjOmFw00nJL4FuPE5Yut0CPyx8vLjVmNJSt/Y8WxxmhutsqJYFgYfWl/vaWkrFLur/Zcmz IklwLw35HLsCZytCN5A3rGKdRbQjD6QPXOTJu0JPrJF6t2xFkWAT7oxnSV0ELhl2g+JfMMz2 Z1PDmS3NRnyEdqEm7NoRGXJJ7bgxDbN+9SXTyOletqGNXj/bSrBvhvZ0RQrzdHAPwQUfVSU2 qBhQEi2apSZstgVNMan0GUPqCdbE2zpysg+zT7Yhvf9EUQbzPL4LpdK1llT9fZbrdMzEXvEF oSvwJFdV3sqKmZc7b+E3PuxK6GTsKqaukd/3Cj8aLHG1T1im1QARAQABzSJBbGxhbiBKdWRl IDxhbGxhbmp1ZGVAZnJlZWJzZC5vcmc+wsF/BBMBAgApBQJVcGXGAhsjBQkSzAMABwsJCAcD AgEGFQgCCQoLBBYCAwECHgECF4AACgkQGZU1PhKYC34Muw/+JOKpSfhhysWFYiRXynGRDe07 Z6pVsn7DzrPUMRNZfHu8Uujmmy3p2nx9FelIY9yjd2UKHhug+whM54MiIFs90eCRVa4XEsPR 4FFAm0DAWrrb7qhZFcE/GhHdRWpZ341WAElWf6Puj2devtRjfYbikvj5+1V1QmDbju7cEw5D mEET44pTuD2VMRJpu2yZZzkM0i+wKFuPxlhqreufA1VNkZXI/rIfkYWK+nkXd9Efw3YdCyCQ zUgTUCb88ttSqcyhik/li1CDbXBpkzDCKI6I/8fAb7jjOC9LAtrZJrdgONywcVFoyK9ZN7EN AVA+xvYCmuYhR/3zHWH1g4hAm1v1+gIsufhajhfo8/wY1SetlzPaYkSkVQLqD8T6zZyhf+AN bC7ci44UsiKGAplB3phAXrtSPUEqM86kbnHg3fSx37kWKUiYNOnx4AC2VXvEiKsOBlpyt3dw WQbOtOYM+vkfbBwDtoGOOPYAKxc4LOIt9r+J8aD+gTooi9Eo5tvphATf9WkCpl9+aaGbSixB tUpvQMRnSMqTqq4Z7DeiG6VMRQIjsXDSLJEUqcfhnLFo0Ko/RiaHd5xyAQ4DhQ9QpkyQjjNf /3f/dYG7JAtoD30txaQ5V8uHrz210/77DRRX+HJjEj6xCxWUGvQgvEZf5XXyxeePvqZ+zQyT DX61bYw6w6bOwU0EVXBlxgEQAMy7YVnCCLN4oAOBVLZ5nUbVPvpUhsdA94/0/P+uqCIh28Cz ar56OCX0X19N/nAWecxL4H32zFbIRyDB2V/MEh4p9Qvyu/j4i1r3Ex5GhOT2hnit43Ng46z5 29Es4TijrHJP4/l/rB2VOqMKBS7Cq8zk1cWqaI9XZ59imxDNjtLLPPM+zQ1yE3OAMb475QwN UgWxTMw8rkA7CEaqeIn4sqpTSD5C7kT1Bh26+rbgJDZ77D6Uv1LaCZZOaW52okW3bFbdozV8 yM2u+xz2Qs8bHz67p+s+BlygryiOyYytpkiK6Iy4N7FTolyj5EIwCuqzfk0SaRHeOKX2ZRjC qatkgoD/t13PNT38V9tw3qZVOJDS0W6WM8VSg+F+bkM9LgJ8CmKV+Hj0k3pfGfYPOZJ/v18i +SmZmL/Uw2RghnwDWGAsPCKu4uZR777iw7n9Io6Vfxndw2dcS0e9klvFYoaGS6H2F13Asygr WBzFNGFQscN4mUW+ZYBzpTOcHkdT7w8WS55BmXYLna+dYer9/HaAuUrONjujukN4SPS1fMJ2 /CS/idAUKyyVVX5vozoNK2JVC1h1zUAVsdnmhEzNPsvBoqcVNfyqBFROEVLIPwq+lQMGNVjH ekLTKRWf59MEhUC2ztjSKkGmwdg73d6xSXMuq45EgIJV2wPvOgWQonoHH/kxABEBAAHCwWUE GAECAA8FAlVwZcYCGwwFCRLMAwAACgkQGZU1PhKYC34w5A//YViBtZyDV5O+SJT9FFO3lb9x Zdxf0trA3ooCt7gdBkdnBM6T5EmjgVZ3KYYyFfwXZVkteuCCycMF/zVw5eE9FL1+zz9gg663 nY9q2F77TZTKXVWOLlOV2bY+xaK94U4ytogOGhh9b4UnQ/Ct3+6aviCF78Go608BXbmF/GVT 7uhddemk7ItxM1gE5Hscx3saxGKlayaOsdPKeGTVJCDEtHDuOc7/+jGh5Zxpk/Hpi+DUt1ot 8e6hPYLIQa4uVx4f1xxxV858PQ7QysSLr9pTV7FAQ18JclCaMc7JWIa3homZQL/MNKOfST0S 2e+msuRwQo7AnnfFKBUtb02KwpA4GhWryhkjUh/kbVc1wmGxaU3DgXYQ5GV5+Zf4kk/wqr/7 KG0dkTz6NLCVLyDlmAzuFhf66DJ3zzz4yIo3pbDYi3HB/BwJXVSKB3Ko0oUo+6/qMrOIS02L s++QE/z7K12CCcs7WwOjfCYHK7VtE0Sr/PfybBdTbuDncOuAyAIeIKxdI2nmQHzl035hhvQX s4CSghsP319jAOQiIolCeSbTMD4QWMK8RL/Pe1FI1jC3Nw9s+jq8Dudtbcj2UwAP/STUEbJ9 5rznzuuhPjE0e++EU/RpWmcaIMK/z1zZDMN+ce2v1qzgV936ZhJ3iaVzyqbEE81gDxg3P+IM kiYh4ZtPB4Q= Subject: Re: ZSTD Project Weekly Status Update Message-ID: <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> Date: Tue, 14 Jul 2020 22:26:01 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:68.0) Gecko/20100101 Thunderbird/68.10.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="11rZcGtLSc3Dz7SCZJTTfFXDO0W11n4If" X-Rspamd-Queue-Id: 4B61Td1gW6z3YKt X-Spamd-Bar: / Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [0.00 / 15.00]; ASN(0.00)[asn:6939, ipnet:209.51.160.0/19, country:US]; local_wl_from(0.00)[freebsd.org] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Jul 2020 02:26:15 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --11rZcGtLSc3Dz7SCZJTTfFXDO0W11n4If Content-Type: multipart/mixed; boundary="66UlQ09ydSSLlog0hd3djPINvizOVW1Iw"; protected-headers="v1" From: Allan Jude To: status-updates@freebsdfoundation.org, freebsd-fs , openzfs-developer Message-ID: <528ca743-7889-d1fd-ca95-a17cd430725b@freebsd.org> Subject: Re: ZSTD Project Weekly Status Update References: <7b8842ad-d520-c575-22ee-2cd77244f2c6@freebsd.org> <708ec9f2-3c5c-6452-f6e6-bfb11a7f7eb2@freebsd.org> In-Reply-To: --66UlQ09ydSSLlog0hd3djPINvizOVW1Iw Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable In my continuing effort to complete the integration of ZSTD into OpenZFS, here is my fourth weekly status report: https://github.com/allanjude/zfs/commit/b0b1270d4e7835ecff413208301375e3d= e2a4153 - Create a new test case to make sure that the ZSTD header we write along with the data is correct. Verify that the physical size of the compressed data is less than the psize for the block pointer, and verify that the level matches. It uses a random level between 1 and 19 and then verifies with zdb that the block was compressed with that level. I am still working on a solution for setting the zstd feature flag to 'active' as soon as it is set, rather than only once a block is born. As well as fixing up compatibility around zfs send/recv with the embedded block points flag. This project is sponsored by the FreeBSD Foundation. On 2020-07-06 20:07, Allan Jude wrote: > In my continuing effort to complete the integration of ZSTD into > OpenZFS, here is my third weekly status report: >=20 > https://github.com/allanjude/zfs/commit/87bb538bdbc4bb8848ed5791b7b0de8= 4a026ebbe > - Completed the rewrite of the way the compression property is handled,= > moving away from the initial approach of storing the compression > property (enum zio_compress) and the level (uint64_t) separately. >=20 > Previously we exposed the list of compression algorithms and levels to > userland as the corresponding value from the enum in the lower 7 bits, > and the level in the remaining upper bits. Then, as part of the propert= y > GET and SET IOCTLs, we read the separate compression=3D and > compress_level=3D properties from the ZAP and returned the combined val= ue, > or split the combined value into those two separate properties. This > worked but caused a lot of headache around property inheritance. >=20 > Instead I've changed to doing the combine/split when reading/writing > from the dataset properties ZAP, via the compression_changed_cb() > function. So the properties ZAP contains the combined value (lower 7 > bits are the compression algorithm, as defined in the enum zio_compress= , > and the upper bits are the compression level). Elsewhere in ZFS we keep= > the two values separate (os_compress and os_complevel, and related > variables in all of the different parts of ZFS). >=20 > So now, inheritance of the property is handled correctly, and avoids > issues where a dataset with compression=3Dzstd-12, would say 'inherited= > from' a dataset with zstd at some other compression level (since both > actually just had compression=3Dzstd, but different compress_level=3D v= alues). >=20 >=20 > I have also further extended zdb to inspect the compression settings > when looking at an object: > https://github.com/allanjude/zfs/commit/3fef3c83b8ce90149110ed989bd9fd3= e289798e0 >=20 >=20 > I am still working on a solution for setting the zstd feature flag to > 'active' as soon as it is set, rather than only once a block is born. >=20 >=20 > Additionally, I am investigating how to best handle the fact that > embedded block pointers compressed with ZSTD will make 'zfs send -e' > streams backwards incompatible, without a way for the user to opt-out o= f > sending a stream that contains zstd compressed blocks that the receivin= g > side may not be able to read. The same can be said for 'zfs send -c' as= > well. I am open to ideas on how best to handle this. I have thought > about only sending ZSTD compressed blocks if the user specifies the -Z > (--zstd) flag, but this can lead to confusion where using -c without -Z= > would have to either error out, or send the ZSTD compressed blocks > uncompressed. >=20 >=20 > This project is sponsored by the FreeBSD Foundation. >=20 --=20 Allan Jude --66UlQ09ydSSLlog0hd3djPINvizOVW1Iw-- --11rZcGtLSc3Dz7SCZJTTfFXDO0W11n4If Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (MingW32) iQIcBAEBAgAGBQJfDmk9AAoJEBmVNT4SmAt+2/wP/RjlyPLcqL7mOplPW54P2uuN q6xgpWvk2odbZz6LOuG4qvo45+WU27rOJpDc1M/U8vs++kYHNfPt/CO9IjHkgMod o7FPPri+evJhnxpoIuK7c7wKfgUvyyzzxgT1MRfVjH0G3f/323Z22Bu5tXYPaN5d PnhCRQlcZyiaXwYneh1eTBFcXh1g3v0mUdm4o0gSrpAJCWCUDyx/6J5AKlZQRAnW kAUXOHrQ7vmA8uZeo6A4hkLiVKszLpTEeDuSxyTWpEXboUUyQYyTGvGAkc1xfLFO DGTMO66hjIfmdkmJAd4Pq8hx4GorX2gZWx21nWxpaqHdDnSj5fLCP55+zrL3FqEt E8IOUzxV9IbRW2IQAuPU2a2/BUEe7oOsaiIpDflb+qi0TGsn9rAGHvBLdKvwmK/f 1Gudt64woGHAVYd2ikCV0NdmwWxTw0r3YULbC7FhGRGMsWMBzWrh4QzkUNkcnVT1 MAV+NjewPXAKe+BIOEQJnxR3xT+/SCh0ZRkZ3dI9jgLPRpgyURzGDXkDajP9IB56 /3KV4vtpZwT3MplCpEKi9qpi7pfPqOCbAzZKFRt9tECnJ0Qv7cP0rb3JxtLEqhgv /Hopl/cjoum2EpLoSl60OlQnUfRStGRIRc/mp/5k/yYm9oR2EE+DUh4ojSGEhqZY eJyDc+7hMFnwh66N9K4U =ge0w -----END PGP SIGNATURE----- --11rZcGtLSc3Dz7SCZJTTfFXDO0W11n4If--