From owner-freebsd-fs@freebsd.org Mon Jul 13 18:02:58 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C0BE936C4F5 for ; Mon, 13 Jul 2020 18:02:58 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from mx1.cksoft.de (mx1.cksoft.de [IPv6:2001:67c:24f8:1::25:1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mx1.cksoft.de", Issuer "CA" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4B5BMQ2lPwz4Jy8; Mon, 13 Jul 2020 18:02:58 +0000 (UTC) (envelope-from ck-lists@cksoft.de) Received: from m.cksoft.de (m.cksoft.de [195.88.109.48]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.cksoft.de (Postfix) with ESMTPSA id 889DF1EAAF1; Mon, 13 Jul 2020 20:02:55 +0200 (CEST) Received: from amavisfra1.cksoft.de (unknown [IPv6:2001:67c:24f8:2003::25:a1]) by m.cksoft.de (Postfix) with ESMTP id 3432E63027; Mon, 13 Jul 2020 20:02:55 +0200 (CEST) X-Virus-Scanned: amavisd-new at cksoft.de Received: from m.cksoft.de ([192.168.35.42]) by amavisfra1.cksoft.de (amavisfra1.cksoft.de [192.168.35.41]) (amavisd-new, port 10051) with ESMTP id yr1viylq0lqm; Mon, 13 Jul 2020 20:02:52 +0200 (CEST) Received: from nocfra1.cksoft.de (nocfra1.cksoft.de [IPv6:2001:67c:24f8:2001::53:1]) by m.cksoft.de (Postfix) with ESMTP id EE44C63026; Mon, 13 Jul 2020 20:02:53 +0200 (CEST) Received: by nocfra1.cksoft.de (Postfix, from userid 1000) id 0132313EBA; Mon, 13 Jul 2020 20:02:53 +0200 (CEST) Received: from localhost (localhost [127.0.0.1]) by nocfra1.cksoft.de (Postfix) with ESMTP id EF41313EB7; Mon, 13 Jul 2020 20:02:53 +0200 (CEST) Date: Mon, 13 Jul 2020 20:02:53 +0200 (CEST) From: Christian Kratzer X-X-Sender: ck@nocfra1.cksoft.de Reply-To: Christian Kratzer To: Allan Jude cc: freebsd-fs@freebsd.org Subject: Re: gptzfsboot targeting wrong vdev In-Reply-To: Message-ID: References: <9400f5f0-e267-932c-b1ce-8436748cf2c0@FreeBSD.org> <78024f0d-4889-713e-15a5-56ec6d8d82b3@freebsd.org> User-Agent: Alpine 2.22 (BSF 395 2020-01-19) X-NCC-RegID: de.cksoft X-Spammer-Kill-Ratio: 75% MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed X-Rspamd-Queue-Id: 4B5BMQ2lPwz4Jy8 X-Spamd-Bar: ---- Authentication-Results: mx1.freebsd.org; none X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:57407, ipnet:2001:67c:24f8::/48, country:DE] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Jul 2020 18:02:58 -0000 Hi, On Mon, 13 Jul 2020, Allan Jude wrote: > bootme is only used by gptboot, for UFS. ok. > You might want to example your disks with 'zdb -l /dev/ada0' (for each > disk, and each partition). It seems there is something not encrypted > somewhere. It may be an old label from a pool that has been destroyed, > and a new pool with the same name was later created or something. let me provide the partitions up front root@zfs1:/home/ck # gpart show /dev/ada0 => 40 468862048 ada0 GPT (224G) 40 1024 1 freebsd-boot (512K) 1064 134217728 2 freebsd-swap (64G) 134218792 33554432 3 freebsd-vinum (16G) 167773224 33554432 4 freebsd-vinum (16G) 201327656 267534424 5 freebsd-zfs [bootme] (128G) 468862080 8 - free - (4.0K) root@zfs1:/home/ck # ada1 is essentially the same. These are the two m2 ssd I have zroot und zil on. The two freebsd-vinum partitions used to be freebsd-zfs Partitions 3 and 4 are used for the log device and these happen to have the names I was getting confused about root@zfs1:/home/ck # zdb -l /dev/ada0p3 | head -10 ------------------------------------ LABEL 0 ------------------------------------ version: 5000 name: 'zp1' state: 0 txg: 37392274 pool_guid: 6725164972097052508 hostid: 694021601 hostname: 'zfs1' root@zfs1:/home/ck # root@zfs1:/home/ck # zdb -l /dev/ada0p4 | head -10 ------------------------------------ LABEL 0 ------------------------------------ version: 5000 name: 'zp2' state: 0 txg: 21037796 pool_guid: 16088850666393843588 hostid: 694021601 hostname: 'zfs1' root@zfs1:/home/ck # It is just a name and not a label. gptzfsboot was trying to tell me it cannot boot from ada0p3 which has the name zp1. My two other pools on the sas hba happen to be called zp1 and zp2. For some reason when I setup the system I moved stuff around and ended up with two zlog partitions first and the zroot last. So now everything makes sense again. The freebsd-vinum hack solves this for me and I have gained some more experience. In the future I will ensure to have the first partition after boot to be the zroot. Greetings Christian -- Christian Kratzer CK Software GmbH Email: ck@cksoft.de Wildberger Weg 24/2 Phone: +49 7032 893 997 - 0 D-71126 Gaeufelden Fax: +49 7032 893 997 - 9 HRB 245288, Amtsgericht Stuttgart Mobile: +49 171 1947 843 Geschaeftsfuehrer: Christian Kratzer Web: http://www.cksoft.de/