From owner-freebsd-current@freebsd.org Sun Jun 7 01:53:21 2020 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id BF68B344DF5 for ; Sun, 7 Jun 2020 01:53:21 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from anubis.delphij.net (anubis.delphij.net [64.62.153.212]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "anubis.delphij.net", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49ffYF29ycz4Rfj; Sun, 7 Jun 2020 01:53:21 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from p51.home.us.delphij.net (unknown [IPv6:2601:646:8600:58ba:e670:b8ff:fe5c:4e69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange ECDHE (P-384) server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) by anubis.delphij.net (Postfix) with ESMTPSA id 1ABCF42E43; Sat, 6 Jun 2020 18:53:19 -0700 (PDT) Reply-To: d@delphij.net To: cem@freebsd.org Cc: Xin LI , freebsd-current References: <202006051612.055GCL11009491@repo.freebsd.org> From: Xin Li Autocrypt: addr=delphij@delphij.net; keydata= mQINBFuSR4oBEACvvEgwRIHs6IcSP/yaDtySF78Ji3rP29qdiQsxhMsOtvtffdbS56VApIWO UFb3/iN2gA8HwLvrmjijN0HEoLVX7na1WARmxRYzQMtApsZIUTtx7hnUYlsi2F5odZa6CDW9 a954DLRzYxiUwYDcu5Zjl9bglK1H8e/N9uC0Vuigr4teWfh86brzOyf819QzwFVYfMIK4ihw QGwMvTzbyVuCFy+LENkmcVYni70oQy6rZ5ktSuYbuOFvu7inRRfhSWPHziV7k+bW88sJ7xhv lBlegcnhkSudWX2M8tZ3MO1PJOcyys0CJlsBY5Weiog2lIPi05h/E9pZ9mc1Vud17iqDaL6w RaggOUhuPfDGCdO5ro82W4BZGeQMRnRF5Ntk+t2ShIH4nn3xRLV0E5nziCiKlgiMqOrz/ZTL QTVbHrCuiwD+fSK14y0oHbkOLYTYLlgh1JbwfY2Ty7elOYiWzyeJ7sJh2dF91NSEneWIOys3 mBpuvtU3nSzzTvAB48VV+Nbg1CpIOgNlPjj7uhIum/Z/VjUaJEyaLpTIRh0MVJVcbP7hXSqZ NA35EEZZVnWEOYdycm4CmEdeNPWkrAf2Ya77iR5VLGypwMlsUMQPh+sKVWDD38M8stFGBBNm d01Hi74Bsq5hKan654dOqMt5eYklrVj0ucMzFQtus7oE502UswARAQABtBxYaW4gTEkgPGRl bHBoaWpAZGVscGhpai5uZXQ+iQJUBBMBCgA+FiEEceNg5NEMZIki80nQQHl/fJX0g08FAluS R/YCGwMFCQmuhAAFCwkIBwMFFQoJCAsFFgIDAQACHgECF4AACgkQQHl/fJX0g0+2Og//bWpE F2V5/M5l6YW1T8oLcT9rIOH6oq9M0LMNRgFeiNNnilGIeeIgtOGBRueG4CZiZAvsRPJkrO70 1R2SrdkCIvwGUzUAxx1NfBWb+vgm4fgkW/MotGonceM5v0qfSKKXasWvDctkK28aG+IoQzmi FjXNW4+ju4zeQFYwD4ZDWqw9MqO0hVb24uW3dxtQhbfmOLgJ/PEDMQaFuANbW1c+iR0BQA3D Go/EeMY4kpN8on6Aqt/S/4JVltudfQ9OXdjQsC7netSaB9K3mHGt9aKAAB7RzlRY00DKkYS/ /eQwLzGPmK7yX13M68mMDjBs6mIR8t/E1S5OdBNhHRPNPlEbwugR4KaiCsN5yqzJoSV99fKY z2VyxjWPaG8yhHE+jmKUgIBKTfFUQEfkriQR4EASoeJ+soaMTiFDBij1Zw5n3ndLRFMB1ZCl fZLER36mAgW4m4kP83TWnDiJLxOxSOxifV8HpTFjff902H85cybg9KMwrfPDr6W19GGk5Vo1 fkza5krRMGbKWb7+74Evusi0ZxJLIOFwp5Y8eVqUMZaAD3f1ZX1M3pgXOp20QgAy+2KvMHij rLa4q+tMGRzYYD1BnFVSVdXAX5VOoTmHBcDz67DkuRwk2Byp1sgd407oEOmSwrNJlKS0TPCm xUJ2fdSQF+1/MMSRfee49vtMvz7cOrC5Ag0EW5JHigEQANiBmIFAfRNH3nzYNWC0yC+tfx3z sUwAsH1VaBM/cTib+yKtbBOSIlXWjJZWX3MHwoI/1LeGghB2mxkkX1L0pJ/vj1eXNR+sFZ32 0pYcl61Fxg/5fioG4QDTM4i3i7NR5PxDnc6UVaynSlII93DedRhZ1ROtdn4vyMgzsDiqhbL7 BthDOt5KxjqdRk4qRPSw7BovEqZLOcG5IJtf/zZUzRbM7SBljEbOAfekDGx1Br+RrYSD7/Ef Pwwzou9T8315IpBpIHyQF/dZNk3iFiB9Ed5CA71ZRYV5YoLWE9lL0j9kxOLQ5vHnX3mVq7QZ Bc7nzwZ6UhQgYmrG5+RWvuiPpGwvDRIsugJUGXucYkAQh5kuNblmkwpv6u9rNMjCNbzAylOa qdogra5EW+RUSbRz0b4iIr8nnZeAlh7BihCe7JjOwbDjoBEEEtSfVc4hD/LENqpcYVrChphf aOLB9YIXhnVDTVvMc9OklWT/81HzAaDQqOQCzEfY92199Ct9/CwRoQ2OpO8TO5+8A7b9Nb33 nmxMn09mb48ruRacMrfHxCWbgU4w9SEfbip4GcS5wGG6yTC+hw55Iwnnwus40NrJ0GEr8a4r cdsLbkvlyoNHB8ZGgyJ4aFCQ1V4qE1BnlTk7Z8BYBUkJM1odPSkVvHpCnMUjVpJ3hEOC+73Z YH1dh7lZABEBAAGJAjwEGAEKACYWIQRx42Dk0QxkiSLzSdBAeX98lfSDTwUCW5JHigIbDAUJ Ca6EAAAKCRBAeX98lfSDTz8DEACMh3poeUb+gWNF4RWFZuLteZVo0+E1JLYXQkmtrRBLXviP +Qy0pXyFAVxLM4hNIBoIDYfK9BcwrBYf7AwSKrH0GiNwFpgHCkbZd6qoZy2gB+adTnCpVCTJ KJetsH/8awkrChJWMK0ckGf3EeWMPvawG7kW7FBz70NYEZ0pOMiaEZNVtzD3wwbYWUiDFYth 83XGglOExg+1ShTW5XjQPRrdyJAO+aUW4o3lVjfyUJXMgI4rmhMiLVm06GuNrbpKIF0s+4Vd jQAjhrDQjfoXi9CkfsA/cONseuHNv1JGj3RqHiqHJq1dbrpodXp925zGDAnUGxCOBPoFopAH gVzR89GTut059GpwqsddZmU6y7rqifuam/ekJ+QRwc16vgt7pHqCrTY8WPxRZr2UpFU1wlTo COdeiFep1gq1F9jzFjJnoMaAdmC6k7bgAA+RQusOgIhJL0jIej7DoAHxmxFFCfRy+lDtpXwF gQ8HMvzHI65QWmQnMo7s6SQH/ZH5s1yR6SJq8+3lDz+dCuT42qJVqIPVvxd10LW0FNN+t7HF eLadU6ekSgD13/EYMYXlvNHkw7dAItSDxIzgRyykLz0bCU9xwNWoS4Z43+ifF9anJ+uR0ltW El1j++h6ZrD3LLuCgJIt1so0m49GzdcSpOI7LCwMlacyvafiEyjUn+tSNDsnfw== Organization: The FreeBSD Project Subject: Re: HEADSUP: GEOM label may be broken [Was Re: svn commit: r361838 - in head/sys/geom: . label] Message-ID: Date: Sat, 6 Jun 2020 18:53:06 -0700 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="yrAQ59uXIrlyXcTHYZzInVGjcg6wUB4bq" X-Rspamd-Queue-Id: 49ffYF29ycz4Rfj X-Spamd-Bar: ---- X-Spamd-Result: default: False [-4.00 / 15.00]; ASN(0.00)[asn:6939, ipnet:64.62.128.0/18, country:US]; REPLY(-4.00)[] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 07 Jun 2020 01:53:21 -0000 This is an OpenPGP/MIME signed message (RFC 4880 and 3156) --yrAQ59uXIrlyXcTHYZzInVGjcg6wUB4bq Content-Type: multipart/mixed; boundary="qFGLwCaSkXN0fQsuCYx3MtJykTa8S4tqK"; protected-headers="v1" From: Xin Li Reply-To: d@delphij.net To: cem@freebsd.org Cc: Xin LI , freebsd-current Message-ID: Subject: Re: HEADSUP: GEOM label may be broken [Was Re: svn commit: r361838 - in head/sys/geom: . label] References: <202006051612.055GCL11009491@repo.freebsd.org> In-Reply-To: --qFGLwCaSkXN0fQsuCYx3MtJykTa8S4tqK Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: quoted-printable Hi, Conrad, On 6/6/20 9:28 AM, Conrad Meyer wrote: > # geli attach /dev/gpt/testingeli > Enter passphrase: > GEOM_ELI: Device md0s1.eli created. > GEOM_ELI: Encryption: AES-XTS 128 > GEOM_ELI: Crypto: software > GEOM_LABEL[2]: Tasting md0s1.eli. > # ls -l /dev/gpt/testingeli* > lrwxr-xr-x 1 root wheel 8 Jun 6 09:22 /dev/gpt/testingeli -> ../md0s1= > lrwxr-xr-x 1 root wheel 12 Jun 6 09:24 /dev/gpt/testingeli.eli -> ../m= d0s1.eli >=20 > https://reviews.freebsd.org/D25168 Thanks for the quick fix! I have applied D24968 and D25168 over r361880 (with some minor unrelated local changes) and can confirm that the gptid label + geli worked fine no= w. However, it seems that the GPT partitions are not showing up under /dev/diskid (in your test 1 you are specifying the diskid label itself so it's not clear to me if they would; on my laptop, the diskid label is now showing up, but not the partitions associated with it (in your case, that would be /dev/diskid/DISK-BHYVE-2613-8EFD-BAF4p2 and /dev/diskid/DISK-BHYVE-2613-8EFD-BAF4p3). Could you please take a look at these too? (BTW. On boot I'm now seeing: g_dev_taste: make_dev_p() failed (gp->name=3Dada1, error=3D17) g_dev_taste: make_dev_p() failed (gp->name=3Dada1p2, error=3D17) which may be a result of ZFS's GEOM integration (ada1p2 is the root zpool); I can't really test the ZFS situation yet as I don't have a USB drive at hand and still waiting for my order to arrive). Cheers, > On Sat, Jun 6, 2020 at 8:52 AM Conrad Meyer wrote: >> >> Hi Xin Li, >> >> Thank you for the report and diagnosis. Sorry for the breakage. >> >> I have reverted the change and hope to address the issues you have ide= ntified. >> >> There seem to be two issues identified: >> 1. When ZFS attaches to a partition of a disk, the /dev/diskid label d= isappears. >> 2. geli(8) attach of symlink labels doesn't work >> >> (1) is interesting, as it does not seem to happen to UFS or swap consu= mers: >> >> # mount >> /dev/gpt/rootfs on / (ufs, local, noatime, soft-updates) >> # swapctl -l >> Device: 1024-blocks Used: >> /dev/vtbd0p2 1048576 0 >> # ls -l /dev/diskid/DISK-BHYVE-2613-8EFD-BAF4 /dev/gpt/{rootfs,swapfs}= >> lrwxr-xr-x 1 root wheel 8 May 22 20:18 >> /dev/diskid/DISK-BHYVE-2613-8EFD-BAF4 -> ../vtbd0 >> lrwxr-xr-x 1 root wheel 10 May 22 20:18 /dev/gpt/rootfs -> ../vtbd0p3 >> lrwxr-xr-x 1 root wheel 10 May 22 20:18 /dev/gpt/swapfs -> ../vtbd0p2 >> >> I wonder what is different in the ZFS case here. >> >> (2) >> >> # ls -l /dev/gpt/testingeli >> lrwxr-xr-x 1 root wheel 8 Jun 6 08:33 /dev/gpt/testingeli -> ../md0s1= >> # geli init /dev/gpt/testingeli >> Enter new passphrase: >> Reenter new passphrase: >> ... >> # geli attach /dev/gpt/testingeli >> Enter passphrase: >> geli: Provider gpt/testingeli is invalid. >> geli: There was an error with at least one provider. >> >> So at least the latter problem is straightforward to resolve. I have >> a patch I'm testing locally now, and will upload to phabricator >> shortly. >> >> Best regards, >> Conrad >> >> >> On Sat, Jun 6, 2020 at 1:17 AM Xin Li wrote: >>> >>> I just spent quite some time to revive my laptop. TL;DR: if you are >>> using /dev/diskid or /dev/gptid labels and GELI, please wait until >>> things settled. >>> >>> =3D=3D=3D >>> >>> On 6/5/20 9:12 AM, Conrad Meyer wrote: >>>> Author: cem >>>> Date: Fri Jun 5 16:12:21 2020 >>>> New Revision: 361838 >>>> URL: https://svnweb.freebsd.org/changeset/base/361838 >>>> >>>> Log: >>>> geom_label: Use provider aliasing to alias upstream geoms >>>> >>>> For synthetic aliases (just pseudonyms inferred from metadata like= GPT or >>>> UFS labels, GPT UUIDs, etc), use the GEOM provider aliasing system= to create >>>> a symlink to the real device instead of creating an independent de= vice. >>>> This makes it more clear which labels and devices correspond, and = we can >>>> safely have multiple labels to a single device accessed at once. >>>> >>>> The confusingly named geom_label on-disk construct continues to be= have >>>> identically to how it did before. >>>> >>>> This requires teaching GEOM's provider aliasing about the possibil= ity >>>> that aliases might be added later in time, and GEOM's devfs intera= ction >>>> layer not to worry about existing aliases during retaste. >>>> >>>> Discussed with: imp >>>> Relnotes: sure, if we don't end up reverting it >>> >>> This would break (and the effect can be quite persistent and hard to >>> repair, see explanation below) existing configuration as some GEOM >>> classes are not converted to support the new way of device >>> representation, so I'd like to request this change be reverted for no= w >>> until these are fixed. >>> >>> Consider the following configuration, where one have a hard drive >>> partitioned with GPT, like: >>> >>> =3D> 40 1953525088 ada1 GPT (932G) >>> 40 262144 1 efi (128M) >>> 262184 8388608 2 freebsd-zfs (4.0G) >>> 8650792 67108864 3 freebsd-swap (32G) >>> 75759656 1877765472 4 freebsd-zfs (895G) >>> >>> Now, the first ZFS pool is created as root pool. ZFS gets an exclusi= ve >>> hold of 'ada1p2' despite the pool is carefully created to use >>> /dev/diskid/p2 instead of ada1p2. >>> >>> ZFS writes the new device path to vdev label. For ZFS, this doesn't >>> matter much as it always checks the label. >>> >>> However, this will prevent GEOM from properly creating >>> /dev/diskid/. >>> >>> In order to prevent accidentally writing data to wrong disk, for "raw= " >>> disk partitions it's usually a good idea to reference them with label= s, >>> either by /dev/diskid or /dev/gptid. With /dev/ada1p2 exclusively >>> accessed by ZFS, the /dev/diskid/ representing the disk i= s >>> now gone. >>> >>> And to make the situation even worse, simply changing the partition >>> reference to the corresponding /dev/gptid/ doesn't really work >>> either, when one is encrypting partitions individually. In the examp= le >>> above, adap4 is an encrypted partition and with the alias change, one= >>> can no longer "geli attach" them via /dev/gptid/. They can sti= ll >>> be attached by the "canonical" path (/dev/ada1p4), but for the swap >>> partition that would completely defeat the purpose of using label (to= >>> prevent accidentally writing to the wrong disk). >>> >>> For my case, reverting to an older kernel is not sufficient to fix th= e >>> configuration, because ZFS is recording the "canonical" device path >>> (/dev/ada1p2) and /dev/diskid label for this disk disappeared somewha= t >>> permanently (I would have to find a USB drive somewhere to fix the ro= ot >>> pool to use the "right" device path). >>> >>>> Differential Revision: https://reviews.freebsd.org/D24968 >>> [...] >>>> Modified: head/sys/geom/label/g_label.c >>>> =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D >>>> --- head/sys/geom/label/g_label.c Fri Jun 5 16:05:09 2020 = (r361837) >>>> +++ head/sys/geom/label/g_label.c Fri Jun 5 16:12:21 2020 = (r361838) >>>> @@ -344,18 +345,16 @@ g_label_taste(struct g_class *mp, struct g_pro= vider *p >>>> { >>>> struct g_label_metadata md; >>>> struct g_consumer *cp; >>>> + struct g_class *clsp; >>>> struct g_geom *gp; >>>> int i; >>>> + bool changed; >>>> >>>> g_trace(G_T_TOPOLOGY, "%s(%s, %s)", __func__, mp->name, pp->na= me); >>>> g_topology_assert(); >>>> >>>> G_LABEL_DEBUG(2, "Tasting %s.", pp->name); >>>> >>>> - /* Skip providers that are already open for writing. */ >>>> - if (pp->acw > 0) >>>> - return (NULL); >>>> - >>>> if (strcmp(pp->geom->class->name, mp->name) =3D=3D 0) >>>> return (NULL); >>>> >>>> @@ -391,9 +390,16 @@ g_label_taste(struct g_class *mp, struct g_prov= ider *p >>>> if (md.md_provsize !=3D pp->mediasize) >>>> break; >>>> >>>> + /* Skip providers that are already open for writing. *= / >>>> + if (pp->acw > 0) { >>>> + g_access(cp, -1, 0, 0); >>>> + goto end; >>>> + } >>>> + >>> >>> (Is this still necessary when the eventual provider would be the real= one?) >>> >>> I think symlink aliasing is a the right direction but we need to be >>> really careful to not break existing and legitimate usage. Since thi= s >>> also breaks GELI when using with labels, I'd like to request that thi= s >>> change be backed out for now until the consumers are correctly fixed.= >>> >>> Cheers, >>> --qFGLwCaSkXN0fQsuCYx3MtJykTa8S4tqK-- --yrAQ59uXIrlyXcTHYZzInVGjcg6wUB4bq Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.2.20 (FreeBSD) iQIzBAEBCgAdFiEEceNg5NEMZIki80nQQHl/fJX0g08FAl7cSI0ACgkQQHl/fJX0 g0/TvQ//dG91Wy1UtbQsUxXF3dUs8G3DZSEnnwRF2ZaPjHD6IBEsF8KTm7FfTqB2 kfZUIkeWf+DOLVJ2yUkT2MxHfFmEv6WOZ9KC5D+9t3eIHeVOUYbgCPsyTEWrGZHD xzIaQbwcPiFoYdOfh37/uDim+raeTo+LUcvoQ5jeZ0nkMO69o2w3fwwVpG6Jo89n plK78VgvN2nbpOQw7kcIX+8j7kzLxjSDPddSFBmntjg8GtS1tDvVaVHTsQ13Lqwu m0dJCaXUSoKcFx+AHc/ehFSwlPo6g6xadaT5OUgIZwP3vSdMIa8s3I+zpXbtfkRo Wq+8+fUQxFIyS2ezIN6fTaHnapBUlcjI8zOtV7qA3uMKuo7T8RaWrH7+RHQx5oC4 rS79wKpJJfmtzElG0R+UGdCJ471JoVQwIax14IP2PdYJ1BNuC/kDYE6G4FFvt0EE zSLECFp8ck/vwQ9GtiHsQ8+QJqSpTydOyixQGDRbIhlr7edY6qhsheuuBDQqaSEI kjGDIfbLI/xopSCH8JbIuG3U/wL9J3jIq6t3hm9GaEMuVYoRIHGN1o6okNz/qywa foqhux4Yfl9t2mBsyuOUiAYiC38gGEOy7FmQ/D1IFZufEvn9l2IM29fCQ6+6spv5 5s05ejqQYvv8pNXuRF+PqBXV8NDi7MTh6eUrIXlNzwlk2PxVA4k= =Aorf -----END PGP SIGNATURE----- --yrAQ59uXIrlyXcTHYZzInVGjcg6wUB4bq--