From owner-freebsd-stable@freebsd.org Mon Aug 24 09:32:13 2020 Return-Path: Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D5A9B3B7CB5 for ; Mon, 24 Aug 2020 09:32:13 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from vtr.rulingia.com (vtr.rulingia.com [IPv6:2001:19f0:5801:ebe:5400:1ff:fe53:30fd]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA512 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "vtr.rulingia.com", Issuer "Let's Encrypt Authority X3" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BZn2h2MlWz4Td6 for ; Mon, 24 Aug 2020 09:32:11 +0000 (UTC) (envelope-from peter@rulingia.com) Received: from server.rulingia.com (ppp239-208.static.internode.on.net [59.167.239.208]) by vtr.rulingia.com (8.15.2/8.15.2) with ESMTPS id 07O9VuXs019966 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 24 Aug 2020 19:32:02 +1000 (AEST) (envelope-from peter@rulingia.com) DKIM-Filter: OpenDKIM Filter v2.10.3 vtr.rulingia.com 07O9VuXs019966 X-Bogosity: Ham, spamicity=0.000000 Received: from server.rulingia.com (localhost.rulingia.com [127.0.0.1]) by server.rulingia.com (8.16.1/8.16.1) with ESMTPS id 07O9VpwG067712 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO) for ; Mon, 24 Aug 2020 19:31:51 +1000 (AEST) (envelope-from peter@server.rulingia.com) Received: (from peter@localhost) by server.rulingia.com (8.16.1/8.16.1/Submit) id 07O9Voej067711 for freebsd-stable@freebsd.org; Mon, 24 Aug 2020 19:31:50 +1000 (AEST) (envelope-from peter) Date: Mon, 24 Aug 2020 19:31:50 +1000 From: Peter Jeremy To: freebsd-stable@freebsd.org Subject: Re: svn commit: r362848 - in stable/12/sys: net netinet sys Message-ID: <20200824093150.GA66115@server.rulingia.com> References: <202007011803.061I3cTs089322@repo.freebsd.org> <20200719112102.GA15535@server.rulingia.com> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha512; protocol="application/pgp-signature"; boundary="Kj7319i9nmIyA2yE" Content-Disposition: inline In-Reply-To: <20200719112102.GA15535@server.rulingia.com> X-PGP-Key: http://www.rulingia.com/keys/peter.pgp X-Rspamd-Queue-Id: 4BZn2h2MlWz4Td6 X-Spamd-Bar: --- X-Spamd-Result: default: False [-3.78 / 15.00]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[rulingia.com:s=default]; FREEFALL_USER(0.00)[peter]; FROM_HAS_DN(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; MIME_GOOD(-0.20)[multipart/signed,text/plain]; TO_DN_NONE(0.00)[]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; RCPT_COUNT_ONE(0.00)[1]; RCVD_COUNT_THREE(0.00)[3]; DKIM_TRACE(0.00)[rulingia.com:+]; DMARC_POLICY_ALLOW(-0.50)[rulingia.com,quarantine]; NEURAL_HAM_SHORT(-0.68)[-0.683]; SIGNED_PGP(-2.00)[]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:20473, ipnet:2001:19f0:5800::/38, country:US]; RCVD_TLS_ALL(0.00)[]; MAILMAN_DEST(0.00)[freebsd-stable] X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Aug 2020 09:32:13 -0000 --Kj7319i9nmIyA2yE Content-Type: text/plain; charset=us-ascii Content-Disposition: inline Content-Transfer-Encoding: quoted-printable TL;DR: Ensure you explicitly destroy all ZFS labels on disused root pools. On 2020-Jul-19 21:21:02 +1000, Peter Jeremy wro= te: >I'm sending this to -stable, rather than the src groups because I >don't believe the problem is the commit itself, rather the commit >has uncovered a latent problem elsewhere. > >On 2020-Jul-01 18:03:38 +0000, Michael Tuexen wrote: >>Author: tuexen >>Date: Wed Jul 1 18:03:38 2020 >>New Revision: 362848 >>URL: https://svnweb.freebsd.org/changeset/base/362848 >> >>Log: >> MFC r353480: Use event handler in SCTP > >I have no idea how, but this update breaks booting amd64 for me (r362847 >works and this doesn't). I have a custom kernel with ZFS but no SCTP so I >have no real idea how this could break booting - presumably the >eventhandler change has uncovered a bug somewhere else. To close the loop on this, the problem was a combination of: * changes in GEOM provider ordering; * insufficient checks when ZFS is looking for the root pool; * my system having remnants of a disused pool with the same name as the roo= t poop. It seems that the order of GEOM providers is relatively unstable - even including a device, that doesn't physically exist, in a kernel can change the provider order. Presumably r362848 also resulted in a change in order. During a root-on-ZFS boot, the kernel scans all providers, looking for ZFS labels with a pool name matching the root pool. Only minimal checks are performed, in particular, there's no check that it's a valid pool, and the first such label found is assumed to describe the root pool. In my case, some time ago, I'd moved things around on my boot disk. My old root pool went to the end of the physical disk but I'd decided to shrink it and left some free space at the end of the disk. This meant that ZFS found one (out of 4) labels when it tasted the physical disk and if GEOM sorted the physical disk prior to its partitions then ZFS would use the pool GUIDs =66rom the stray label on the physical disk and then fail to find a usable pool matching those GUIDs. My fix was to zero the end of my disk. --=20 Peter Jeremy --Kj7319i9nmIyA2yE Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQKTBAEBCgB9FiEE2M6l8vfIeOACl4uUHZIUommfjLIFAl9DiPxfFIAAAAAALgAo aXNzdWVyLWZwckBub3RhdGlvbnMub3BlbnBncC5maWZ0aGhvcnNlbWFuLm5ldEQ4 Q0VBNUYyRjdDODc4RTAwMjk3OEI5NDFEOTIxNEEyNjk5RjhDQjIACgkQHZIUommf jLK1IA/+OGhdGUPxr0Mg81frybtekzDHYfAZ5IUfRuo8T3MQzQYLMBvaRLMxVq/Y y+dl0cPF5kmDBCa7E5tTBgKz8sRGgNUhAKXgjLSQekEPi5TEErDfThfrZ/FMymHV FUh2CFnMIrc/ZCq7CnTcUamw09KVnfzDkVBoqEA+jqH+p9LtJwPWH7tn+HIodAWn blYbF9/epqokOELzeCesj6yRt8hpKRBrycNq5nwLrrJo7UufPpN14aiWdkzbWlcq P3ulA1pQB4Mj6gCXkY8EyzkJ5QLlqXlNNos6A/uLMgGB1Cyrma2Sd2fU98hfEGEl t6cLbZCUyaJStWSW0xgweUStUjcwCZANUenUcE9vV0uugkf0yvU5GLl7JBeEXUNq fQ2CW29RZH9JDDiiu0QtxP/CzV2GTShTbYvw/iCkGGLiHqIPE8zBISaHawsgqn3Z w026Vwbnfms3r2KbalzxH5lTufuIxdwtyPAfD/gEzA3Dd43z2TbmfZfIXpd0dfSs mU/NW9QG+Vk2QSp3CYWKBVt8CEwBrRb84e8oivH1+ixXGtGsthKBmWYne4qia7vh m1StEgRsInrhkgaGcEYVwFf0t+7iur5eCYVk9MrNvT+5Nr3slmyoLhDsx/8u+5Oh jLhkpPNObBb9YIuuHLC1LUyqU2ovkd0LOaqk2cLg+AqJnUef5qo= =gn3Y -----END PGP SIGNATURE----- --Kj7319i9nmIyA2yE--