From owner-freebsd-fs@freebsd.org Sat Mar 14 12:40:10 2020 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 7EAE525895F for ; Sat, 14 Mar 2020 12:40:10 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from mail.lysator.liu.se (mail.lysator.liu.se [130.236.254.3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 48fhwk6Bh0z4YB1 for ; Sat, 14 Mar 2020 12:40:06 +0000 (UTC) (envelope-from pen@lysator.liu.se) Received: from mail.lysator.liu.se (localhost [127.0.0.1]) by mail.lysator.liu.se (Postfix) with ESMTP id 6A98840006 for ; Sat, 14 Mar 2020 13:40:02 +0100 (CET) Received: by mail.lysator.liu.se (Postfix, from userid 1004) id 5766840003; Sat, 14 Mar 2020 13:40:02 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on bernadotte.lysator.liu.se X-Spam-Level: X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED, AWL, HTML_MESSAGE autolearn=disabled version=3.4.2 X-Spam-Score: -1.0 Received: from [IPv6:2001:9b1:28ff:d901:ece3:3af1:128f:2a35] (unknown [IPv6:2001:9b1:28ff:d901:ece3:3af1:128f:2a35]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.lysator.liu.se (Postfix) with ESMTPSA id 6A72640002; Sat, 14 Mar 2020 13:40:01 +0100 (CET) From: Peter Eriksson Message-Id: <2137BCAA-03B7-4DFB-B4F3-B0B26A56D416@lysator.liu.se> Mime-Version: 1.0 (Mac OS X Mail 13.0 \(3608.60.0.2.5\)) Subject: Re: ZFS pools in "trouble" Date: Sat, 14 Mar 2020 13:40:00 +0100 In-Reply-To: <15bde4a5-0a2e-9984-dfd6-fce39f079f52@digiware.nl> Cc: FreeBSD Filesystems To: Willem Jan Withagen References: <71e1f22a-1261-67d9-e41d-0f326bf81469@digiware.nl> <91e1cd09-b6b8-f107-537f-ae2755aba087@FreeBSD.org> <15bde4a5-0a2e-9984-dfd6-fce39f079f52@digiware.nl> X-Mailer: Apple Mail (2.3608.60.0.2.5) X-Virus-Scanned: ClamAV using ClamSMTP X-Rspamd-Queue-Id: 48fhwk6Bh0z4YB1 X-Spamd-Bar: ----- Authentication-Results: mx1.freebsd.org; dkim=none; dmarc=pass (policy=none) header.from=liu.se; spf=pass (mx1.freebsd.org: domain of pen@lysator.liu.se designates 130.236.254.3 as permitted sender) smtp.mailfrom=pen@lysator.liu.se X-Spamd-Result: default: False [-5.63 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+a:mail.lysator.liu.se]; MV_CASE(0.50)[]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-fs@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCVD_COUNT_THREE(0.00)[4]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; RCVD_IN_DNSWL_MED(-0.20)[3.254.236.130.list.dnswl.org : 127.0.11.2]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[liu.se,none]; RCVD_TLS_LAST(0.00)[]; FROM_EQ_ENVFROM(0.00)[]; R_DKIM_NA(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; ASN(0.00)[asn:2843, ipnet:130.236.0.0/16, country:SE]; MID_RHS_MATCH_FROM(0.00)[]; IP_SCORE(-3.13)[ip: (-8.11), ipnet: 130.236.0.0/16(-4.17), asn: 2843(-3.34), country: SE(-0.03)] Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.29 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 14 Mar 2020 12:40:10 -0000 That probably corresponds to: ZPOOL_STATUS_BAD_GUID_SUM /* sum of device guids didn't match */ (=46rom /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs.h) Which get set in:=20 /usr/src/cddl/contrib/opensolaris/lib/libzfs/common/libzfs_status.c /* = =20 * Check that the config is complete. = =20 */ if (vs->vs_state =3D=3D VDEV_STATE_CANT_OPEN && vs->vs_aux =3D=3D VDEV_AUX_BAD_GUID_SUM) return (ZPOOL_STATUS_BAD_GUID_SUM); /usr/src/sys # find . -name '*.c' -print0 | xargs -0 fgrep = VDEV_AUX_BAD_GUID_SUM ./cddl/contrib/opensolaris/uts/common/fs/zfs/vdev_missing.c: * = VDEV_AUX_BAD_GUID_SUM. So we pretend to succeed, knowing that we ./cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c: return = (spa_vdev_err(rvd, VDEV_AUX_BAD_GUID_SUM, ENXIO)); ./cddl/contrib/opensolaris/uts/common/fs/zfs/spa.c: return = (spa_vdev_err(rvd, VDEV_AUX_BAD_GUID_SUM, ./cddl/contrib/opensolaris/uts/common/fs/zfs/vdev.c: = case VDEV_AUX_BAD_GUID_SUM: error =3D spa_check_for_missing_logs(spa); if (error !=3D 0) return (spa_vdev_err(rvd, VDEV_AUX_BAD_GUID_SUM, = ENXIO)); if (rvd->vdev_guid_sum !=3D spa->spa_uberblock.ub_guid_sum) { spa_load_failed(spa, "uberblock guid sum doesn't match = MOS " "guid sum (%llu !=3D %llu)", (u_longlong_t)spa->spa_uberblock.ub_guid_sum, (u_longlong_t)rvd->vdev_guid_sum); return (spa_vdev_err(rvd, VDEV_AUX_BAD_GUID_SUM, ENXIO)); } Missing the logs perhaps? - Peter >>> */ >>> assert(reason =3D=3D ZPOOL_STATUS_OK); >>>=20 >=20 > Error: Reason not found: 5