From nobody Fri Jan 5 17:28:55 2024 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4T69Rc6tZ6z55NxV; Fri, 5 Jan 2024 17:29:00 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from smtp.freebsd.org (smtp.freebsd.org [IPv6:2610:1c1:1:606c::24b:4]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "smtp.freebsd.org", Issuer "R3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4T69Rc6Jqjz56Jg; Fri, 5 Jan 2024 17:29:00 +0000 (UTC) (envelope-from lev@FreeBSD.org) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704475740; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9flr6SSfYF155xUHIMgvwpRLAu/oCTD80kkHszSIMkc=; b=h2Gm9janLRzJPwjOrlkEuARFDT1ogG7HG6C6YzlxYzrX0X8CAJa3Ksx9Ew8EhXmsugg/mi kBnBloPZMjuFI5WxeGTxWydBam4oAyGhgaHueJBzzMVFVLqT+9YTEm7yGwoUzZdW8jzwa9 5kaVu8D3jL1+rpvQmhT8VidBOl7EvwUwad32uEeYu2WBUst3XoXIJGBbIXJ9tQAZdFSx34 Y6gkEGK0he7d0o/Vsmjpk+fgWJPpWAms29sFCUEKGizXZckvFuOHeMizh2ncr5RE7C3v6e dQKEJC8YQgI/LVwdiSYFc01cRuaT7wqJKDYJmIS7l/IGxBC0hDYAYimsTcrSNg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=freebsd.org; s=dkim; t=1704475740; h=from:from:reply-to:reply-to:subject:subject:date:date: message-id:message-id:to:to:cc:mime-version:mime-version: content-type:content-type: content-transfer-encoding:content-transfer-encoding; bh=9flr6SSfYF155xUHIMgvwpRLAu/oCTD80kkHszSIMkc=; b=KbYU3d/Up4oiZMGymCAmitdGBecwsePuKlokgVJR9XD7F0mS8Qu5IyXMqRwDW2OuXgw4mQ +LMm4iSqRcO9clJSkW81f8lIbTFJayMXgFtFUbINyvIMw4/KFvZgnf+WWRFPsoGBxwXnqG 06jEHoZ46Bd0pAQa0fuFdB8CnaM+prUjpQbKLQp6hHEbmVThW7yXxmlU2nrLRfBeaB+XEM 07SgTlf9x2wambSJYnVt7aflQ+T3s/hX3/oiOoQTALo4mw/oJdgwHZO62cxx6/lgosGYy3 5leNeLj4lGJ9wXHQMlrKSiO1GFeTg4EreOm5orQGMzPYHQdhw7unyHypzNffBw== ARC-Authentication-Results: i=1; mx1.freebsd.org; none ARC-Seal: i=1; s=dkim; d=freebsd.org; t=1704475740; a=rsa-sha256; cv=none; b=dOvTpHrvAuB/fJgphZgQK7wEEzCMvViAOYTp4Iuo+6sG+cBC9K60IOmoOkZEDII+dVwM19 ayfs8hrDS+/LXz32mQBnEuZKn98UjkwCvrhN/e6uEFy8FsESTJ4HrxcdBHvi+fs8qwr7gD MxRk8XgYnK/cnWmgWLtnZnnX79BnuSK05xf9L8lOMqfKzC6cSql45oOx5J4Z7eLEBbw5Jr Qs6NLIvpjaVnGEr/qOAqHy3kYpNXuby5hOGH7cVDFSP7dKh/aZQOlhkCQXtzhCzCZNsYe4 Qco0tojjjp8UZWZkrb446gAigYtXbi1qA4pNADAKXG4W4sXc5+495sbWo9OWTw== Received: from onlyone.not-for.work (onlyone.not-for.work [IPv6:2a01:4f8:201:6350::2]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (Client did not present a certificate) (Authenticated sender: lev/mail) by smtp.freebsd.org (Postfix) with ESMTPSA id 4T69Rc4bYtzYZ7; Fri, 5 Jan 2024 17:29:00 +0000 (UTC) (envelope-from lev@FreeBSD.org) Received: from [IPV6:2001:470:923f:1:8c5d:e3a1:bb4c:7024] (unknown [IPv6:2001:470:923f:1:8c5d:e3a1:bb4c:7024]) (Authenticated sender: lev@serebryakov.spb.ru) by onlyone.not-for.work (Postfix) with ESMTPSA id 4287E15; Fri, 5 Jan 2024 20:28:56 +0300 (MSK) Message-ID: Date: Fri, 5 Jan 2024 18:28:55 +0100 List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 User-Agent: Mozilla Thunderbird To: freebsd-fs , freebsd-stable , Alexander Motin Content-Language: en-US From: Lev Serebryakov Reply-To: lev@FreeBSD.org Subject: FreeBSD 13.2-STABLE can not boot from damaged mirror AND pool stuck in "resilver" state even without new devices. Organization: FreeBSD Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hello! I have (remote) physical server with 2 SATA disks. These disks were partitioned with GPT into "freebsd-boot" (ada{0|1}p1, legacy one, not EFI), "freebsd-swap" (ada{0|1}p2) and "freebsd-zfs" (ada{0|1}p3). Both disks were 512/512 (it looks important). I have only one ZFS pool "zroot", mirror of "ada0p3" and "ada1p3". I have very fresh "gptzfsboot" on both "ada0p1" and "ada1p1". Now, ada0 failed. It was replaced by DC support with new disk, which is 512/4096. After that my server fails to boot, gtpzfsboot from second disk (ada1) reports several "zio_read error: 5" and ZFS: i/o error - all block copies unavailable ZFS: can't read MOS of pool zroot after that. I've booted to rescue Linux (unfortunately, there is NO rescue FreeBSD at Hetzner anymore), and Linux could import (degraded) pool no problem. But Linux has problems with detecting pool on partition, so I don't do nothing under Linux. I've checked "live" disk under Linux, though: it reads, SMART is clear, everything is Ok. I've booted FreeBSD 13.2 from installation ISO under qemu with physical devices as disks. Then I partitioned fresh HDD and started disk replacement in mirror. It worked, but resilver was unbearable slow. I stopped VM with FreeBSD to continue process after normal boot. NO LUCK. "zio_read error: 5", boot failed. Then I've overwrite ada0 (new disk) with FreeBSD memstick IMG and boot it - it can import pool from ada1p3 but, of course, resilver is stopped. I've removed all faulted components, effectivly converting mirror to "simple" device. But "zpool status" shows that there is resilver! And "gptzfsboot" still CAN NOT read this ZFS pool and find loader! Ok, I've converted swap to UFS boot form UFS. It works. It can use pool as root. But pool still is "reslivering". Now I have very strange situation: (1) I have ZFS pool with 1 device which says: % zpool status -v zroot pool: zroot state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Fri Jan 5 19:24:07 2024 750G scanned at 472B/s, 40.5G issued at 25B/s, 974G total 0B resilvered, 4.16% done, no estimated completion time config: NAME STATE READ WRITE CKSUM zroot ONLINE 0 0 0 ada1p3 ONLINE 0 0 0 errors: No known data errors % (2) gtpzfsboot from very this system version can not read this pool and bot from it (3) kernel can use this pool as source of root (and all other) filesystems. -- // Lev Serebryakov