Date: Wed, 21 Oct 2020 16:36:17 -0400 (EDT) From: Walter Cramer <wfc@mintsol.com> To: Cassiano Peixoto <peixotocassiano@gmail.com> Cc: freebsd-stable@freebsd.org Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 Message-ID: <20201021162554.K14777@mulder.mintsol.com> In-Reply-To: <CAJajdNU0fP7RVJuvZT7%2BuUTquRKSeRLVx9L7cBJ10XVJEZ5LOw@mail.gmail.com> References: <CAJajdNWgiKyd1jmX4uPogWU4Ty=OiEgDnQGaoLZUWgj_pr9WKw@mail.gmail.com> <CAJajdNXPhGj-mKj6R_N94Bfnhvm=nHr3RwfBm4qoTB3g_=6kGw@mail.gmail.com> <CAJajdNV0nGdhAm6cgs2SKMBsY=Hi1Azb8fA_0PFrLUuejCT8ag@mail.gmail.com> <fea719f4-8918-38e6-7566-d77e9a244ef0@sentex.net> <CAJajdNU9-pc9De0tDge=4HPaT1jc%2B7evfZSjCrcNMXqtSpDs2Q@mail.gmail.com> <20201021160923.T14777@mulder.mintsol.com> <CAJajdNU0fP7RVJuvZT7%2BuUTquRKSeRLVx9L7cBJ10XVJEZ5LOw@mail.gmail.com>
index | next in thread | previous in thread | raw e-mail
I very much doubt that there's any remotely sane way to re-partition da1 & da2 while they are in the pool. If you carefully edit the output of the 'gpart backup' command, you probably can use 'gpart restore' to partition da1 and da2. Read the 'gpart' man page very carefully, if you aren't familiar with that. Also, you'll probably want to install bootstrap code on da1 & da2 with 'gpart bootstrap'. -Walter On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > Walter, Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. But is there a way to make the partition of da1 and da2 since both are already inserted on pool? I think it's not allowed... Thanks. On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer <wfc@mintsol.com> wrote: > My guess - there is a work-around or two, but you'll face a lot more > grief, long-term, if you don't do things the right way (aka do a bunch of > re-install work) now. > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), to > see how the original disk is partitioned. Then duplicate that > partitioning on da1 and da2. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Hi guys, > > Thank your for your answer. > > @Ricchard First of all I didn't have a chance to run zpool upgrade, because > after the system update reboot i ran into the issue. > > @Walter and @mike Regarding making a partition, I never see any > recommendation about this, I've been always using the entire disk on a pool > to expand it. Until FreeBSD 11.2 it worked fine. > > Is there a workaround for this or should I reinstall the entire pool? > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa <mike@sentex.net> wrote: > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 is > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > boot info ? Generally its not recommended to use the entire disk as > > part of a pool. Create a partition scheme first > > > > gpart create -s gpt da1 > > > > gpart add -t freebsd-zfs da1 > > > > then add da1p1 to the pool > > > > ---Mike > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > Hi there, > > > > > > Anyone can help please? I've many servers with this same issue. Thanks > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > peixotocassiano@gmail.com> > > > wrote: > > > > > >> Hi there, > > >> > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > > added two more disks to expand my pool. BTW I already did it many time > with > > no issues. > > >> > > >> I ran: > > >> > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> status: Some supported features are not enabled on the pool. The pool > > can > > >> still be used, but some features are unavailable. > > >> action: Enable all features using 'zpool upgrade'. Once this is done, > > >> the pool may no longer be accessible by software that does not > > support > > >> the features. See zpool-features(7) for details. > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> > > >> # zpool add -f zroot da1 > > >> # zpool add -f zroot da2 > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> da1 ONLINE 0 0 0 > > >> da2 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> # reboot > > >> > > >> Then my system doesn’t boot anymore, i got the following error: > > >> > > >> gptzfsboot: error 4 lba 2038346899 > > >> gptzfsboot: error 4 lba 1361327267 > > >> /boot/config: -Dh > > >> > > >> BTX loader 1.00 BTX version is 1.02 > > >> Consoles: internal video/keyboard serial port > > >> BIOS drive A: is fd0 > > >> BIOS drive C: is disk0 > > >> BIOS drive D: is disk1 > > >> BIOS drive E: is disk2 > > >> BIOS drive F: is disk3 > > >> BIOS drive G: is disk4 > > >> BIOS drive H: is disk5 > > >> ZFS: i/o error - all block copies unavailable > > >> ZFS: failed to read pool zroot directory object > > >> BIOS 638kB/3143616kB available memory > > >> > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > >> > > >> Type '?' for list of commands, 'help' for more datailed help. > > >> OK > > >> > > >> I can import my pool with no problems using the lived, but I could not > > fix it. > > >> > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok that? > > >> > > >> Thanks. > > >> > > >> > > >> > > >> > > > _______________________________________________ > > > freebsd-stable@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > To unsubscribe, send any mail to " > freebsd-stable-unsubscribe@freebsd.org > > " > > > > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 21:50:46 2020 Return-Path: <owner-freebsd-stable@freebsd.org> Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6F2042E9A6 for <freebsd-stable@mailman.nyi.freebsd.org>; Wed, 21 Oct 2020 21:50:46 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGkh600hZz3ZcF for <freebsd-stable@freebsd.org>; Wed, 21 Oct 2020 21:50:45 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ed1-x52c.google.com with SMTP id t20so4043067edr.11 for <freebsd-stable@freebsd.org>; Wed, 21 Oct 2020 14:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s 161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=M0PcM0UELISwSLrr/9zSNpqPEhGMsMgb+jNX8AbXYZ0=; b=mRREEDuu7HmoUppzumxG5OWdWzyaQTrKcEkBK7MBjtGynRpgJGcPyfOPxY/0sAoqT5 vFR8nmhfGU5lfPnvCrcsvYGjH+rNb1dCh5nMWy/Bc/dCyG1/qsiH6DZBQhVW4M5AqsrM TbRv4QPuq3/CZiDsggFD3RkmvDhahqesJgvjGZd+SjcZLP3L8yAbKEobkx24Sah3kGN4 HktHUedb+KPVKNhAxcK5W0UpssNLGy5IvbD6/mWOd2PrKBzwGLXizzf3c5DlkJScpXpW xBnKvJj9meMtnAUZcCtYUsPvVQq7MFfwHsSI9EJ0w5WAyJ5lsc0iucGPV/Kk+8d62JvP m7wA=X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d 100.net; s 161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=M0PcM0UELISwSLrr/9zSNpqPEhGMsMgb+jNX8AbXYZ0=; b=LYH4b6VdkOOxgk74POJ7OvnNigIXMvVNsnXaYkzdTkF03i1c0B0hSsqSQ7Y2TwjmM5 AUib+PxMN96/+iMvryTsZudJaDhezOIR27CrzsLHLOCWbxU5dcdziz8h88Mf6rzetOvw ACMJmiOfG701XeEbDN+PdQ2NIYrtggK31HBPw00DyoQYD+YEGadBND4shLpHGWGCQdTV i+4h4vwVDCDJZRFxHG0fjF8J46n5qzJOMmaXMndzan++cpDUkoHk7WYW7Hbc280a6Yel 66NcR6OLG+XdB9Mp6+KpZQAHg6XPHPx5IcsWOfG48Jwjgs9yejR4o5qkBWpykkCCx+C5 nfjg=X-Gm-Message-State: AOAM5303xZDsvElisrB0QKjCTDMnh+Y7c5f5/CMne2J7EogTiRjv8Jtv Dt3S8D8CjDYE/qjXITH34ReS5JfxqjcMIua4ZhgX-Google-Smtp-Source: ABdhPJyU1T6wBB9So5SlzI1mYwZIZQxNapHE73D8o8YFNAgv27fQ18zNAhZ9wf3vqC3HTLHONimWYhx+HKzV0LKa9RoX-Received: by 2002:a05:6402:744:: with SMTP id p4mr4969539edy.190.1603317044291; Wed, 21 Oct 2020 14:50:44 -0700 (PDT) MIME-Version: 1.0 References: <CAJajdNWgiKyd1jmX4uPogWU4Ty=OiEgDnQGaoLZUWgj_pr9WKw@mail.gmail.com> <CAJajdNXPhGj-mKj6R_N94Bfnhvm=nHr3RwfBm4qoTB3g_=6kGw@mail.gmail.com> <CAJajdNV0nGdhAm6cgs2SKMBsY=Hi1Azb8fA_0PFrLUuejCT8ag@mail.gmail.com> <fea719f4-8918-38e6-7566-d77e9a244ef0@sentex.net> <CAJajdNU9-pc9De0tDge=4HPaT1jc+7evfZSjCrcNMXqtSpDs2Q@mail.gmail.com> <20201021160923.T14777@mulder.mintsol.com> <CAJajdNU0fP7RVJuvZT7+uUTquRKSeRLVx9L7cBJ10XVJEZ5LOw@mail.gmail.com> <20201021162554.K14777@mulder.mintsol.com> In-Reply-To: <20201021162554.K14777@mulder.mintsol.com> From: Cassiano Peixoto <peixotocassiano@gmail.com> Date: Wed, 21 Oct 2020 18:50:32 -0300 Message-ID: <CAJajdNUSHp4A2S_WwzC0tP-LiMYow3b4bs4OAtWBkFCyNDFA@mail.gmail.com> Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Walter Cramer <wfc@mintsol.com> Cc: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CGkh600hZz3ZcF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s 161025 header.b=mRREEDuu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::52c as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.53 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s 161025]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.004]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52c:from]; NEURAL_HAM_SHORT(-0.52)[-0.519]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>; List-Post: <mailto:freebsd-stable@freebsd.org> List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>, <mailto:freebsd-stable-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 21 Oct 2020 21:50:47 -0000 Thanks for the tip! I've fixed the issue following the following steps: 1) added two new disks with the same size: da3 and da4 2) Made a partition for both: gpart create -s gpt da3 gpart create -s gpt da4 gpart add -t freebsd-zfs da3 gpart add -t freebsd-zfs da4 3) replace both disks on pool zpool replace zroot da2 da3p1 zpool replace zroot da3 da4p1 4) rebuild bootcode gpart boocode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 Done! I rebooted and everything is working again. Thank you guys for the support. On Wed, Oct 21, 2020 at 5:36 PM Walter Cramer <wfc@mintsol.com> wrote: > I very much doubt that there's any remotely sane way to re-partition da1 & > da2 while they are in the pool. > > If you carefully edit the output of the 'gpart backup' command, you > probably can use 'gpart restore' to partition da1 and da2. Read the > 'gpart' man page very carefully, if you aren't familiar with that. Also, > you'll probably want to install bootstrap code on da1 & da2 with 'gpart > bootstrap'. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Walter, > > Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. > But is there a way to make the partition of da1 and da2 since both are > already inserted on pool? I think it's not allowed... > > Thanks. > > > On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer <wfc@mintsol.com> wrote: > > > My guess - there is a work-around or two, but you'll face a lot more > > grief, long-term, if you don't do things the right way (aka do a bunch of > > re-install work) now. > > > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), to > > see how the original disk is partitioned. Then duplicate that > > partitioning on da1 and da2. > > > > -Walter > > > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > > > Hi guys, > > > > Thank your for your answer. > > > > @Ricchard First of all I didn't have a chance to run zpool upgrade, > because > > after the system update reboot i ran into the issue. > > > > @Walter and @mike Regarding making a partition, I never see any > > recommendation about this, I've been always using the entire disk on a > pool > > to expand it. Until FreeBSD 11.2 it worked fine. > > > > Is there a workaround for this or should I reinstall the entire pool? > > > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa <mike@sentex.net> wrote: > > > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 > is > > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > > boot info ? Generally its not recommended to use the entire disk as > > > part of a pool. Create a partition scheme first > > > > > > gpart create -s gpt da1 > > > > > > gpart add -t freebsd-zfs da1 > > > > > > then add da1p1 to the pool > > > > > > ---Mike > > > > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > > Hi there, > > > > > > > > Anyone can help please? I've many servers with this same issue. > Thanks > > > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > > peixotocassiano@gmail.com> > > > > wrote: > > > > > > > >> Hi there, > > > >> > > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > > > added two more disks to expand my pool. BTW I already did it many time > > with > > > no issues. > > > >> > > > >> I ran: > > > >> > > > >> # zpool status > > > >> pool: zroot > > > >> state: ONLINE > > > >> status: Some supported features are not enabled on the pool. The > pool > > > can > > > >> still be used, but some features are unavailable. > > > >> action: Enable all features using 'zpool upgrade'. Once this is > done, > > > >> the pool may no longer be accessible by software that does not > > > support > > > >> the features. See zpool-features(7) for details. > > > >> scan: none requested > > > >> config: > > > >> > > > >> NAME STATE READ WRITE CKSUM > > > >> zroot ONLINE 0 0 0 > > > >> gpt/disk0 ONLINE 0 0 0 > > > >> > > > >> errors: No known data errors > > > >> > > > >> # zpool add -f zroot da1 > > > >> # zpool add -f zroot da2 > > > >> # zpool status > > > >> pool: zroot > > > >> state: ONLINE > > > >> scan: none requested > > > >> config: > > > >> > > > >> NAME STATE READ WRITE CKSUM > > > >> zroot ONLINE 0 0 0 > > > >> gpt/disk0 ONLINE 0 0 0 > > > >> da1 ONLINE 0 0 0 > > > >> da2 ONLINE 0 0 0 > > > >> > > > >> errors: No known data errors > > > >> # reboot > > > >> > > > >> Then my system doesn’t boot anymore, i got the following error: > > > >> > > > >> gptzfsboot: error 4 lba 2038346899 > > > >> gptzfsboot: error 4 lba 1361327267 > > > >> /boot/config: -Dh > > > >> > > > >> BTX loader 1.00 BTX version is 1.02 > > > >> Consoles: internal video/keyboard serial port > > > >> BIOS drive A: is fd0 > > > >> BIOS drive C: is disk0 > > > >> BIOS drive D: is disk1 > > > >> BIOS drive E: is disk2 > > > >> BIOS drive F: is disk3 > > > >> BIOS drive G: is disk4 > > > >> BIOS drive H: is disk5 > > > >> ZFS: i/o error - all block copies unavailable > > > >> ZFS: failed to read pool zroot directory object > > > >> BIOS 638kB/3143616kB available memory > > > >> > > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > > >> > > > >> Type '?' for list of commands, 'help' for more datailed help. > > > >> OK > > > >> > > > >> I can import my pool with no problems using the lived, but I could > not > > > fix it. > > > >> > > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok > that? > > > >> > > > >> Thanks. > > > >> > > > >> > > > >> > > > >> > > > > _______________________________________________ > > > > freebsd-stable@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > > To unsubscribe, send any mail to " > > freebsd-stable-unsubscribe@freebsd.org > > > " > > > > > > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org > " > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"help
Want to link to this message? Use this
URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20201021162554.K14777>