Date: Wed, 21 Oct 2020 16:36:17 -0400 (EDT) From: Walter Cramer <wfc@mintsol.com> To: Cassiano Peixoto <peixotocassiano@gmail.com> Cc: freebsd-stable@freebsd.org Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 Message-ID: <20201021162554.K14777@mulder.mintsol.com> In-Reply-To: <CAJajdNU0fP7RVJuvZT7%2BuUTquRKSeRLVx9L7cBJ10XVJEZ5LOw@mail.gmail.com> References: <CAJajdNWgiKyd1jmX4uPogWU4Ty=OiEgDnQGaoLZUWgj_pr9WKw@mail.gmail.com> <CAJajdNXPhGj-mKj6R_N94Bfnhvm=nHr3RwfBm4qoTB3g_=6kGw@mail.gmail.com> <CAJajdNV0nGdhAm6cgs2SKMBsY=Hi1Azb8fA_0PFrLUuejCT8ag@mail.gmail.com> <fea719f4-8918-38e6-7566-d77e9a244ef0@sentex.net> <CAJajdNU9-pc9De0tDge=4HPaT1jc%2B7evfZSjCrcNMXqtSpDs2Q@mail.gmail.com> <20201021160923.T14777@mulder.mintsol.com> <CAJajdNU0fP7RVJuvZT7%2BuUTquRKSeRLVx9L7cBJ10XVJEZ5LOw@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I very much doubt that there's any remotely sane way to re-partition da1 &= =20 da2 while they are in the pool. If you carefully edit the output of the 'gpart backup' command, you=20 probably can use 'gpart restore' to partition da1 and da2. Read the=20 'gpart' man page very carefully, if you aren't familiar with that. Also,= =20 you'll probably want to install bootstrap code on da1 & da2 with 'gpart=20 bootstrap'. -Walter On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > Walter, Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. But is there a way to make the partition of da1 and da2 since both are already inserted on pool? I think it's not allowed... Thanks. On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer <wfc@mintsol.com> wrote: > My guess - there is a work-around or two, but you'll face a lot more > grief, long-term, if you don't do things the right way (aka do a bunch of > re-install work) now. > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), to > see how the original disk is partitioned. Then duplicate that > partitioning on da1 and da2. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Hi guys, > > Thank your for your answer. > > @Ricchard First of all I didn't have a chance to run zpool upgrade, becau= se > after the system update reboot i ran into the issue. > > @Walter and @mike Regarding making a partition, I never see any > recommendation about this, I've been always using the entire disk on a po= ol > to expand it. Until FreeBSD 11.2 it worked fine. > > Is there a workaround for this or should I reinstall the entire pool? > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa <mike@sentex.net> wrote: > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 i= s > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > boot info ? Generally its not recommended to use the entire disk as > > part of a pool. Create a partition scheme first > > > > gpart create -s gpt da1 > > > > gpart add -t freebsd-zfs da1 > > > > then add da1p1 to the pool > > > > ---Mike > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > Hi there, > > > > > > Anyone can help please? I've many servers with this same issue. Thank= s > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > peixotocassiano@gmail.com> > > > wrote: > > > > > >> Hi there, > > >> > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then I > > added two more disks to expand my pool. BTW I already did it many time > with > > no issues. > > >> > > >> I ran: > > >> > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> status: Some supported features are not enabled on the pool. The poo= l > > can > > >> still be used, but some features are unavailable. > > >> action: Enable all features using 'zpool upgrade'. Once this is done= , > > >> the pool may no longer be accessible by software that does not > > support > > >> the features. See zpool-features(7) for details. > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> > > >> # zpool add -f zroot da1 > > >> # zpool add -f zroot da2 > > >> # zpool status > > >> pool: zroot > > >> state: ONLINE > > >> scan: none requested > > >> config: > > >> > > >> NAME STATE READ WRITE CKSUM > > >> zroot ONLINE 0 0 0 > > >> gpt/disk0 ONLINE 0 0 0 > > >> da1 ONLINE 0 0 0 > > >> da2 ONLINE 0 0 0 > > >> > > >> errors: No known data errors > > >> # reboot > > >> > > >> Then my system doesn=E2=80=99t boot anymore, i got the following err= or: > > >> > > >> gptzfsboot: error 4 lba 2038346899 > > >> gptzfsboot: error 4 lba 1361327267 > > >> /boot/config: -Dh > > >> > > >> BTX loader 1.00 BTX version is 1.02 > > >> Consoles: internal video/keyboard serial port > > >> BIOS drive A: is fd0 > > >> BIOS drive C: is disk0 > > >> BIOS drive D: is disk1 > > >> BIOS drive E: is disk2 > > >> BIOS drive F: is disk3 > > >> BIOS drive G: is disk4 > > >> BIOS drive H: is disk5 > > >> ZFS: i/o error - all block copies unavailable > > >> ZFS: failed to read pool zroot directory object > > >> BIOS 638kB/3143616kB available memory > > >> > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > >> > > >> Type '?' for list of commands, 'help' for more datailed help. > > >> OK > > >> > > >> I can import my pool with no problems using the lived, but I could n= ot > > fix it. > > >> > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok tha= t? > > >> > > >> Thanks. > > >> > > >> > > >> > > >> > > > _______________________________________________ > > > freebsd-stable@freebsd.org mailing list > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > To unsubscribe, send any mail to " > freebsd-stable-unsubscribe@freebsd.org > > " > > > > > > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" _______________________________________________ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org" From owner-freebsd-stable@freebsd.org Wed Oct 21 21:50:46 2020 Return-Path: <owner-freebsd-stable@freebsd.org> Delivered-To: freebsd-stable@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id E6F2042E9A6 for <freebsd-stable@mailman.nyi.freebsd.org>; Wed, 21 Oct 2020 21:50:46 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: from mail-ed1-x52c.google.com (mail-ed1-x52c.google.com [IPv6:2a00:1450:4864:20::52c]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1O1" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4CGkh600hZz3ZcF for <freebsd-stable@freebsd.org>; Wed, 21 Oct 2020 21:50:45 +0000 (UTC) (envelope-from peixotocassiano@gmail.com) Received: by mail-ed1-x52c.google.com with SMTP id t20so4043067edr.11 for <freebsd-stable@freebsd.org>; Wed, 21 Oct 2020 14:50:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=M0PcM0UELISwSLrr/9zSNpqPEhGMsMgb+jNX8AbXYZ0=; b=mRREEDuu7HmoUppzumxG5OWdWzyaQTrKcEkBK7MBjtGynRpgJGcPyfOPxY/0sAoqT5 vFR8nmhfGU5lfPnvCrcsvYGjH+rNb1dCh5nMWy/Bc/dCyG1/qsiH6DZBQhVW4M5AqsrM TbRv4QPuq3/CZiDsggFD3RkmvDhahqesJgvjGZd+SjcZLP3L8yAbKEobkx24Sah3kGN4 HktHUedb+KPVKNhAxcK5W0UpssNLGy5IvbD6/mWOd2PrKBzwGLXizzf3c5DlkJScpXpW xBnKvJj9meMtnAUZcCtYUsPvVQq7MFfwHsSI9EJ0w5WAyJ5lsc0iucGPV/Kk+8d62JvP m7wA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=M0PcM0UELISwSLrr/9zSNpqPEhGMsMgb+jNX8AbXYZ0=; b=LYH4b6VdkOOxgk74POJ7OvnNigIXMvVNsnXaYkzdTkF03i1c0B0hSsqSQ7Y2TwjmM5 AUib+PxMN96/+iMvryTsZudJaDhezOIR27CrzsLHLOCWbxU5dcdziz8h88Mf6rzetOvw ACMJmiOfG701XeEbDN+PdQ2NIYrtggK31HBPw00DyoQYD+YEGadBND4shLpHGWGCQdTV i+4h4vwVDCDJZRFxHG0fjF8J46n5qzJOMmaXMndzan++cpDUkoHk7WYW7Hbc280a6Yel 66NcR6OLG+XdB9Mp6+KpZQAHg6XPHPx5IcsWOfG48Jwjgs9yejR4o5qkBWpykkCCx+C5 nfjg== X-Gm-Message-State: AOAM5303xZDsvElisrB0QKjCTDMnh+Y7c5f5/CMne2J7EogTiRjv8Jtv Dt3S8D8CjDYE/qjXITH34ReS5JfxqjcMIua4Zhg= X-Google-Smtp-Source: ABdhPJyU1T6wBB9So5SlzI1mYwZIZQxNapHE73D8o8YFNAgv27fQ18zNAhZ9wf3vqC3HTLHONimWYhx+HKzV0LKa9Ro= X-Received: by 2002:a05:6402:744:: with SMTP id p4mr4969539edy.190.1603317044291; Wed, 21 Oct 2020 14:50:44 -0700 (PDT) MIME-Version: 1.0 References: <CAJajdNWgiKyd1jmX4uPogWU4Ty=OiEgDnQGaoLZUWgj_pr9WKw@mail.gmail.com> <CAJajdNXPhGj-mKj6R_N94Bfnhvm=nHr3RwfBm4qoTB3g_=6kGw@mail.gmail.com> <CAJajdNV0nGdhAm6cgs2SKMBsY=Hi1Azb8fA_0PFrLUuejCT8ag@mail.gmail.com> <fea719f4-8918-38e6-7566-d77e9a244ef0@sentex.net> <CAJajdNU9-pc9De0tDge=4HPaT1jc+7evfZSjCrcNMXqtSpDs2Q@mail.gmail.com> <20201021160923.T14777@mulder.mintsol.com> <CAJajdNU0fP7RVJuvZT7+uUTquRKSeRLVx9L7cBJ10XVJEZ5LOw@mail.gmail.com> <20201021162554.K14777@mulder.mintsol.com> In-Reply-To: <20201021162554.K14777@mulder.mintsol.com> From: Cassiano Peixoto <peixotocassiano@gmail.com> Date: Wed, 21 Oct 2020 18:50:32 -0300 Message-ID: <CAJajdNUSHp4A2S_WwzC0tP-LiMYow=33b4bs4OAtWBkFCyNDFA@mail.gmail.com> Subject: Re: Zpool doesn't boot anymore after FreeBSD 12.1 To: Walter Cramer <wfc@mintsol.com> Cc: freebsd-stable@freebsd.org X-Rspamd-Queue-Id: 4CGkh600hZz3ZcF X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20161025 header.b=mRREEDuu; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of peixotocassiano@gmail.com designates 2a00:1450:4864:20::52c as permitted sender) smtp.mailfrom=peixotocassiano@gmail.com X-Spamd-Result: default: False [-3.53 / 15.00]; RCVD_TLS_ALL(0.00)[]; ARC_NA(0.00)[]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20161025]; NEURAL_HAM_MEDIUM(-1.01)[-1.009]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; FREEMAIL_FROM(0.00)[gmail.com]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-1.004]; TO_MATCH_ENVRCPT_SOME(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::52c:from]; NEURAL_HAM_SHORT(-0.52)[-0.519]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; MAILMAN_DEST(0.00)[freebsd-stable]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.33 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-stable>, <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>; List-Post: <mailto:freebsd-stable@freebsd.org> List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-stable>, <mailto:freebsd-stable-request@freebsd.org?subject=subscribe> X-List-Received-Date: Wed, 21 Oct 2020 21:50:47 -0000 Thanks for the tip! I've fixed the issue following the following steps: 1) added two new disks with the same size: da3 and da4 2) Made a partition for both: gpart create -s gpt da3 gpart create -s gpt da4 gpart add -t freebsd-zfs da3 gpart add -t freebsd-zfs da4 3) replace both disks on pool zpool replace zroot da2 da3p1 zpool replace zroot da3 da4p1 4) rebuild bootcode gpart boocode -b /boot/pmbr -p /boot/gptzfsboot -i 1 da0 Done! I rebooted and everything is working again. Thank you guys for the support. On Wed, Oct 21, 2020 at 5:36 PM Walter Cramer <wfc@mintsol.com> wrote: > I very much doubt that there's any remotely sane way to re-partition da1 = & > da2 while they are in the pool. > > If you carefully edit the output of the 'gpart backup' command, you > probably can use 'gpart restore' to partition da1 and da2. Read the > 'gpart' man page very carefully, if you aren't familiar with that. Also, > you'll probably want to install bootstrap code on da1 & da2 with 'gpart > bootstrap'. > > -Walter > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > Walter, > > Yes, gpt/disk0 is da0. I can do the partition backup booting from livecd. > But is there a way to make the partition of da1 and da2 since both are > already inserted on pool? I think it's not allowed... > > Thanks. > > > On Wed, Oct 21, 2020 at 5:18 PM Walter Cramer <wfc@mintsol.com> wrote: > > > My guess - there is a work-around or two, but you'll face a lot more > > grief, long-term, if you don't do things the right way (aka do a bunch = of > > re-install work) now. > > > > I'd start with 'gpart backup da0' (guessing that gpt/disk0 is on da0), = to > > see how the original disk is partitioned. Then duplicate that > > partitioning on da1 and da2. > > > > -Walter > > > > On Wed, 21 Oct 2020, Cassiano Peixoto wrote: > > > > > Hi guys, > > > > Thank your for your answer. > > > > @Ricchard First of all I didn't have a chance to run zpool upgrade, > because > > after the system update reboot i ran into the issue. > > > > @Walter and @mike Regarding making a partition, I never see any > > recommendation about this, I've been always using the entire disk on a > pool > > to expand it. Until FreeBSD 11.2 it worked fine. > > > > Is there a workaround for this or should I reinstall the entire pool? > > > > On Wed, Oct 21, 2020 at 4:45 PM mike tancsa <mike@sentex.net> wrote: > > > > > Just a guess, Is your VM still trying to boot from whatever gpt/disk0 > is > > > ? Or is it perhaps trying to boot from da1 or da2 which does not have > > > boot info ? Generally its not recommended to use the entire disk as > > > part of a pool. Create a partition scheme first > > > > > > gpart create -s gpt da1 > > > > > > gpart add -t freebsd-zfs da1 > > > > > > then add da1p1 to the pool > > > > > > ---Mike > > > > > > > > > On 10/21/2020 8:20 AM, Cassiano Peixoto wrote: > > > > Hi there, > > > > > > > > Anyone can help please? I've many servers with this same issue. > Thanks > > > > > > > > On Fri, Oct 16, 2020 at 10:24 AM Cassiano Peixoto < > > > peixotocassiano@gmail.com> > > > > wrote: > > > > > > > >> Hi there, > > > >> > > > >> I have a FreeBSD 12.1-STABLE running on VMWARE with one disk. Then= I > > > added two more disks to expand my pool. BTW I already did it many tim= e > > with > > > no issues. > > > >> > > > >> I ran: > > > >> > > > >> # zpool status > > > >> pool: zroot > > > >> state: ONLINE > > > >> status: Some supported features are not enabled on the pool. The > pool > > > can > > > >> still be used, but some features are unavailable. > > > >> action: Enable all features using 'zpool upgrade'. Once this is > done, > > > >> the pool may no longer be accessible by software that does no= t > > > support > > > >> the features. See zpool-features(7) for details. > > > >> scan: none requested > > > >> config: > > > >> > > > >> NAME STATE READ WRITE CKSUM > > > >> zroot ONLINE 0 0 0 > > > >> gpt/disk0 ONLINE 0 0 0 > > > >> > > > >> errors: No known data errors > > > >> > > > >> # zpool add -f zroot da1 > > > >> # zpool add -f zroot da2 > > > >> # zpool status > > > >> pool: zroot > > > >> state: ONLINE > > > >> scan: none requested > > > >> config: > > > >> > > > >> NAME STATE READ WRITE CKSUM > > > >> zroot ONLINE 0 0 0 > > > >> gpt/disk0 ONLINE 0 0 0 > > > >> da1 ONLINE 0 0 0 > > > >> da2 ONLINE 0 0 0 > > > >> > > > >> errors: No known data errors > > > >> # reboot > > > >> > > > >> Then my system doesn=E2=80=99t boot anymore, i got the following e= rror: > > > >> > > > >> gptzfsboot: error 4 lba 2038346899 > > > >> gptzfsboot: error 4 lba 1361327267 > > > >> /boot/config: -Dh > > > >> > > > >> BTX loader 1.00 BTX version is 1.02 > > > >> Consoles: internal video/keyboard serial port > > > >> BIOS drive A: is fd0 > > > >> BIOS drive C: is disk0 > > > >> BIOS drive D: is disk1 > > > >> BIOS drive E: is disk2 > > > >> BIOS drive F: is disk3 > > > >> BIOS drive G: is disk4 > > > >> BIOS drive H: is disk5 > > > >> ZFS: i/o error - all block copies unavailable > > > >> ZFS: failed to read pool zroot directory object > > > >> BIOS 638kB/3143616kB available memory > > > >> > > > >> FreeBSD/x86 bootstrap loader, Revision 1.1 > > > >> ERROR: cannot open /boot/lua/loader.lua: invalid argument. > > > >> > > > >> Type '?' for list of commands, 'help' for more datailed help. > > > >> OK > > > >> > > > >> I can import my pool with no problems using the lived, but I could > not > > > fix it. > > > >> > > > >> Seems a bug after 12.1-STABLE. Please, anyone can take a look ok > that? > > > >> > > > >> Thanks. > > > >> > > > >> > > > >> > > > >> > > > > _______________________________________________ > > > > freebsd-stable@freebsd.org mailing list > > > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > > > To unsubscribe, send any mail to " > > freebsd-stable-unsubscribe@freebsd.org > > > " > > > > > > > > > _______________________________________________ > > freebsd-stable@freebsd.org mailing list > > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.or= g > " > _______________________________________________ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20201021162554.K14777>