From nobody Tue Apr 26 19:14:20 2022 X-Original-To: freebsd-stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id A92B8199AA15 for ; Tue, 26 Apr 2022 19:14:34 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-ej1-x629.google.com (mail-ej1-x629.google.com [IPv6:2a00:1450:4864:20::629]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4Kns556LSYz3pyR for ; Tue, 26 Apr 2022 19:14:33 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: by mail-ej1-x629.google.com with SMTP id gh6so13970583ejb.0 for ; Tue, 26 Apr 2022 12:14:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=LxNa+DizZtI8AiBBrumC8VzUa18K0pKsLDzp1F9E6ok=; b=RMsuWbctj3Mo8WBVZiMj+0Obza+PndMZ7MRQP+rc2SaerPVDY2g6VF7QefSSf3NCuB MbuRP0jMIF0U+qer0OMV75U046sL1aBLOM41VQunJp90cdUL7u+ODrlzfFDnO7g0bgul fdtEV92QdQNaYeekro6vM1EtKn5PVBADBPmsNrSzJWwpUVxYVJmFR/AlulTsicYFuXuC yxn0did3c2RjFcCoS93BaNQMDLThFe8VK+4k65LaXRYnzJ1r/+GiVO7z326ChwzNnI43 LWXqxC+6dat1MwzJi2ApfACZNZwEUEYastNjzPv0+EipE9DcwpArFABce06d18S1oOQH pfrQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=LxNa+DizZtI8AiBBrumC8VzUa18K0pKsLDzp1F9E6ok=; b=6rJIuNOk+QWZ0PcPGgleAl7QHPG4XXh6416pmukvkCfO3rcYsj6XAKRWKy80dbE/lJ QqQe6iHhSCUE87Y4dZ8OnlCmcd0OludjsAe0b3V9SdgRbK8HIn65ypUqFWt75z/70ePW QMtsE/nXd3uMV2DHR9fqva4ZKEpo8AAPoox84NVipHiusK4A0gcXmF7MolTws4KV/Fow UciqCMSSUcR+EFkS68MwLb2ClWQxSRUzkFBlpgw4I5ewFB/3B2eaQCQxIZITczUpw3GU fMYWt0hPtaZ123ATuYSBPTjrsVNsEzmyEs9bXROEH4AU1PBC6yUaRhdAp1IuGTYEZEtY XQmg== X-Gm-Message-State: AOAM533TCZiAitAF+uEzlKjHQwZE5Q9yxTBfLul4vfUkv28eFbZg/1iE gC+loKOCqU8pZkpo1yxgzFKhSSv0fISXE5hIw1nD+UE= X-Google-Smtp-Source: ABdhPJxqU9dAzk50ddK+cV1tFvMyMi6Xn3AwlHrENG/WaKihyHT8zWctoaBQJ3MHUOV1jPokfMxYFIpRp1ioUMUDyoU= X-Received: by 2002:a17:907:7745:b0:6f3:674a:339 with SMTP id kx5-20020a170907774500b006f3674a0339mr19235038ejc.207.1651000472670; Tue, 26 Apr 2022 12:14:32 -0700 (PDT) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <2c2d774c-3677-c3b7-d099-71318743a434@zhegan.in> In-Reply-To: <2c2d774c-3677-c3b7-d099-71318743a434@zhegan.in> From: Zaphod Beeblebrox Date: Tue, 26 Apr 2022 15:14:20 -0400 Message-ID: Subject: Re: FreeBSD boot from root pool larger than 1Tb/cannot read MOS/all block copies unavailable To: "Eugene M. Zheganin" Cc: FreeBSD Stable Content-Type: multipart/alternative; boundary="000000000000b2944b05dd938046" X-Rspamd-Queue-Id: 4Kns556LSYz3pyR X-Spamd-Bar: - Authentication-Results: mx1.freebsd.org; dkim=pass header.d=gmail.com header.s=20210112 header.b=RMsuWbct; dmarc=pass (policy=none) header.from=gmail.com; spf=pass (mx1.freebsd.org: domain of zbeeble@gmail.com designates 2a00:1450:4864:20::629 as permitted sender) smtp.mailfrom=zbeeble@gmail.com X-Spamd-Result: default: False [-1.99 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.998]; R_DKIM_ALLOW(-0.20)[gmail.com:s=20210112]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2a00:1450:4000::/36:c]; FREEMAIL_FROM(0.00)[gmail.com]; MIME_GOOD(-0.10)[multipart/alternative,text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-stable@freebsd.org]; NEURAL_HAM_LONG(-1.00)[-0.995]; NEURAL_SPAM_SHORT(1.00)[1.000]; TO_MATCH_ENVRCPT_SOME(0.00)[]; TO_DN_ALL(0.00)[]; DKIM_TRACE(0.00)[gmail.com:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[gmail.com,none]; RCVD_IN_DNSWL_NONE(0.00)[2a00:1450:4864:20::629:from]; MID_RHS_MATCH_FROMTLD(0.00)[]; MLMMJ_DEST(0.00)[freebsd-stable]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+,1:+,2:~]; FREEMAIL_ENVFROM(0.00)[gmail.com]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; DWL_DNSWL_NONE(0.00)[gmail.com:dkim] X-ThisMailContainsUnwantedMimeParts: N --000000000000b2944b05dd938046 Content-Type: text/plain; charset="UTF-8" I have had this happen too... in my case an TR4 motherboard booting from 2x 4T (WD red) and an NMVE cache/log. What I've found is that the BIOS is faulty for some reason on reboot --- and that a complete poweroff (where the board power -- and the LEDs go out) is required ... and then everything boots just fine. ... but the not being able to find all block copies is the error that the bootblock loader emits while trying to suck in loader. ... and in my case, the only drives BIOS can see are the 2x 4T and the NVMe. The system also has a SAS card in it, but this card is configured to not talk to BIOS and presents no disks. This card has also been changed out with other hardware several times whlie the system has shown this "not all block copies" error since switching to this motherboard. On Tue, Apr 26, 2022 at 10:12 AM Eugene M. Zheganin wrote: > Hello, > > recently I found another server that was running 12.x and became > unbootable because of the famous "cannot read MOS/all block copies > unavailable/etc" gptzfsboot message which can be randomly displayed when > boot blocks migrate beyond 1st Tb on a large root pool. I managed to > boot the server from CD (pointing out the root pool with > vfs.root.mountfrom) and then upgraded it to the recent 13.0, where is > showed no signs of being able to boot from disks again (so I'm treating > this as still not fixed, and probably as won't be fixed issue). > > Since this issue is like 10 years old and I see no light at the end of > the tunnel, could the bsdinstall at least be made aware about this so it > won't permit the large root pool creation by default ? Because as far as > I remember there's still no option in it to create a pool smaller than > the disk, and I have to use the custom installation script in order to > do this. > > I'm asking because unfortunately recently I've found two more of these > time bombs among my servers. > > > Thanks. > > > Eugene. > > > P.S. There have been speculations that switching to the UEFI loader may > help, because it's more functional and probably can see the migrated > over 1Tb boot blocks, can anyone confirm this ? > > > --000000000000b2944b05dd938046 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
I have had this happen too... in my case an TR4 mothe= rboard booting from 2x 4T (WD red) and an NMVE cache/log.=C2=A0 What I'= ve found is that the BIOS is faulty for some reason on reboot --- and that = a complete poweroff (where the board power -- and the LEDs go out) is requi= red ... and then everything boots just fine.

... b= ut the not being able to find all block copies is the error that the bootbl= ock loader emits while trying to suck in loader.

.= .. and in my case, the only drives BIOS can see are the 2x 4T and the NVMe.= =C2=A0 The system also has a SAS card in it, but this card is configured to= not talk to BIOS and presents no disks.=C2=A0 This card has also been chan= ged out with other hardware several times whlie the system has shown this &= quot;not all block copies" error since switching to this motherboard.<= br>

On Tue, Apr 26, 2022 at 10:12 AM Eugene M. Zheganin <eugene@zhegan.in> wrote:
Hello,

recently I found another server that was running 12.x and became
unbootable because of the famous "cannot read MOS/all block copies unavailable/etc" gptzfsboot message which can be randomly displayed wh= en
boot blocks migrate beyond 1st Tb on a large root pool. I managed to
boot the server from CD (pointing out the root pool with
vfs.root.mountfrom) and then upgraded it to the recent 13.0, where is
showed no signs of being able to boot from disks again (so I'm treating=
this as still not fixed, and probably as won't be fixed issue).

Since this issue is like 10 years old and I see no light at the end of
the tunnel, could the bsdinstall at least be made aware about this so it won't permit the large root pool creation by default ? Because as far a= s
I remember there's still no option in it to create a pool smaller than =
the disk, and I have to use the custom installation script in order to
do this.

I'm asking because unfortunately recently I've found two more of th= ese
time bombs among my servers.


Thanks.


Eugene.


P.S. There have been speculations that switching to the UEFI loader may help, because it's more functional and probably can see the migrated over 1Tb boot blocks, can anyone confirm this ?


--000000000000b2944b05dd938046--