From nobody Sat Dec 2 05:34:45 2023 X-Original-To: stable@mlmmj.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mlmmj.nyi.freebsd.org (Postfix) with ESMTP id 4ShzCS0dYDz52mlH for ; Sat, 2 Dec 2023 05:35:00 +0000 (UTC) (envelope-from zbeeble@gmail.com) Received: from mail-lf1-x12f.google.com (mail-lf1-x12f.google.com [IPv6:2a00:1450:4864:20::12f]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (2048 bits) client-digest SHA256) (Client CN "smtp.gmail.com", Issuer "GTS CA 1D4" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4ShzCR5jxFz4SML for ; Sat, 2 Dec 2023 05:34:59 +0000 (UTC) (envelope-from zbeeble@gmail.com) Authentication-Results: mx1.freebsd.org; none Received: by mail-lf1-x12f.google.com with SMTP id 2adb3069b0e04-50bc8a9503fso3971908e87.3 for ; Fri, 01 Dec 2023 21:34:59 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1701495298; x=1702100098; darn=freebsd.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=VwSBomomza4Cst1yNVK5IOAviVrsTpFPLIZwPQuuRAE=; b=ndfaatvEOD+5Q8P3NjnP7EfEF7BQCQhZ0hlJWNV/c66S0+av2CVnRW0qHUfMSXvYAz l7t5D8PiUry+tiLFsP/8KICiFSeVfxSIP6Ltgo4FlrpL8dGGxLEkUO5L+zdW6PGLJqXe MCDowHUnNcyBMC5wZFts8HLTkyOzETLFuBTKr8DE0sxBWBPhSJ4kL7xLWMcdYpcpy/IJ NBTfyY9+d5dt/AvfYZBG+uSH31P3rWeX7n421d1msdTfajeFNvkEuOFRGzHHuIWO3lbY UTqsCKXq0rPLB/MqBu/uo4ArwY48fb2Tte1MP7copZ/yT3PTts0aVYmzlbknuwXcVFIz frgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1701495298; x=1702100098; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=VwSBomomza4Cst1yNVK5IOAviVrsTpFPLIZwPQuuRAE=; b=sGVTy0yyVSdktqYMhYnkxnK7xDKjXp5AT7tnERyh1++1MPkIK3RclyCNbnJvjpmCv6 BC61G0Vb69CdhqKAO/xHckVpi0jACCMFOrdEuKuwWxjAqMr+ZXRRLG6C2HxmjEziBQM7 GghDQCNOxYVZZzcVsK61kflrb6CrUsVsXoiFjWWNkzQ+b1lsTlBegL2pn7ZCeFq2aTgN SYbDq0yydM9mnrEc++hN+82JqOAPAKbCUVSStImpO3AzE40AEK6JpFExIfEbpO/eBw/H fftJed3bSZk68jZWyocsrFiBGIWkBm6jsrzL60mKLa2R0H0RTS6omBOM6q6aEN+9Cx41 dNAQ== X-Gm-Message-State: AOJu0YyHpb+D8JULK/w4DDID5gFXBhomqJijjwSIlycjUfuKSURXSplG zqNgSnZ3Y/4aiF3QL6k09Wzo7NpaQxAfxAcJxOud7hg= X-Google-Smtp-Source: AGHT+IF+33tgSA/TF9OXVeKpUvLKN0TuU1RMrWtwa3kAB42znGLL1gGlqeFaNQL9blyJi41zJDc9rsnqCoEct+gnHg8= X-Received: by 2002:a05:6512:e98:b0:50b:d764:8810 with SMTP id bi24-20020a0565120e9800b0050bd7648810mr1523682lfb.92.1701495297561; Fri, 01 Dec 2023 21:34:57 -0800 (PST) List-Id: Production branch of FreeBSD source code List-Archive: https://lists.freebsd.org/archives/freebsd-stable List-Help: List-Post: List-Subscribe: List-Unsubscribe: Sender: owner-freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org MIME-Version: 1.0 References: <86d04457-5018-45f9-849f-eb20ed5cf380@twisted.org.uk> In-Reply-To: From: Zaphod Beeblebrox Date: Sat, 2 Dec 2023 00:34:45 -0500 Message-ID: Subject: Re: EFI and zfs raid mirror partial fail (14.0 and RELENG_13) To: Warner Losh Cc: Pete French , FreeBSD Stable ML Content-Type: multipart/alternative; boundary="000000000000cc5789060b803e27" X-Spamd-Bar: ---- X-Rspamd-Pre-Result: action=no action; module=replies; Message is reply to one we originated X-Spamd-Result: default: False [-4.00 / 15.00]; REPLY(-4.00)[]; ASN(0.00)[asn:15169, ipnet:2a00:1450::/32, country:US] X-Rspamd-Queue-Id: 4ShzCR5jxFz4SML --000000000000cc5789060b803e27 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable It can be more straightforward to update the gmirror, however. I've done this with UFS --- old boot, pair of UFS/GMIRROR usb sticks that then boot to a ZFS array that the BIOS couldn't see (so UFS only contained /boot and /rescue). It's easier to know that the boot is updated identically if gmirrored. Gmirror also has tools to verify, etc. On Fri, Dec 1, 2023 at 7:46=E2=80=AFPM Warner Losh wrote: > > > On Fri, Dec 1, 2023, 4:57 PM Pete French wrote: > >> >> On 01/12/2023 21:53, mike tancsa wrote: >> > Should have looked at open PRs. There is one from a while ago >> > >> > https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=3D258987 >> > >> > >> >> Was thinking about this, and I was wondering if it would be possible to >> make the EFI partition a gmirror. So its across all discs, mounted only >> once, but would still boot from any of them. My understanding is geom >> has the label at the end, yes ? So the firmware would see the filesystem >> on a single partition quite happily ? >> > > I've done this. It works ok. But I don't run like this in production. If = I > write a new file, that has so many writes to the different disks. If they > all go through then life is good (this is what gets us to OK). > > BUT, if there is a power failure or crash and only some of them make it t= o > disk, then you have a corrupt ESP and the BIOS may pick that ESP to boot > off of, booting corrupt data. > > Since this is infrequently updated, you can use a safe sequence to update > things one partition a time, then you might lose the file entirely, but i= t > will either be there and good. Or it will be gone. You can't get into a b= ad > situation. Either you boot old or new loader and can just quit from the > boot loader if it's the old one and it can't boot. Efi will try the next > one on the list. > > Here manual mirroring, if scripted, can be more reliable than gmirror. > > Warner > > -pete. >> >> >> --000000000000cc5789060b803e27 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
It can be more straightforward to update the gmirror, howe= ver.=C2=A0 I've done this with UFS --- old boot, pair of UFS/GMIRROR us= b sticks that then boot to a ZFS array that the BIOS couldn't see (so U= FS only contained /boot and /rescue).=C2=A0 It's easier to know that th= e boot is updated identically if gmirrored.=C2=A0 Gmirror also has tools to= verify, etc.

On Fri, Dec 1, 2023 at 7:46=E2=80=AFPM Warner Losh <imp@bsdimp.com> wrote:

On Fri, D= ec 1, 2023, 4:57 PM Pete French <pete@twisted.org.uk> wrote:

On 01/12/2023 21:53, mike tancsa wrote:
> Should have looked at open PRs. There is one from a while ago
>
> https://bugs.freebsd.org/b= ugzilla/show_bug.cgi?id=3D258987
>
>

Was thinking about this, and I was wondering if it would be possible to make the EFI partition a gmirror. So its across all discs, mounted only once, but would still boot from any of them. My understanding is geom
has the label at the end, yes ? So the firmware would see the filesystem on a single partition quite happily ?

I've done this. It works ok. But = I don't run like this in production. If I write a new file, that has so= many writes to the different disks. If they all go through then life is go= od (this is what gets us to OK).

BUT, if there is a power failure or crash and only some of them m= ake it to disk, then you have a corrupt ESP and the BIOS may pick that ESP = to boot off of, booting corrupt data.

Since this is infrequently updated, you can use a safe sequen= ce to update things one partition a time, then you might lose the file enti= rely, but it will either be there and good. Or it will be gone. You can'= ;t get into a bad situation. Either you boot old or new loader and can just= quit from the boot loader if it's the old one and it can't boot. E= fi will try the next one on the list.

Here manual mirroring, if scripted, can be more reliable than= gmirror.

Warner

-pete.


--000000000000cc5789060b803e27--