Date: Tue, 22 Nov 2022 20:01:50 -0600 From: Larry Rosenman <ler@lerctr.org> To: Eric Borisch <eborisch@gmail.com> Cc: Warner Losh <imp@bsdimp.com>, Alan Somers <asomers@freebsd.org>, FreeBSD FS <freebsd-fs@freebsd.org>, John Kennedy <warlock@phouka.net>, Nathan Whitehorn <nwhitehorn@freebsd.org> Subject: Re: UEFI question Message-ID: <1e6a680f170f85563caba1d0842bfa46@lerctr.org> In-Reply-To: <CAMsT2=nB46rnvFTbuwG9OCqJdWSRV%2BsB3PKHhC=15PiS8DQegg@mail.gmail.com> References: <20220531123221.n7ubs365ceeqo2d4@x1> <YpYzbgexVa0mULas@phouka1.phouka.net> <CAOtMX2g4-2PPTSSP-SfrG8m7gQDDqP0J%2BuwOKKutiQxryye6sw@mail.gmail.com> <CANCZdfo7Zixx%2Bd_DkVi3ymbEjtd=f1yFYyJmMixjGCSFyP8h%2Bw@mail.gmail.com> <CAMsT2=nB46rnvFTbuwG9OCqJdWSRV%2BsB3PKHhC=15PiS8DQegg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
--=_7e89b74620cdf99cbfe4dd82a77f1676 Content-Transfer-Encoding: 8bit Content-Type: text/plain; charset=UTF-8; format=flowed On 11/22/2022 5:50 pm, Eric Borisch wrote: > On Tue, Nov 22, 2022 at 1:50 PM Warner Losh <imp@bsdimp.com> wrote: > > On Tue, Nov 22, 2022, 11:42 AM Alan Somers <asomers@freebsd.org> wrote: > On Tue, May 31, 2022 at 9:27 AM John Kennedy <warlock@phouka.net> > wrote: >> >> On Tue, May 31, 2022 at 02:32:21PM +0200, Julien Cigar wrote: >>> I have a Dell R340 server on which installed a 13.0-RELEASE and then >>> upgraded to 13.1-RELEASE (through freebsd-update). ZFS is used >>> (raidz2). >>> >>> The system has the following partitions: >>> https://gist.github.com/silenius/2defdd5480c5c1bc9ba2ff8940756466 >>> Some things regarding UEFI are not clear to me: >>> - as you can see in the partition list, the installer created an EFI >>> partition on all 4 drives, however it looks like only the first one >>> has >>> has been populated (an mounted): >>> https://gist.github.com/silenius/1220c953f905d868c1615fd0e7122bbf .. >>> why ? >>> - if I understand well if my da0 disk dies the system becomes >>> unbootable >>> (https://gist.github.com/silenius/51d202053295270eaaeb2c02316165ee).. >>> correct? >>> - what's the correct way to fix this? should I newfs_msdos on each >>> EFI >>> partition and copy /boot/loader.efi as /EFI/BOOT/BOOTX64.EFI? or >>> should I use efibootmgr? >>> >>> Thanks for enlightening me :) >> >> I use UEFI to multi-boot a lot myself, but not through efibootmgr (vs >> picking it via UEFI/BIOS on boot manually). I'm multi-booting windows >> and FreeBSD, so I'm being (justifiably) paranoid, but not based on >> being burnt in recent memory. >> >> Depending on how your system works (drives are enumerated at startup), >> in theory any of your disks could be da0. If you've only got a >> working >> EFI partition on one, that is probably hedging your bets a bit but >> yes, >> if that drive died, you'd probably be dead. >> >> In my dual-disk setups, I'd see an EFI and a freebsd-boot (BIOS) >> partition >> on both disks, I split my total swap between the two (not mirrored), >> and >> did ZFS "raiding" (mirroring-ish, usually) across the freebsd-zfs >> partitions. >> >> In my case, using the UEFI/BIOS from the motherboard, I could manually >> pick a disk to boot from which let me vet UEFI or ZFS bootblock >> changes >> for sanity before committing to both. But yes, I'd upgrade them both >> to >> the same thing once I was sure it would work. > > I just noticed the same problem. For mirrored installs, the > 13.1-RELEASE installer only formats the efi partition on one hard > disk. Shouldn't it do both? > > We go back and forth on this question.... I think we should, but others > got grumpy when I suggested it. Maybe just all the drives in the zpool > used for the BE? > > Warner Just a "yes, please." If you're installing zfs root on a mirror of disks, POLA suggests the required boot bits are on both disks, too. Thanks, - Eric > This is my common setup for every FreeBSD box I build: ❯ gpart show => 40 3905945520 mfid0 GPT (1.8T) 40 532480 1 efi (260M) 532520 16779224 3 freebsd-swap (8.0G) 17311744 3888633816 4 freebsd-zfs (1.8T) => 40 3905945520 mfid1 GPT (1.8T) 40 532480 1 efi (260M) 532520 16779224 3 freebsd-swap (8.0G) 17311744 3888633816 4 freebsd-zfs (1.8T) => 40 5859442608 mfid2 GPT (2.7T) 40 532480 1 efi (260M) 532520 16779224 3 freebsd-swap (8.0G) 17311744 5842130904 4 freebsd-zfs (2.7T) => 40 5859442608 mfid3 GPT (2.7T) 40 532480 1 efi (260M) 532520 16779224 3 freebsd-swap (8.0G) 17311744 5842130904 4 freebsd-zfs (2.7T) => 40 3905945520 mfid4 GPT (1.8T) 40 532480 1 efi (260M) 532520 16779224 3 freebsd-swap (8.0G) 17311744 3888633816 4 freebsd-zfs (1.8T) => 40 3905945520 mfid5 GPT (1.8T) 40 532480 1 efi (260M) 532520 16777216 3 freebsd-swap (8.0G) 17309736 3888635824 4 freebsd-zfs (1.8T) ler in 🌐 borg in ~ via C v14.0.5-clang on ☁️ (us-east-1) ❯ Disk number and sizes vary but the setup is the same EFI or BOOT or Both, swap, and ZFS on *ALL* disks -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 --=_7e89b74620cdf99cbfe4dd82a77f1676 Content-Transfer-Encoding: quoted-printable Content-Type: text/html; charset=UTF-8 <html><head><meta http-equiv=3D"Content-Type" content=3D"text/html; charset= =3DUTF-8" /></head><body style=3D'font-size: 10pt; font-family: Arial,Helve= tica,sans-serif'> <p id=3D"reply-intro">On 11/22/2022 5:50 pm, Eric Borisch wrote:</p> <blockquote type=3D"cite" style=3D"padding: 0 0.4em; border-left: #1010ff 2= px solid; margin: 0"> <div id=3D"replybody1"> <div dir=3D"auto">On Tue, Nov 22, 2022 at 1:50 PM Warner Losh <<a href= =3D"mailto:imp@bsdimp.com" rel=3D"noreferrer">imp@bsdimp.com</a>> wrote:= </div> <div> <div class=3D"v1gmail_quote"> <blockquote class=3D"v1gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; bor= der-left-width: 1px; border-left-style: solid; padding-left: 1ex; border-le= ft-color: #cccccc;"> <div dir=3D"auto"> <div><br /><br /> <div class=3D"v1gmail_quote"> <div class=3D"v1gmail_attr" dir=3D"ltr">On Tue, Nov 22, 2022, 11:42 AM Alan= Somers <<a href=3D"mailto:asomers@freebsd.org" rel=3D"noreferrer">asome= rs@freebsd.org</a>> wrote:</div> <blockquote class=3D"v1gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; bor= der-left-width: 1px; border-left-style: solid; padding-left: 1ex; border-le= ft-color: #cccccc;">On Tue, May 31, 2022 at 9:27 AM John Kennedy <<a hre= f=3D"mailto:warlock@phouka.net" rel=3D"noreferrer">warlock@phouka.net</a>&g= t; wrote:<br />><br />> On Tue, May 31, 2022 at 02:32:21PM +0200, Jul= ien Cigar wrote:<br />> > I have a Dell R340 server on which installe= d a 13.0-RELEASE and then<br />> > upgraded to 13.1-RELEASE (through = freebsd-update). ZFS is used (raidz2).<br />> ><br />> > The sy= stem has the following partitions: <a href=3D"https://gist.github.com/silen= ius/2defdd5480c5c1bc9ba2ff8940756466" target=3D"_blank" rel=3D"noopener nor= eferrer">https://gist.github.com/silenius/2defdd5480c5c1bc9ba2ff8940756466<= /a><br />> > Some things regarding UEFI are not clear to me:<br />>= ; > - as you can see in the partition list, the installer created an EFI= <br />> > partition on all 4 drives, however it looks like only the f= irst one has<br />> > has been populated (an mounted): <a href=3D"htt= ps://gist.github.com/silenius/1220c953f905d868c1615fd0e7122bbf" target=3D"_= blank" rel=3D"noopener noreferrer">https://gist.github.com/silenius/1220c95= 3f905d868c1615fd0e7122bbf</a> .. why ?<br />> > - if I understand wel= l if my da0 disk dies the system becomes<br />> > unboota= ble (<a href=3D"https://gist.github.com/silenius/51d202053295270eaaeb2c0231= 6165ee)" target=3D"_blank" rel=3D"noopener noreferrer">https://gist.github.= com/silenius/51d202053295270eaaeb2c02316165ee)</a>.. correct?<br />> >= ; - what's the correct way to fix this? should I newfs_msdos on each EFI<br= />> > partition and copy /boot/loader.efi as /EFI/BOOT/B= OOTX64.EFI? or<br />> > should I use efibootmgr?<br />>= ; ><br />> > Thanks for enlightening me :)<br />><br />> I u= se UEFI to multi-boot a lot myself, but not through efibootmgr (vs<br />>= ; picking it via UEFI/BIOS on boot manually). I'm multi-booting windo= ws<br />> and FreeBSD, so I'm being (justifiably) paranoid, but not base= d on<br />> being burnt in recent memory.<br />><br />> Depending = on how your system works (drives are enumerated at startup),<br />> in t= heory any of your disks could be da0. If you've only got a working<br= />> EFI partition on one, that is probably hedging your bets a bit but = yes,<br />> if that drive died, you'd probably be dead.<br />><br />&= gt; In my dual-disk setups, I'd see an EFI and a freebsd-boot (BIOS) partit= ion<br />> on both disks, I split my total swap between the two (not mir= rored), and<br />> did ZFS "raiding" (mirroring-ish, usually) across the= freebsd-zfs partitions.<br />><br />> In my case, using the UEFI/BIO= S from the motherboard, I could manually<br />> pick a disk to boot from= which let me vet UEFI or ZFS bootblock changes<br />> for sanity before= committing to both. But yes, I'd upgrade them both to<br />> the = same thing once I was sure it would work.<br /><br />I just noticed the sam= e problem. For mirrored installs, the<br />13.1-RELEASE installer onl= y formats the efi partition on one hard<br />disk. Shouldn't it do bo= th?</blockquote> </div> </div> <div dir=3D"auto"> </div> <div dir=3D"auto"> </div> <div dir=3D"auto">We go back and forth on this question.... I think we shou= ld, but others got grumpy when I suggested it. Maybe just all the drives in= the zpool used for the BE?</div> </div> <div dir=3D"auto"> <div dir=3D"auto"> </div> <div dir=3D"auto">Warner</div> </div> </blockquote> <div dir=3D"auto"> </div> <div dir=3D"auto">Just a "yes, please." If you're installing zfs root on a = mirror of disks, POLA suggests the required boot bits are on both disks, to= o.</div> <div dir=3D"auto"> </div> <div dir=3D"auto">Thanks,</div> <div dir=3D"auto"> - Eric</div> <blockquote class=3D"v1gmail_quote" style=3D"margin: 0px 0px 0px 0.8ex; bor= der-left-width: 1px; border-left-style: solid; padding-left: 1ex; border-le= ft-color: #cccccc;"> <div dir=3D"auto"> <div dir=3D"auto"> </div> <div dir=3D"auto"> <div class=3D"v1gmail_quote"> </div> </div> </div> </blockquote> </div> </div> </div> </blockquote> <p>This is my common setup for every FreeBSD box I build:</p> <div><span>=E2=9D=AF gpart show</span></div> <div><span>=3D> 40 3905945520 mfi= d0 GPT (1.8T)</span></div> <div><span> 40 532480= 1 efi (260M)</span></div> <div><span> 532520 16779224 &= nbsp; 3 freebsd-swap (8.0G)</span></div> <div><span> 17311744 3888633816 4&nb= sp; freebsd-zfs (1.8T)</span></div> <div><span> </span></div> <div><span>=3D> 40 3905945520 mfi= d1 GPT (1.8T)</span></div> <div><span> 40 532480= 1 efi (260M)</span></div> <div><span> 532520 16779224 &= nbsp; 3 freebsd-swap (8.0G)</span></div> <div><span> 17311744 3888633816 4&nb= sp; freebsd-zfs (1.8T)</span></div> <div><span> </span></div> <div><span>=3D> 40 5859442608 mfi= d2 GPT (2.7T)</span></div> <div><span> 40 532480= 1 efi (260M)</span></div> <div><span> 532520 16779224 &= nbsp; 3 freebsd-swap (8.0G)</span></div> <div><span> 17311744 5842130904 4&nb= sp; freebsd-zfs (2.7T)</span></div> <div><span> </span></div> <div><span>=3D> 40 5859442608 mfi= d3 GPT (2.7T)</span></div> <div><span> 40 532480= 1 efi (260M)</span></div> <div><span> 532520 16779224 &= nbsp; 3 freebsd-swap (8.0G)</span></div> <div><span> 17311744 5842130904 4&nb= sp; freebsd-zfs (2.7T)</span></div> <div><span> </span></div> <div><span>=3D> 40 3905945520 mfi= d4 GPT (1.8T)</span></div> <div><span> 40 532480= 1 efi (260M)</span></div> <div><span> 532520 16779224 &= nbsp; 3 freebsd-swap (8.0G)</span></div> <div><span> 17311744 3888633816 4&nb= sp; freebsd-zfs (1.8T)</span></div> <div><span> </span></div> <div><span>=3D> 40 3905945520 mfi= d5 GPT (1.8T)</span></div> <div><span> 40 532480= 1 efi (260M)</span></div> <div><span> 532520 16777216 &= nbsp; 3 freebsd-swap (8.0G)</span></div> <div><span> 17309736 3888635824 4&nb= sp; freebsd-zfs (1.8T)</span></div> <div><span> </span></div> <div><span> </span></div> <div><span>ler in =F0=9F=8C=90 borg in ~ via C v14.0.5-clang on =E2=98=81= =EF=B8=8F (us-east-1)</span></div> <div><span>=E2=9D=AF</span></div> <div> </div> <div><span>Disk number and sizes vary but the setup is the same EFI or BOOT= or Both, swap, and ZFS on *ALL* disks</span></div> <div id=3D"signature"> <div class=3D"pre" style=3D"margin: 0; padding: 0; font-family: monospace">= <span class=3D"sig">-- <br />Larry Rosenman  = ; <a href=3D"http://www.lerctr.or= g/~ler" target=3D"_blank" rel=3D"noopener noreferrer">http://www.lerctr.org= /~ler</a><br />Phone: +1 214-642-9640 &n= bsp; E-Mail: <a href=3D"mailto:ler@lerctr.org">ler@lerctr.org= </a><br />US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106<br /></span></= div> </div> </body></html> --=_7e89b74620cdf99cbfe4dd82a77f1676--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1e6a680f170f85563caba1d0842bfa46>