Date: Wed, 11 Oct 2023 09:44:10 -0600 From: Warner Losh <imp@bsdimp.com> To: "azathanai@icloud.com" <azathanai@icloud.com> Cc: "freebsd-hackers@FreeBSD.org" <freebsd-hackers@freebsd.org> Subject: Re: Question of installing over SAS controller and USB flash Message-ID: <CANCZdfq5wT9j%2B_YDfk2w0_p_WvQB3X1tO2GLF9-ate0JgxkUJQ@mail.gmail.com> In-Reply-To: <AM6PR02MB4471FE504225A2FBAF35DDA4A5CCA@AM6PR02MB4471.eurprd02.prod.outlook.com> References: <AM6PR02MB4471B8EABD99C690DE9B9F49A5CDA@AM6PR02MB4471.eurprd02.prod.outlook.com> <CANCZdfrz1KZGavcCCpSUjpWUVN2Vem5XBWEzOii6yd=dUxTNJw@mail.gmail.com> <AM6PR02MB4471FE504225A2FBAF35DDA4A5CCA@AM6PR02MB4471.eurprd02.prod.outlook.com>
next in thread | previous in thread | raw e-mail | index | archive | help
[-- Attachment #1 --] There's not a FreeBSD port/pkg for lsiutil, but I'ved used it at $WORK for things. https://github.com/thomaslovell/LSIUtil has an archive of sources. I've not used this at all, and it hasn't been ported to FreeBSD in this repo. But I know it allows one to turn on/off the POST / BIOS in the card, which is what you need. It runs under Linux for sure (again, don't know what packages are available there). I don't think I can share what I have for $WORK with the public since I think we got it in a way that doesn't let me disclose the sources. One might also be able move the code lsiutil that enables it to mpsutil. It's going to be sending a raw command to the device. It shouldn't be too hard.... Warner On Wed, Oct 11, 2023 at 6:02 AM azathanai@icloud.com <azathanai@icloud.com> wrote: > Hey Warner, > the box is very featureLESS (an old Xyratex HS-1235T box, as far as I > researched it). > T2029_QR v1.3.ai (hackspace.org.uk) > <https://wiki.london.hackspace.org.uk/w/images/b/b2/HS-1235T-ATX_Quick_Reference_Sheet.pdf>; > (5) Xyratex 0944037-03 Motherboard Mystery SOLVED : homelab (reddit.com) > <https://www.reddit.com/r/homelab/comments/8loge6/xyratex_094403703_motherboard_mystery_solved/>; > > Through the POST process and the BIOS there is no option to get into the > SAS controller and enable its BIOS. I would love to install FreeBSD > directly on the drives and not to deal with the flash card, but I suppose > the lack of controller BIOS is the reason the box comes with reader and > card directly. > > As I'm new to FreeBSD and ZFS I'm struggling to set up the partitioning. I > was thinking to: > > 1. Partition the entire FLASH card (4gigs) on MBR with freebsd-ufs and > mount /boot. As I've read there should not be ESP (efi) partition as this > is flash, thus MBR, which is "legacy" and without ESP. > 2. Partition one of the drives (3tera) with GPT, freebsd-zfs stripe > and mount / > 3. When the system is up and running, add the rest of the drives (11 > more) to the pool and change the pool to RAIDZ2. However I'm not sure this > is possible, to change the type of the pool without erasing everything. > Logical thinking says it is not possible. > 4. After all of these comes the mounting of filesystems and creating > fstab to continue the installation, which is full darkness at the moment, > as I'm still struggling with the initial start and not have read anything > on that. > > > All of these should be done in shell with the gpart utility during install > and I cannot figure out, how to make points 2 and 3 on one go having > directly the RAIDZ2. > > > ------------------------------ > *From:* owner-freebsd-hackers@freebsd.org < > owner-freebsd-hackers@freebsd.org> on behalf of Warner Losh < > imp@bsdimp.com> > *Sent:* Wednesday, October 11, 2023 00:20 > *To:* azathanai@icloud.com <azathanai@icloud.com> > *Cc:* freebsd-hackers@FreeBSD.org <freebsd-hackers@freebsd.org> > *Subject:* Re: Question of installing over SAS controller and USB flash > > > > On Tue, Oct 10, 2023 at 2:36 PM azathanai@icloud.com <azathanai@icloud.com> > wrote: > > Hi all, > a newbie in FreeBSD here trying to install a RAIDZ2 on a server with mps > SAS controller with drives which are not visible in the boot order in BIOS. > So, I cannot install directly on the drives, despite that the installer > "sees" them, because the BIOS does not see the controller and its drives as > option. > > > Have you enabled the BIOS on the mps card itself? Otherwise it won't > install a BIOS fragment to run at boot, and the drives won't be seen. > > On the other hand, I have a card reader with a flash card on it (4gigs) > from which I can boot and where I can have my /boot partition with the > kernel. > I've a lot of questions and I still cannot find the answers to most of > them, despite going through the handbook, the boot process explanation, the > installation process handbook and a lot of more reading. > > > - How should the flash card be partitioned and with what mountpoints? > The server is BIOS capable. > - How should the rest of the drives be partitioned with ZFS and the > mountpoints? The standard configuration from the installer on RAIDZ2 is > good for me. > > I've experience with Linux but is not helping a lot here. The setup looks > quite different .... probably because of the ZFS. > > Any help, links, ideas and comments will be much appreciated! > > > That's a workaround, but you should be able to enable the BIOS part of the > mps/mpr cards and boot off them. I've done that before. > > Warner > > > > Cyberly yours, > deragoth > > > [-- Attachment #2 --] <div dir="ltr">There's not a FreeBSD port/pkg for lsiutil, but I'ved used it at $WORK for things.<div><br></div><div><a href="https://github.com/thomaslovell/LSIUtil">https://github.com/thomaslovell/LSIUtil</a><br></div><div><br></div><div>has an archive of sources. I've not used this at all, and it hasn't been ported to FreeBSD in this repo.</div><div>But I know it allows one to turn on/off the POST / BIOS in the card, which is what you need. It runs</div><div>under Linux for sure (again, don't know what packages are available there). I don't think I can share</div><div>what I have for $WORK with the public since I think we got it in a way that doesn't let me disclose the</div><div>sources.</div><div><br></div><div>One might also be able move the code lsiutil that enables it to mpsutil. It's going to be sending a raw</div><div>command to the device. It shouldn't be too hard....</div><div><br></div><div>Warner</div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, Oct 11, 2023 at 6:02 AM <a href="mailto:azathanai@icloud.com">azathanai@icloud.com</a> <<a href="mailto:azathanai@icloud.com">azathanai@icloud.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div class="msg6490224784667056540"> <div dir="ltr"> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> Hey Warner,</div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> the box is very featureLESS (an old Xyratex HS-1235T box, as far as I researched it). </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <a href="https://wiki.london.hackspace.org.uk/w/images/b/b2/HS-1235T-ATX_Quick_Reference_Sheet.pdf" id="m_6490224784667056540OWAd4dc15b8-bba5-56c4-c5d4-b428c3cd49ed" target="_blank">T2029_QR v1.3.ai (hackspace.org.uk)</a><br> </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <a href="https://www.reddit.com/r/homelab/comments/8loge6/xyratex_094403703_motherboard_mystery_solved/" id="m_6490224784667056540LPlnkOWA36414efc-3743-e518-0991-1356629114c5" target="_blank">(5) Xyratex 0944037-03 Motherboard Mystery SOLVED : homelab (reddit.com)</a><br> </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> Through the POST process and the BIOS there is no option to get into the SAS controller and enable its BIOS. I would love to install FreeBSD directly on the drives and not to deal with the flash card, but I suppose the lack of controller BIOS is the reason the box comes with reader and card directly.</div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> As I'm new to FreeBSD and ZFS I'm struggling to set up the partitioning. I was thinking to:</div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <ol> <li style="list-style-type:"1. ""><span>Partition the entire FLASH card (4gigs) on MBR with freebsd-ufs and mount /boot. As I've read there should not be ESP (efi) partition as this is flash, thus MBR, which is "legacy" and without ESP.</span></li><li style="list-style-type:"2. ""><span>Partition one of the drives (3tera) with GPT, freebsd-zfs stripe and mount /</span></li><li style="list-style-type:"3. ""><span>When the system is up and running, add the rest of the drives (11 more) to the pool and change the pool to RAIDZ2. However I'm not sure this is possible, to change the type of the pool without erasing everything. Logical thinking says it is not possible.</span></li><li style="list-style-type:"4. "">After all of these comes the mounting of filesystems and creating fstab to continue the installation, which is full darkness at the moment, as I'm still struggling with the initial start and not have read anything on that.</li></ol> <div><br> </div> <div><span>All of these should be done in shell with the gpart utility during install and I cannot figure out, how to make points 2 and 3 on one go having directly the RAIDZ2.</span></div> <div><span><br> </span></div> <div><span><br> </span></div> </div> <div id="m_6490224784667056540appendonsend"></div> <hr style="display:inline-block;width:98%"> <div id="m_6490224784667056540divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> <a href="mailto:owner-freebsd-hackers@freebsd.org" target="_blank">owner-freebsd-hackers@freebsd.org</a> <<a href="mailto:owner-freebsd-hackers@freebsd.org" target="_blank">owner-freebsd-hackers@freebsd.org</a>> on behalf of Warner Losh <<a href="mailto:imp@bsdimp.com" target="_blank">imp@bsdimp.com</a>><br> <b>Sent:</b> Wednesday, October 11, 2023 00:20<br> <b>To:</b> <a href="mailto:azathanai@icloud.com" target="_blank">azathanai@icloud.com</a> <<a href="mailto:azathanai@icloud.com" target="_blank">azathanai@icloud.com</a>><br> <b>Cc:</b> freebsd-hackers@FreeBSD.org <<a href="mailto:freebsd-hackers@freebsd.org" target="_blank">freebsd-hackers@freebsd.org</a>><br> <b>Subject:</b> Re: Question of installing over SAS controller and USB flash</font> <div> </div> </div> <div> <div dir="ltr"> <div dir="ltr"><br> </div> <br> <div> <div dir="ltr">On Tue, Oct 10, 2023 at 2:36 PM <a href="mailto:azathanai@icloud.com" target="_blank"> azathanai@icloud.com</a> <<a href="mailto:azathanai@icloud.com" target="_blank">azathanai@icloud.com</a>> wrote:<br> </div> <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <div dir="ltr"> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> Hi all,</div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> a newbie in FreeBSD here trying to install a RAIDZ2 on a server with mps SAS controller with drives which are not visible in the boot order in BIOS.</div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> So, I cannot install directly on the drives, despite that the installer "sees" them, because the BIOS does not see the controller and its drives as option.</div> </div> </div> </blockquote> <div><br> </div> <div>Have you enabled the BIOS on the mps card itself? Otherwise it won't install a BIOS fragment to run at boot, and the drives won't be seen.</div> <div><br> </div> <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <div dir="ltr"> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> On the other hand, I have a card reader with a flash card on it (4gigs) from which I can boot and where I can have my /boot partition with the kernel. </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> I've a lot of questions and I still cannot find the answers to most of them, despite going through the handbook, the boot process explanation, the installation process handbook and a lot of more reading.</div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <br> </div> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <ul> <li style="list-style-type:"- ""><span>How should the flash card be partitioned and with what mountpoints? The server is BIOS capable.<br> </span></li><li style="list-style-type:"- ""><span>How should the rest of the drives be partitioned with ZFS and the mountpoints? The standard configuration from the installer on RAIDZ2 is good for me.</span></li></ul> <div><span>I've experience with Linux but is not helping a lot here. The setup looks quite different .... probably because of the ZFS.</span></div> <div><span><br> </span></div> <div><span>Any help, links, ideas and comments will be much appreciated!</span></div> </div> </div> </div> </blockquote> <div><br> </div> <div>That's a workaround, but you should be able to enable the BIOS part of the mps/mpr cards and boot off them. I've done that before.</div> <div><br> </div> <div>Warner</div> <div> </div> <blockquote style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"> <div> <div dir="ltr"> <div style="font-family:Aptos,Aptos_EmbeddedFont,Aptos_MSFontService,Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)"> <div><span><br> </span></div> <div><span>Cyberly yours,</span></div> <div><span>deragoth</span></div> <div><span><br> </span></div> <div><span><br> </span></div> </div> </div> </div> </blockquote> </div> </div> </div> </div> </div></blockquote></div>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CANCZdfq5wT9j%2B_YDfk2w0_p_WvQB3X1tO2GLF9-ate0JgxkUJQ>