Date: Fri, 17 Jun 2022 18:30:07 -0500 From: Larry Rosenman <ler@lerctr.org> To: Michael Gmelin <grembo@freebsd.org> Cc: Alexander Motin <mav@freebsd.org>, Freebsd current <freebsd-current@freebsd.org> Subject: Re: SAS/SATA controllers: 8 port that support 8TB Drives Message-ID: <5936c6368df3c7d762d4710d23e7c57c@lerctr.org> In-Reply-To: <736314DA-354C-465C-8D8C-AF67AAD68F5D@freebsd.org> References: <f31ff2942ca3d3f1389f21157986ce46@lerctr.org> <736314DA-354C-465C-8D8C-AF67AAD68F5D@freebsd.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 06/17/2022 6:20 pm, Michael Gmelin wrote: >> On 18. Jun 2022, at 00:57, Larry Rosenman <ler@lerctr.org> wrote: >> On 06/17/2022 5:48 pm, Michael Gmelin wrote: >>>> On 18. Jun 2022, at 00:31, Alexander Motin <mav@freebsd.org> wrote: >>>> >>>>> On 17.06.2022 18:24, Alexander Motin wrote: >>>>>> On 17.06.2022 18:16, Larry Rosenman wrote: >>>>>> On 06/17/2022 5:08 pm, Alexander Motin wrote: >>>>>>> On 17.06.2022 11:59, Larry Rosenman wrote: >>>>>>>> I'm looking to upgrade the controllers in my TrueNAS box to >>>>>>>> something that will >>>>>>>> support 8TB drives because apparently my LSI 2108 controllers do >>>>>>>> not support 8TB drives. >>>>>>>> What's the communities recommendation? >>>>>>>> needs to support SFF connectors for a total of 4 SFF connectors, >>>>>>>> as I have 16 slots. >>>>>>> We at iX are still using LSI/Broadcom HBAs, just moved from long >>>>>>> discontinued mps(4) to newer mpr(4). And I don't believe the >>>>>>> problem >>>>>>> is directly related to capacity. According to my observations it >>>>>>> may >>>>>>> be Seagate HDDs of/above certain (8TB) generation. We do not use >>>>>>> Seagate HDDs in our products, so about that instability I only >>>>>>> heard >>>>>>> from forums and TrueNAS community user reports. >>>>>> This is a mfi(4) set of controllers, and a ST80000Nm0045 8TB (CMR) >>>>>> drive. >>>>>> Is this a bad combo? >>>>>> mfi0: 9973 (708793330s/0x0002/WARN) - PD 00(e0xfc/s3) is not >>>>>> supported >>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an >>>>>> error >>>>>> (probe0:mfi0:0:0:0): Retrying command, 3 more tries remain >>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an >>>>>> error >>>>>> (probe0:mfi0:0:0:0): Retrying command, 2 more tries remain >>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an >>>>>> error >>>>>> (probe0:mfi0:0:0:0): Retrying command, 1 more tries remain >>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an >>>>>> error >>>>>> (probe0:mfi0:0:0:0): Retrying command, 0 more tries remain >>>>>> (probe0:mfi0:0:0:0): INQUIRY. CDB: 12 00 00 00 24 00 >>>>>> (probe0:mfi0:0:0:0): CAM status: CCB request completed with an >>>>>> error >>>>>> (probe0:mfi0:0:0:0): Error 5, Retries exhausted >>>>>> mfi0 Physical Drives: >>>>>> 0 ( 932G) UNCONFIGURED GOOD <ST8000NM0045-1RL UG07 >>>>>> serial=ZA1AC912> SATA E1:S3 >>>>> mfi(4) are RAIDs, not HBAs. We do not recommend RAIDs with TrueNAS >>>>> due to problems with hot-plug, disk identification, etc. and so >>>>> have limited experience with them. But I know some of LSI RAIDs >>>>> can be reflashed into equivalent HBAs, so if they share the >>>>> hardware, I can speculate that they may share some issues. >>>> I've just noticed "932G" instead of "8000G". It is obviously a >>>> bigger problem than what we heard for HBAs. It looks like a kind of >>>> problems that should not happen to HBAs, since they should not care >>>> about disk capacity. >>> What does `smartctl -a <device>` report (especially sector sizes)? >>> -m >>>> -- >>>> Alexander Motin >> It's not even making a mfid* node (it is a 4Kn disk) > > Ok, that’s sad (and explains the wrong size calculation as 4096/512=8). > > Is this in HBA mode? (Like Alexander suggested, re-/crossflashing > using an IT firmware might be an option). What controller / firmware > image version is it? > > -m > > >> -- >> Larry Rosenman http://www.lerctr.org/~ler >> Phone: +1 214-642-9640 E-Mail: ler@lerctr.org >> US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106 mfi0@pci0:8:0:0: class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 device=0x0079 subvendor=0x1734 subdevice=0x1176 vendor = 'Broadcom / LSI' device = 'MegaRAID SAS 2108 [Liberator]' class = mass storage subclass = RAID mfi1@pci0:3:0:0: class=0x010400 rev=0x05 hdr=0x00 vendor=0x1000 device=0x0079 subvendor=0x1734 subdevice=0x1176 vendor = 'Broadcom / LSI' device = 'MegaRAID SAS 2108 [Liberator]' class = mass storage subclass = RAID mfi0: <LSI MegaSAS Gen2> port 0xd000-0xd0ff mem 0xfbc9c000-0xfbc9ffff,0xfbcc0000 -0xfbcfffff irq 26 at device 0.0 on pci3 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: FW MaxCmds = 1008, limiting to 128 mfip0: <SCSI Passthrough Bus> on mfi0 mfi0: 10014 (708822708s/0x0020/info) - Shutdown command received from host mfi0: 10015 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 00 79/1000/1176/1734) mfi0: 10016 (boot + 3s/0x0020/info) - Firmware version 2.130.353-2727 mfi0: 10017 (boot + 6s/0x0020/info) - Package version 12.12.0-0174 mfi0: 10018 (boot + 6s/0x0020/info) - Board Revision mfi1: <LSI MegaSAS Gen2> port 0xc000-0xc0ff mem 0xfba9c000-0xfba9ffff,0xfbac0000 -0xfbafffff irq 16 at device 0.0 on pci8 mfi1: Using MSI mfi1: Megaraid SAS driver Ver 4.23 mfi1: FW MaxCmds = 1008, limiting to 128 -- Larry Rosenman http://www.lerctr.org/~ler Phone: +1 214-642-9640 E-Mail: ler@lerctr.org US Mail: 5708 Sabbia Dr, Round Rock, TX 78665-2106
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?5936c6368df3c7d762d4710d23e7c57c>