Date: Thu, 23 Sep 2010 14:55:02 +0200 From: Niklas Saers <niklas@saers.com> To: freebsd-scsi@freebsd.org Subject: mfi - setting up disks Message-ID: <2EA9CBBC-3F97-4AF2-BFB5-96DF39FDE376@saers.com>
next in thread | raw e-mail | index | archive | help
Hi guys, In the SuperMicro system where I had problems with the mpt controller, I = switched it for a mfi-based controller. I had it set up with 36x RAID0 = volumes with each their own disk (no way to access the disk otherwise I = found), and added them to a ZFS system. The numbering became a bit = weird, so I pulled the disks out one by one and put them back to figure = out and note down what disk number was in what slot. Only test data on = my ZFS volume, so I didn't mind that crashing. Now that all disks have been taken out and put back in one by one, I do: # mfiutil show volumes mfi0 Volumes: Id Size Level Stripe State Cache Name Whoops, no volumes? That can't be good. I check up on the disks, they're = all there: mfiutil show drives mfi0 Physical Drives: ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721953> = SATA enclosure 1, slot 2 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722143> = SATA enclosure 1, slot 3 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722152> = SATA enclosure 1, slot 4 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722133> = SATA enclosure 1, slot 14 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722139> = SATA enclosure 1, slot 15 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721963> = SATA enclosure 1, slot 17 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722160> = SATA enclosure 1, slot 23 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722127> = SATA enclosure 2, slot 1 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722155> = SATA enclosure 2, slot 4 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721968> = SATA enclosure 2, slot 5 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721956> = SATA enclosure 2, slot 6 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722137> = SATA enclosure 2, slot 7 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721969> = SATA enclosure 2, slot 8 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722122> = SATA enclosure 2, slot 10 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722149> = SATA enclosure 2, slot 11 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721951> = SATA enclosure 1, slot 0 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722128> = SATA enclosure 1, slot 5 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721966> = SATA enclosure 1, slot 6 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721952> = SATA enclosure 1, slot 7 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722145> = SATA enclosure 1, slot 8 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722131> = SATA enclosure 1, slot 9 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721961> = SATA enclosure 1, slot 10 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721954> = SATA enclosure 1, slot 11 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722158> = SATA enclosure 1, slot 12 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721967> = SATA enclosure 1, slot 13 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721971> = SATA enclosure 1, slot 16 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722157> = SATA enclosure 1, slot 19 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721970> = SATA enclosure 1, slot 21 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722140> = SATA enclosure 1, slot 22 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722125> = SATA enclosure 2, slot 2 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722146> = SATA enclosure 2, slot 3 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722134> = SATA enclosure 2, slot 9 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721965> = SATA enclosure 1, slot 1 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722151> = SATA enclosure 1, slot 20 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721964> = SATA enclosure 2, slot 0 ( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721955> = SATA enclosure 1, slot 18 Well, that's an excelent way of adding the volumes in the sequence they = appear physically, so I start with the first one from top left: # mfiutil create raid0 -v E01:S05 Adding drive 26 to array 0 Adding array 0 to volume 0 mfiutil: Command failed: Wrong firmware or drive state mfiutil: Failed to add volume: Input/output error Firmware error? Invalid drive state? # mfiutil good E01:S05 mfiutil: Drive 26 is already in the desired state Seems to be good... do I have a firmware issue? Check dmesg: mfi0: <LSI MegaSAS Gen2> port 0xc000-0xc0ff mem = 0xfad7c000-0xfad7ffff,0xfadc0000-0xfadfffff irq 16 at device 0.0 on pci5 mfi0: Megaraid SAS driver Ver 3.00=20 mfi0: 1966 (338565595s/0x0020/info) - Shutdown command received from = host mfi0: 1967 (boot + 3s/0x0020/info) - Firmware initialization started = (PCI ID 0079/1000/9261/1000) mfi0: 1968 (boot + 3s/0x0020/info) - Firmware version 2.0.03-0673 mfi0: 1969 (boot + 4s/0x0020/info) - Board Revision=20 mfi0: 1970 (boot + 24s/0x0004/info) - Enclosure (SES) discovered on PD = 08(c Port 0 - 3/p1) mfi0: 1971 (boot + 24s/0x0004/info) - Enclosure (SES) discovered on PD = 09(c Port 0 - 3/p2) mfi0: 1972 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) = communication restored mfi0: 1973 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) = fan 1 speed changed mfi0: 1974 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) = fan 2 speed changed mfi0: 1975 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) = fan 3 speed changed mfi0: 1976 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) = communication restored mfi0: 1977 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) = fan 1 speed changed mfi0: 1978 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) = fan 2 speed changed mfi0: 1979 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) = fan 3 speed changed mfi0: 1980 (boot + 24s/0x0002/info) - Inserted: Encl PD 08 mfi0: 1981 (boot + 24s/0x0002/info) - Inserted: PD 08(c Port 0 - 3/p1) = Info: enclPd=3D08, scsiType=3Dd, portMap=3D00, = sasAddr=3D50030480008fb0fd,0000000000000000 mfi0: 1982 (boot + 24s/0x0002/info) - Inserted: Encl PD 09 mfi0: 1983 (boot + 24s/0x0002/info) - Inserted: PD 09(c Port 0 - 3/p2) = Info: enclPd=3D09, scsiType=3Dd, portMap=3D00, = sasAddr=3D50030480008e7b7d,0000000000000000 mfi0: 1984 (boot + 24s/0x0002/info) - Inserted: PD 0a(e0x08/s2) and then lots of disks.... looks fine, right? What am I missing? I badly want mfid0-mfid35 back so that I can recreate = my ZFS and get to work :-) On a side note, the ZFS I'll make is this, any comments on the = configuration? zpool create tank \ raidz2 /dev/mfid0 /dev/mfid1 /dev/mfid2 /dev/mfid3 /dev/mfid4 /dev/mfid5 = /dev/mfid30 \ raidz2 /dev/mfid6 /dev/mfid7 /dev/mfid8 /dev/mfid9 /dev/mfid10 = /dev/mfid11 /dev/mfid31 \ raidz2 /dev/mfid12 /dev/mfid13 /dev/mfid14 /dev/mfid15 /dev/mfid16 = /dev/mfid17 /dev/mfid32 \ raidz2 /dev/mfid18 /dev/mfid19 /dev/mfid20 /dev/mfid21 /dev/mfid22 = /dev/mfid23 /dev/mfid33 \ raidz2 /dev/mfid24 /dev/mfid25 /dev/mfid26 /dev/mfid27 /dev/mfid28 = /dev/mfid29 /dev/mfid34 \ spare /dev/mfid35 Cheers Nik=
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2EA9CBBC-3F97-4AF2-BFB5-96DF39FDE376>