Skip site navigation (1)Skip section navigation (2)
Date:      Thu, 23 Sep 2010 14:55:02 +0200
From:      Niklas Saers <niklas@saers.com>
To:        freebsd-scsi@freebsd.org
Subject:   mfi - setting up disks
Message-ID:  <2EA9CBBC-3F97-4AF2-BFB5-96DF39FDE376@saers.com>

next in thread | raw e-mail | index | archive | help
Hi guys,
In the SuperMicro system where I had problems with the mpt controller, I =
switched it for a mfi-based controller. I had it set up with 36x RAID0 =
volumes with each their own disk (no way to access the disk otherwise I =
found), and added them to a ZFS system. The numbering became a bit =
weird, so I pulled the disks out one by one and put them back to figure =
out and note down what disk number was in what slot. Only test data on =
my ZFS volume, so I didn't mind that crashing.

Now that all disks have been taken out and put back in one by one, I do:

# mfiutil show volumes
mfi0 Volumes:
  Id     Size    Level   Stripe State  Cache   Name

Whoops, no volumes? That can't be good. I check up on the disks, they're =
all there:

 mfiutil show drives
mfi0 Physical Drives:
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721953> =
SATA enclosure 1, slot 2
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722143> =
SATA enclosure 1, slot 3
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722152> =
SATA enclosure 1, slot 4
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722133> =
SATA enclosure 1, slot 14
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722139> =
SATA enclosure 1, slot 15
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721963> =
SATA enclosure 1, slot 17
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722160> =
SATA enclosure 1, slot 23
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722127> =
SATA enclosure 2, slot 1
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722155> =
SATA enclosure 2, slot 4
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721968> =
SATA enclosure 2, slot 5
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721956> =
SATA enclosure 2, slot 6
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722137> =
SATA enclosure 2, slot 7
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721969> =
SATA enclosure 2, slot 8
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722122> =
SATA enclosure 2, slot 10
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722149> =
SATA enclosure 2, slot 11
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721951> =
SATA enclosure 1, slot 0
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722128> =
SATA enclosure 1, slot 5
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721966> =
SATA enclosure 1, slot 6
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721952> =
SATA enclosure 1, slot 7
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722145> =
SATA enclosure 1, slot 8
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722131> =
SATA enclosure 1, slot 9
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721961> =
SATA enclosure 1, slot 10
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721954> =
SATA enclosure 1, slot 11
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722158> =
SATA enclosure 1, slot 12
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721967> =
SATA enclosure 1, slot 13
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721971> =
SATA enclosure 1, slot 16
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722157> =
SATA enclosure 1, slot 19
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721970> =
SATA enclosure 1, slot 21
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722140> =
SATA enclosure 1, slot 22
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722125> =
SATA enclosure 2, slot 2
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722146> =
SATA enclosure 2, slot 3
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722134> =
SATA enclosure 2, slot 9
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721965> =
SATA enclosure 1, slot 1
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ722151> =
SATA enclosure 1, slot 20
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721964> =
SATA enclosure 2, slot 0
( 1863G) UNCONFIGURED GOOD <SAMSUNG HD203WI 0003 serial=3DS1UYJDWZ721955> =
SATA enclosure 1, slot 18

Well, that's an excelent way of adding the volumes in the sequence they =
appear physically, so I start with the first one from top left:
# mfiutil create raid0 -v E01:S05
Adding drive 26 to array 0
Adding array 0 to volume 0
mfiutil: Command failed: Wrong firmware or drive state
mfiutil: Failed to add volume: Input/output error

Firmware error? Invalid drive state?

# mfiutil good E01:S05
mfiutil: Drive 26 is already in the desired state

Seems to be good... do I have a firmware issue? Check dmesg:

mfi0: <LSI MegaSAS Gen2> port 0xc000-0xc0ff mem =
0xfad7c000-0xfad7ffff,0xfadc0000-0xfadfffff irq 16 at device 0.0 on pci5
mfi0: Megaraid SAS driver Ver 3.00=20
mfi0: 1966 (338565595s/0x0020/info) - Shutdown command received from =
host
mfi0: 1967 (boot + 3s/0x0020/info) - Firmware initialization started =
(PCI ID 0079/1000/9261/1000)
mfi0: 1968 (boot + 3s/0x0020/info) - Firmware version 2.0.03-0673
mfi0: 1969 (boot + 4s/0x0020/info) - Board Revision=20
mfi0: 1970 (boot + 24s/0x0004/info) - Enclosure (SES) discovered on PD =
08(c Port 0 - 3/p1)
mfi0: 1971 (boot + 24s/0x0004/info) - Enclosure (SES) discovered on PD =
09(c Port 0 - 3/p2)
mfi0: 1972 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) =
communication restored
mfi0: 1973 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) =
fan 1 speed changed
mfi0: 1974 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) =
fan 2 speed changed
mfi0: 1975 (boot + 24s/0x0004/info) - Enclosure PD 08(c Port 0 - 3/p1) =
fan 3 speed changed
mfi0: 1976 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) =
communication restored
mfi0: 1977 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) =
fan 1 speed changed
mfi0: 1978 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) =
fan 2 speed changed
mfi0: 1979 (boot + 24s/0x0004/info) - Enclosure PD 09(c Port 0 - 3/p2) =
fan 3 speed changed
mfi0: 1980 (boot + 24s/0x0002/info) - Inserted: Encl PD 08
mfi0: 1981 (boot + 24s/0x0002/info) - Inserted: PD 08(c Port 0 - 3/p1) =
Info: enclPd=3D08, scsiType=3Dd, portMap=3D00, =
sasAddr=3D50030480008fb0fd,0000000000000000
mfi0: 1982 (boot + 24s/0x0002/info) - Inserted: Encl PD 09
mfi0: 1983 (boot + 24s/0x0002/info) - Inserted: PD 09(c Port 0 - 3/p2) =
Info: enclPd=3D09, scsiType=3Dd, portMap=3D00, =
sasAddr=3D50030480008e7b7d,0000000000000000
mfi0: 1984 (boot + 24s/0x0002/info) - Inserted: PD 0a(e0x08/s2)

and then lots of disks.... looks fine, right?

What am I missing? I badly want mfid0-mfid35 back so that I can recreate =
my ZFS and get to work :-)

On a side note, the ZFS I'll make is this, any comments on the =
configuration?

zpool create tank \
raidz2 /dev/mfid0 /dev/mfid1 /dev/mfid2 /dev/mfid3 /dev/mfid4 /dev/mfid5 =
/dev/mfid30 \
raidz2 /dev/mfid6 /dev/mfid7 /dev/mfid8 /dev/mfid9 /dev/mfid10 =
/dev/mfid11 /dev/mfid31 \
raidz2 /dev/mfid12 /dev/mfid13 /dev/mfid14 /dev/mfid15 /dev/mfid16 =
/dev/mfid17 /dev/mfid32 \
raidz2 /dev/mfid18 /dev/mfid19 /dev/mfid20 /dev/mfid21 /dev/mfid22 =
/dev/mfid23 /dev/mfid33 \
raidz2 /dev/mfid24 /dev/mfid25 /dev/mfid26 /dev/mfid27 /dev/mfid28 =
/dev/mfid29 /dev/mfid34 \
spare /dev/mfid35


Cheers

	Nik=



Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?2EA9CBBC-3F97-4AF2-BFB5-96DF39FDE376>