Date: Sun, 19 Aug 2012 10:58:05 -0700 (PDT) From: Don Lewis <truckman@FreeBSD.org> To: gkontos.mail@gmail.com Cc: freebsd-stable@FreeBSD.org Subject: Re: LSI 9240-4i 4K alignment Message-ID: <201208191758.q7JHw5xG010417@gw.catspoiler.org> In-Reply-To: <CA%2BdUSyrdPsDJyZ0Ow3%2B6sNzc10we2BGV10tUYtzkuE5u9Ook2A@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 8 Aug, George Kontostanos wrote: > Hi all, > > We have a server with a LSI 9240-4i controller configured in JBOD with > 4 SATA disks. Running FreeBSD 9.1-Beta1: > > Relevant dmesg: > > FreeBSD 9.1-BETA1 #0: Thu Jul 12 09:38:51 UTC 2012 > root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 > CPU: Intel(R) Xeon(R) CPU E31230 @ 3.20GHz (3200.09-MHz K8-class CPU) > Origin = "GenuineIntel" Id = 0x206a7 Family = 6 Model = 2a Stepping = 7 > Features=0xbfebfbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,DTS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,PBE> > Features2=0x1fbae3ff<SSE3,PCLMULQDQ,DTES64,MON,DS_CPL,VMX,SMX,EST,TM2,SSSE3,CX16,xTPR,PDCM,PCID,SSE4.1,SSE4.2,x2APIC,POPCNT,TSCDLT,AESNI,XSAVE,OSXSAVE,AVX> > AMD Features=0x28100800<SYSCALL,NX,RDTSCP,LM> > AMD Features2=0x1<LAHF> > TSC: P-state invariant, performance statistics > real memory = 17179869184 (16384 MB) > avail memory = 16471670784 (15708 MB) > ... > mfi0: <Drake Skinny> port 0xe000-0xe0ff mem > 0xf7a60000-0xf7a63fff,0xf7a00000-0xf7a3ffff irq 16 at device 0.0 on > pci1 > mfi0: Using MSI > mfi0: Megaraid SAS driver Ver 4.23 > ... > mfi0: 321 (397672301s/0x0020/info) - Shutdown command received from host > mfi0: 322 (boot + 3s/0x0020/info) - Firmware initialization started > (PCI ID 0073/1000/9241/1000) > mfi0: 323 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 > mfi0: 324 (boot + 3s/0x0020/info) - Firmware initialization started > (PCI ID 0073/1000/9241/1000) > mfi0: 325 (boot + 3s/0x0020/info) - Firmware version 2.130.354-1664 > mfi0: 326 (boot + 5s/0x0020/info) - Package version 20.10.1-0107 > mfi0: 327 (boot + 5s/0x0020/info) - Board Revision 03A > mfi0: 328 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) > ... > mfisyspd0 on mfi0 > mfisyspd0: 1907729MB (3907029168 sectors) SYSPD volume > mfisyspd0: SYSPD volume attached > mfisyspd1 on mfi0 > mfisyspd1: 1907729MB (3907029168 sectors) SYSPD volume > mfisyspd1: SYSPD volume attached > mfisyspd2 on mfi0 > mfisyspd2: 1907729MB (3907029168 sectors) SYSPD volume > mfisyspd2: SYSPD volume attached > mfisyspd3 on mfi0 > mfisyspd3: 1907729MB (3907029168 sectors) SYSPD volume > mfisyspd3: SYSPD volume attached > ... > mfi0: 329 (boot + 25s/0x0002/info) - Inserted: PD 04(e0xff/s3) Info: > enclPd=ffff, scsiType=0, portMap=00, > sasAddr=4433221100000000,0000000000000000 > mfi0: 330 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) > mfi0: 331 (boot + 25s/0x0002/info) - Inserted: PD 05(e0xff/s1) Info: > enclPd=ffff, scsiType=0, portMap=02, > sasAddr=4433221102000000,0000000000000000 > mfi0: 332 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) > mfi0: 333 (boot + 25s/0x0002/info) - Inserted: PD 06(e0xff/s2) Info: > enclPd=ffff, scsiType=0, portMap=03, > sasAddr=4433221101000000,0000000000000000 > mfi0: 334 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) > mfi0: 335 (boot + 25s/0x0002/info) - Inserted: PD 07(e0xff/s0) Info: > enclPd=ffff, scsiType=0, portMap=01, > sasAddr=4433221103000000,0000000000000000 > mfi0: 336 (397672376s/0x0020/info) - Time established as 08/07/12 > 16:32:56; (28 seconds since power on) > > The problem: > > When trying to create a RaidZ pool using gpart and perform a 4K > alignment using gnop, we get the follwoing error immediately after > exporting the pool and destroying the .nop devices: > > id: 8043746387654554958 > state: FAULTED > status: One or more devices contains corrupted data. > action: The pool cannot be imported due to damaged devices or data. > The pool may be active on another system, but can be imported using > the '-f' flag. > see: http://illumos.org/msg/ZFS-8000-5E > config: > > Pool FAULTED corrupted data > raidz1-0 ONLINE > 13283347160590042564 UNAVAIL corrupted data > 16981727992215676534 UNAVAIL corrupted data > 6607570030658834339 UNAVAIL corrupted data > 3435463242860701988 UNAVAIL corrupted data I'm planning on doing something similar, but I'm curious about how gnop and GPT labels interact. I want to partition the drives for my pool slightly on the small side so that I'm less likely to run into problems if I have to replace a drive in the future. If I used gpart to create and label a GPT partition on the drive, the partition will show up as /dev/gpt/label. The gnop man page says that running gnop on dev creates /dev/dev.nop. What happens if you gnop /dev/gpt/label? Is this what you are doing? > When we use glabel for the same purpose with the combination of gnop, > the pool imports fine. > > Any suggestions? It should be sufficient to only gnop one of the devices. You should be able to create the pool with only one gnop device to get the ashift value that you desire, export the pool, destroy the .nop device, and import the pool. If it things the device is corrupted (which seems like a bug of some sort), then ZFS should be able to resilver it.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201208191758.q7JHw5xG010417>