Date: Fri, 17 Feb 2012 09:16:21 -0800 From: Maksim Yevmenkin <maksim.yevmenkin@gmail.com> To: "Desai, Kashyap" <Kashyap.Desai@lsi.com> Cc: "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org> Subject: Re: LSI2008 controller clobbers first disk with new LSI mps driver Message-ID: <CAFPOs6pwb44oNabH5vabDPJyFutMKa5mhgvHY=HkQVpV20YiYw@mail.gmail.com> In-Reply-To: <B2FD678A64EAAD45B089B123FDFC3ED72B96D34500@inbmail01.lsi.com> References: <CAAAm0r2NFhF=eh2bOPMnVN8E6e2o0KfaST0N-M_gWoJHpFOLmQ@mail.gmail.com> <CAAAm0r37H0Pf_HvFTqo%2B1RSCV%2BBkzD5EcXj5M40H9xGREr%2Bfow@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D344CA@inbmail01.lsi.com> <CAAAm0r3MVHZixAKDF=ChD977PbNUC8dyiBFB=NzC30S8FeZ3Nw@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D344F2@inbmail01.lsi.com> <CAAAm0r02bXNve6Do5C1m1RyzLhWA9xx97KKMeLhcHC66U2SF4g@mail.gmail.com> <B2FD678A64EAAD45B089B123FDFC3ED72B96D34500@inbmail01.lsi.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, Feb 17, 2012 at 1:07 AM, Desai, Kashyap <Kashyap.Desai@lsi.com> wro= te: > Jason, > > Can you switch your mail client to "text mode". It is difficult to do inl= ine reply in html format. > > <mpslsi> version 08.55.00.00, <mps> version 11.255.03.00-fbsd and <mps> v= ersion 13.00.00.00-fbsd detected SES device as Target Id =3D 8. > This target Id is retrieved from FW. I am able to map my theory in your c= ase. > > Now, Only problem is when you use "Old mps FreeBSD developers" things are= different (and you would like to see same result using other driver versio= ns ). > Here is a reason why behavior is different. > Our older mps driver does not had mapping code available in few driver re= leases. (see mps_mapping.c file) > Because of that reason Driver will assign Target Id sequential (Internal = sequence) from Event received from FW. > > Summary is all driver behavior with FW 10.00.02.00 is as expected. > > Can you boot into verbose mode and send me log where it actual hangs ? = =A0Also what is your /etc/fstab entry. we've actually had exactly the same problem with 16-port LSI card running IT (non-RAID) firmware. i was looking at the source and found that driver does some sort of persistent mapping for scsi target. personally, i do net get why its required, especially when running in IT mode. to fix this issue we had to "lobotomize" LSI mps(4) card. the following procedure is know to work (i'm typing from memory, so double check) 1) start lsiutil 2) select card 3) enable expert mode 4) select option 33 then sub option 7 (erase all non-production configuration or something like that) 5) return to main menu and select option 61 (reset to all defaults) 6) select option 37 (force full rescan) repeat steps 2 - 6 for every card and reboot. activate LSI configuration bios utility (ctrl+c) and do optional post-configuration (we disable boot order on LSI adapters). save and reboot. we also use hints system to ensure drives/ports stay wired correctly to their respective device nodes. hope this helps. thanks, max > From: Jason Wolfe [mailto:nitroboost@gmail.com] > Sent: Friday, February 17, 2012 2:19 PM > To: Desai, Kashyap > Cc: freebsd-scsi@freebsd.org > Subject: Re: LSI2008 controller clobbers first disk with new LSI mps driv= er > > Kashyap, > > Yes I've run all 4 drivers on the same machine with the 10.00.02.00 FW: > > Old mps from (I believe) FreeBSD developers: > > mps0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd400= 00-0xfbd7ffff irq 26 at device 0.0 on pci4 > mps0: Firmware: 10.00.02.00 > mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRet= ry,EventReplay,HostDisc> > mps0: [ITHREAD] > da0 at mps0 bus 0 scbus0 target 1 lun 0 > da1 at mps0 bus 0 scbus0 target 2 lun 0 > da2 at mps0 bus 0 scbus0 target 3 lun 0 > da3 at mps0 bus 0 scbus0 target 4 lun 0 > da4 at mps0 bus 0 scbus0 target 5 lun 0 > da5 at mps0 bus 0 scbus0 target 6 lun 0 > da6 at mps0 bus 0 scbus0 target 7 lun 0 > da8 at mps0 bus 0 scbus0 target 9 lun 0 > da7 at mps0 bus 0 scbus0 target 8 lun 0 > da10 at mps0 bus 0 scbus0 target 11 lun 0 > da9 at mps0 bus 0 scbus0 target 10 lun 0 > da11 at mps0 bus 0 scbus0 target 12 lun 0 > ses0 at mps0 bus 0 scbus0 target 13 lun 0 > ses0: <LSI CORP SAS2X28 0717> Fixed Enclosure Services SCSI-5 device > ses0: 600.000MB/s transfers > ses0: Command Queueing enabled > ses0: SCSI-3 SES Device > > > > mpslsi.ko binary from 2011: > > mpslsi0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd= 40000-0xfbd7ffff irq 26 at device 0.0 on pci4 > mpslsi0: Firmware: 10.00.02.00, Driver: 08.55.00.00 > mpslsi0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,Trans= Retry,EventReplay,HostDisc> > mpslsi0: [ITHREAD] > ses0 at mpslsi0 bus 0 target 8 lun 0 > ses0: <LSI CORP SAS2X28 0717> Fixed Enclosure Services SCSI-5 device > ses0: 600.000MB/s transfers > ses0: Command Queueing Enabled > ses0: SCSI-3 SES Device > da0 at mpslsi0 bus 0 target 9 lun 0 > da1 at mpslsi0 bus 0 target 10 lun 0 > da2 at mpslsi0 bus 0 target 11 lun 0 > da3 at mpslsi0 bus 0 target 12 lun 0 > da4 at mpslsi0 bus 0 target 13 lun 0 > da5 at mpslsi0 bus 0 target 14 lun 0 > da6 at mpslsi0 bus 0 target 15 lun 0 > da7 at mpslsi0 bus 0 target 16 lun 0 > da8 at mpslsi0 bus 0 target 17 lun 0 > da9 at mpslsi0 bus 0 target 18 lun 0 > da10 at mpslsi0 bus 0 target 19 lun 0 > > > > 11.255.03.00-fbsd: > > mps0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd400= 00-0xfbd7ffff irq 26 at device 0.0 on pci4 > mps0: Firmware: 10.00.02.00, Driver: 11.255.03.00-fbsd > mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRet= ry,EventReplay,HostDisc> > mps0: [ITHREAD] > ses0 at mps0 bus 0 scbus0 target 8 lun 0 > ses0: <LSI CORP SAS2X28 0717> Fixed Enclosure Services SCSI-5 device > ses0: 600.000MB/s transfers > ses0: Command Queueing enabled > ses0: SCSI-3 SES Device > da0 at mps0 bus 0 scbus0 target 9 lun 0 > da1 at mps0 bus 0 scbus0 target 10 lun 0 > da2 at mps0 bus 0 scbus0 target 11 lun 0 > da3 at mps0 bus 0 scbus0 target 12 lun 0 > da4 at mps0 bus 0 scbus0 target 13 lun 0 > da5 at mps0 bus 0 scbus0 target 14 lun 0 > da6 at mps0 bus 0 scbus0 target 15 lun 0 > da7 at mps0 bus 0 scbus0 target 16 lun 0 > da8 at mps0 bus 0 scbus0 target 17 lun 0 > da9 at mps0 bus 0 scbus0 target 18 lun 0 > da10 at mps0 bus 0 scbus0 target 19 lun 0 > > > > Current 13.00.00.00-fbsd: > > mps0: <LSI SAS2008> port 0xe000-0xe0ff mem 0xfbd3c000-0xfbd3ffff,0xfbd400= 00-0xfbd7ffff irq 26 at device 0.0 on pci4 > mps0: Firmware: 10.00.02.00, Driver: 13.00.00.00-fbsd > mps0: IOCCapabilities: 1285c<ScsiTaskFull,DiagTrace,SnapBuf,EEDP,TransRet= ry,EventReplay,HostDisc> > mps0: [ITHREAD] > da1 at mps0 bus 0 scbus0 target 10 lun 0 > da0 at mps0 bus 0 scbus0 target 9 lun 0 > da2 at mps0 bus 0 scbus0 target 11 lun 0 > da4 at mps0 bus 0 scbus0 target 13 lun 0 > da5 at mps0 bus 0 scbus0 target 14 lun 0 > da3 at mps0 bus 0 scbus0 target 12 lun 0 > da7 at mps0 bus 0 scbus0 target 16 lun 0 > da8 at mps0 bus 0 scbus0 target 17 lun 0 > da6 at mps0 bus 0 scbus0 target 15 lun 0 > da9 at mps0 bus 0 scbus0 target 18 lun 0 > da10 at mps0 bus 0 scbus0 target 19 lun 0 > ses0 at mps0 bus 0 scbus0 target 8 lun 0 > ses0: <LSI CORP SAS2X28 0717> Fixed Enclosure Services SCSI-5 device > ses0: 600.000MB/s transfers > ses0: Command Queueing enabled > ses0: SCSI-3 SES Device > > Jason > > On Fri, Feb 17, 2012 at 1:24 AM, Desai, Kashyap <Kashyap.Desai@lsi.com<ma= ilto:Kashyap.Desai@lsi.com>> wrote: > Jason, > > Me also surprised when I see so many queries on <mps>. > Really wonderful to know that there are good amount of <mpslsi> driver us= e as well. > > I was under impression that if you keep same FW and just change Driver, t= here should not be any difference in Target IDs assigned to Device connecte= d behind that HBA. > > Is this possible for you to keep everything unchanged and just change Dri= ver version and see how things behaves. Please share "dmesg" logs of your b= oth drivers. =A0BTW, did you tested "13.00.00.00-fbsd" and "11.00.00.00" = =A0on same machine ? > > FYI: We never observe this kind of issue in our lab. > > ` Kashyap > > From: Jason Wolfe [mailto:nitroboost@gmail.com<mailto:nitroboost@gmail.co= m>] > Sent: Friday, February 17, 2012 1:31 PM > To: Desai, Kashyap > Cc: freebsd-scsi@freebsd.org<mailto:freebsd-scsi@freebsd.org> > Subject: Re: LSI2008 controller clobbers first disk with new LSI mps driv= er > Kashyap, > > Ah a response from LSI, that's a pleasant surprise :) =A0Everything you'v= e stated looks correct to me, the FreeBSD developed driver that has been re= placed by the LSI driver has no issues with either firmware. =A0Your likely= aware, but just to confirm, here is the history of the 3 various LSI drive= rs that have the issue on the 10.00.02.00 FW: > > 11.00.00.00 - binary driver I had received from you guys in mid =A02011, = mpslsi.ko, one for each 7.2-RELEASE and 8.2-RELEASE > > 11.255.03.00-fbsd - initial LSI driver committed to 8-STABLE on 2/2, r230= 922 > > 13.00.00.00-fbsd - commited to 8-STABLE on 2/14, r231680 > > I have about 40 boxes with the 10.00.02.00 FW I've tested, so I'm fairly = certain it's not bad hardware or a fluke. =A0You guys haven't seen anything= like this in house? =A0I'd hate to hear I have to update the FW on these b= oxes as they are all quite a ways from me, though it seems there is some wa= y to work around the behavior in the driver as the FreeBSD one does? =A0I h= ave a few of these boxes out of service so I'm game to try some things out = should that help. > > Thank for the response, > Jason > > On Thu, Feb 16, 2012 at 11:29 PM, Desai, Kashyap <Kashyap.Desai@lsi.com<m= ailto:Kashyap.Desai@lsi.com>> wrote: > Jason, > > I have gone through your data provided in this thread. It is well underst= ood because of your descriptive data. > > So What I understood here is: > > 1. You tested with HBA Fw "07.00.00.00" and "10.00.02.00" > > 2. you have run your test on two different LSI BIOS versions. > Grabbed from below line. > MPT2BIOS-7.11.00.00 (2010.07.29) / PRODUCT REVISION 7.00.00.00 > MPT2BIOS-7.19.00.00 (2011.05.16) / PRODUCT REVISION 10.00.02.00 > > Now I am able to see below three difference in your setup. > > See FW version and check starting target id, all three has different way = of assigning TargetIDs. > For first two case target id start with "8" but SES device assignment is = different. > Last case target id start with "1" > > > mps0: Firmware: 07.00.00.00, Driver: 11.255.03.00-fbsd (OR) 13.00.00.00-f= bsd >> > <SEAGATE ST91000640SS 0001> at scbus0 target 8 lun 0 (pass0,da0) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (pass1,da1) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (pass2,da2) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (pass3,da3) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (pass4,da4) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 13 lun 0 (pass5,da5) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 14 lun 0 (pass6,da6) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 15 lun 0 (pass7,da7) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 16 lun 0 (pass8,da8) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 17 lun 0 (pass9,da9) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 18 lun 0 (pass10,da10) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 19 lun 0 (pass11,da11) >> > <LSI CORP SAS2X28 0717> at scbus0 target 20 lun 0 (ses0,pass12) > > mps0: Firmware: 10.00.02.00, Driver: 13.00.00.00-fbsd > mps0: Firmware: 10.00.02.00, Driver: 11.00.00.00 (OR) 8.2-STABLE Inbox >> > <LSI CORP SAS2X28 0717> at scbus0 target 8 lun 0 (ses0,pass0) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (da0,pass1) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (da1,pass2) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (da2,pass3) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (da3,pass4) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 13 lun 0 (da4,pass5) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 14 lun 0 (da5,pass6) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 15 lun 0 (da6,pass7) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 16 lun 0 (da7,pass8) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 17 lun 0 (da8,pass9) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 18 lun 0 (da9,pass10) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 19 lun 0 (da10,pass11) > > On the FBSD developed driver active in 8-STABLE prior to the LSI > Release (Firmware: 10.00.02.00) > >> > <SEAGATE ST91000640SS 0001> at scbus0 target 1 lun 0 (pass0,da0) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 2 lun 0 (pass1,da1) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 3 lun 0 (pass2,da2) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 4 lun 0 (pass3,da3) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 5 lun 0 (pass4,da4) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 6 lun 0 (pass5,da5) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 7 lun 0 (pass6,da6) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 8 lun 0 (pass7,da7) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 9 lun 0 (pass8,da8) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 10 lun 0 (pass9,da9) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 11 lun 0 (pass10,da10) >> > <SEAGATE ST91000640SS 0001> at scbus0 target 12 lun 0 (pass11,da11) >> > <LSI CORP SAS2X28 0717> at scbus0 target 13 lun 0 (ses0,pass12) > > > In summary, =A0(please confirm) > 1. you have not seen any issue if you use "07.00.00.00" FW version. > 2. _but_ when you use "10.00.02.00" FW, with "13.00.00.00-fbsd" driver ve= rsion you are seeing > SES is detected before Drives as pass0. > 3. When you use "10.00.02.00" FW with 8-STABLE inbox FBSD driver, you are= finding SES device detected after Drives. > > > All driver is doing here is asking CAM layer to scan Bus when there is an= y device added on that bus. > So depending upon actual target Id =A0assigned by FW, it will be detected= to camcontrol. > > So bottom line is FW plays major role in sequencing Drives behind LSI con= troller. > > =A0~ Kashyap > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAFPOs6pwb44oNabH5vabDPJyFutMKa5mhgvHY=HkQVpV20YiYw>