Date: Mon, 20 Aug 2012 07:25:13 -0700 From: Josh Paetzel <josh@tcbug.org> To: Don Lewis <truckman@FreeBSD.org> Cc: freebsd-stable@FreeBSD.org Subject: Re: LSI 9240-4i 4K alignment Message-ID: <503248C9.8000509@tcbug.org> In-Reply-To: <201208200719.q7K7JYqZ011355@gw.catspoiler.org> References: <201208200719.q7K7JYqZ011355@gw.catspoiler.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On 08/20/2012 00:19, Don Lewis wrote: > On 19 Aug, Josh Paetzel wrote: >> On 08/19/2012 14:04, Steven Hartland wrote: > >>> HBA's are the way to go if your using ZFS to manage the disks, you only >>> need RAID if your using a FS which doesn't manage the disk side well >>> such as UFS. >>> >>> Its often quite common for RAID controllers to actually be slower >>> vs RAID controllers as the RAID stack can get in the way. > > Any idea of what kind of performance penalty I might see by using the > RAID firmware in JBOD mode vs flashing the IT firmware? > I don't have any current numbers, on ZFS v14 14 RAID controllers were actually a bit faster, but that's all changed dramatically. On our high end stuff we can get HBAs to go over 25% faster than high end RAID controllers, like the 9260/9280, but we don't test with RAID controllers anymore at all, so I don't have up to the minute info. ZFS does block checksums, and so do LSI mfi cards, even when in "JBOD" mode, you also can't bypass the cache on the card without a huge performance hit, so you end up with 256MB or whatever in between your disks and the OS. In addition because the 9240 is based on the 2008 which lacks hardware assist for RAID5/6 those two modes are done in software, so you take another hit there. Advantages: ZFS doesn't work with hot spares as of this moment on FreeBSD, but LSI controllers do, so if your strategy involves hot spares the RAID card is the better choice. LSI controllers can be set to auto-replace, ZFS can't. Enclosure management works better on RAID controllers than through FreeBSD in many cases. >> Just to clear up, >> >> The 9240 is a sas2008 based card with the megaraid software on top of >> it. In it's default config from LSI the FreeBSD mfi will recognize it >> in later versions of FreeBSD (The upcoming 9.1 for sure) Older >> versions of mfi will not recognize it. >> >> The card can be flashed with IT firmware and then becomes a 9211 HBA, >> but it's a bit more expensive than a 9211 is so that doesn't make sense >> to do in many cases. > > The price difference was pretty minor when I looked. Confusingly > enough, the 9211 HBA also has some RAID capabilities. > > For me, the biggest advantage of the 9211 would be that it would have > allowed me to use shorter cables. > >> On the dmesg posted the firmware on the card is phase 11. This *must* >> be in lockstep with the driver version or the card may not play nicely. >> FreeBSD 8.3 and 9.0 have v13 of the driver, the upcoming 9.1 will have >> v14. Note that v14 fixes a *ton* of stability bugs, including issues >> where bad drives would hang the controller or prevent systems from booting. > > Where do those version numbers come from? The mfi driver in 9.0-RELEASE > claims to be version 3.00 and the the driver in 9.1 claims to be version > 4.23. > I was talking about mps, not mfi. The dmesg I was responding to showed an mps. > This is what shows up in dmesg on my machine: > > mfi0: <Drake Skinny> port 0xce00-0xceff mem 0xfcefc000-0xfcefffff,0xfce80000-0xf > cebffff irq 18 at device 0.0 on pci1 > mfi0: Using MSI > mfi0: Megaraid SAS driver Ver 4.23 > mfi0: 333 (398082533s/0x0020/info) - Shutdown command received from host > mfi0: 334 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 > /1000/9240/1000) > mfi0: 335 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 > mfi0: 336 (boot + 5s/0x0020/info) - Board Revision 04A > mfi0: 337 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 > /1000/9240/1000) > mfi0: 338 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 > mfi0: 339 (boot + 5s/0x0020/info) - Board Revision 04A > mfi0: 340 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 > /1000/9240/1000) > mfi0: 341 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 > mfi0: 342 (boot + 5s/0x0020/info) - Board Revision 04A > mfi0: 343 (boot + 3s/0x0020/info) - Firmware initialization started (PCI ID 0073 > /1000/9240/1000) > mfi0: 344 (boot + 3s/0x0020/info) - Firmware version 2.70.04-0862 > mfi0: 345 (boot + 5s/0x0020/info) - Board Revision 04A > mfi0: 346 (398759025s/0x0020/info) - Time established as 08/20/12 6:23:45; (25 > seconds since power on) > mfi0: 347 (398759051s/0x0020/info) - Time established as 08/20/12 6:24:11; (51 > seconds since power on) > mfi0: 348 (398759078s/0x0020/WARN) - Patrol Read can't be started, as PDs are ei > ther not ONLINE, or are in a VD with an active process, or are in an excluded VD > > > % mfiutil show firmware > mfi0 Firmware Package Version: 20.5.1-0003 > mfi0 Firmware Images: > Name Version Date Time Status > BIOS 4.14.00 active > PCLI 03.02-001:#%00008 Feb 09 2010 13:09:06 active > BCON 4.0-22-e_10-Rel Mar 11 2010 12:38:08 active > NVDT 3.04.03-0002 Apr 05 2010 18:50:27 active > APP 2.70.04-0862 May 05 2010 18:12:07 active > BTBL 2.01.00.00-0019 May 14 2009 15:52:08 active > > > The only firmware file on LSI's web site for the 9240-8i is version > 20.10.1-107, which appears to be newer than what is on the card if the > 20.5.1-0003 is the version number that I should be looking at. Is the > BIOS Version 4.14 the v14 version that you mention above? > > If the FreeBSD mfi driver expects a certain firmware version, shouldn't > it complain if it doesn't find it? > I'm not sure if mfi has the same hard requirements for firmware that mps has, but if it requires a certain firmware version it would be reasonable to complain if it wasn't there, or auto flash the firmware ala chelsio 10Gbe NICS. Thanks, Josh
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?503248C9.8000509>