Date: Sun, 21 Oct 2012 03:20:19 +0100 From: "Steven Hartland" <killing@multiplay.co.uk> To: "Dennis Glatting" <freebsd@pki2.com>, <freebsd-fs@FreeBSD.org> Subject: Re: ZFS HBAs + LSI chip sets (Was: ZFS hang (system #2)) Message-ID: <897B9997FC6547C9935A8FC3801AB832@multiplay.co.uk> References: <1350698905.86715.33.camel@btw.pki2.com> <1350711509.86715.59.camel@btw.pki2.com> <50825598.3070505@FreeBSD.org> <1350744349.88577.10.camel@btw.pki2.com> <1350765093.86715.69.camel@btw.pki2.com> <508322EC.4080700@FreeBSD.org> <1350778257.86715.106.camel@btw.pki2.com>
next in thread | previous in thread | raw e-mail | index | archive | help
----- Original Message -----
From: "Dennis Glatting" <freebsd@pki2.com>
To: <freebsd-fs@FreeBSD.org>
Sent: Sunday, October 21, 2012 1:10 AM
Subject: ZFS HBAs + LSI chip sets (Was: ZFS hang (system #2))
>
> I chosen the LSI2008 chip set because the code was donated by LSI, and
> they therefore demonstrated interest in supporting their products under
> FreeBSD, and that chip set is found in a lot of places, notably
> Supermicro boards. Additionally, there were stories of success on the
> lists for several boards. That said, I have received private email from
> others expressing frustration with ZFS and the "hang" problems, which I
> believe are also the LSI chips.
>
> I have two questions for the broader list:
>
> 1) What HBAs are you using for ZFS and what is your level
> of success/stability? Also, what is your load?
>
> 2) How well is the LSI chip sets supported under FreeBSD?
>
> I'm not all that crazy on the idea of replacing eight to ten LSI boards
> but I am less crazy on the idea of this problem's continuance.
>
> I do have three Areca 1880i boards (not used for ZFS, rather dumb RAID)
> however two of them failed within the first year for no apparent reason
> -- I just walked in my lab one day and was greeted by hosed systems.
> Consequently, I'm not all that keen on Areca.
>
> I am in the process of releasing a spec for purchase of a Supermicro in
> a SC848 chassis, and an identical one next FY. I would appreciate a clue
> as to what HBA would be a good/better choice. The vendor spec'd
> Supermicro boards with the LSI2008 chips. I own four or six of the
> AOC-USAS2-L8e boards, two of which I destroyed, but there wasn't much
> difference between them and the 9211.
We have a number of machines using mps although under 8.2 which is
quite a old revision of the driver, 8.3 + had quite some changes
committed by LSI iirc.
With those machines not had a single problem under ZFS but I'm not sure
how highly loaded the client has them.
To give you an idea of the specs:-
One of said boxes is a backups box with 32TB array (2 x 16TB raidz2)
and a mirrored OS + dual SSD cache disks across 3x LSI 2008's on firmware
11.00.00.00
Another is DB box with 10TB array (5 x 2TB mirror) with dual SSD cache
and mirrored SSD logs again 11.00.00.00 firmware.
They are all supermicro machines with retail LSI controllers
(not supermicro but that shouldn't make much difference)
Not totally relevant but worth mentioning is that we're currently having
some serious issues with other supermicro kit when trying to run SATA 3
speeds off the onboard Intel patsburg controller via the hotswap backplane.
When we do so the disks randomly stall waiting for timeout or drop
completely.
We have determined the issue is the backplane that connects the MB to
the hotswap backplane as when the disks are corrected directly using
SATA cables the problem goes away. We're currently waiting for LSI
2008 backplane replacements which we hope will solve the problem.
Regards
Steve
ZFS.
================================================
This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it.
In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337
or return the E.mail to postmaster@multiplay.co.uk.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?897B9997FC6547C9935A8FC3801AB832>