Date: Tue, 18 Nov 2014 10:48:36 -0500 From: Mike Tancsa <mike@sentex.net> To: FreeBSD Hackers <freebsd-hackers@freebsd.org> Subject: Re: OT-ish SATA port replicators vs. SAS "expanders" Message-ID: <546B6A54.1060403@sentex.net> In-Reply-To: <20141117225044.GS24601@funkthat.com> References: <CACpH0Me7Y%2Bm6cENsg2otcP9agDp%2BwKPGnPxDafFfySPNkhsKBA@mail.gmail.com> <CAJm423_%2B5wL4G48ftVOmmkyyjSpekS3=sv801pyquRkqcz=fTQ@mail.gmail.com> <CACpH0MdfzpTXceHnRkAo60ztu4Bo%2BbKSpOYZJyV6goY0Q0TbUg@mail.gmail.com> <54693866.9040009@sentex.net> <CAJm4238NDHV_uzertrrcce9kwFwyKoYY6pO5E58s-2DoZkwm-w@mail.gmail.com> <20141117225044.GS24601@funkthat.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On 11/17/2014 5:50 PM, John-Mark Gurney wrote:
> Brandon Vincent wrote this message on Sun, Nov 16, 2014 at 17:26 -0700:
>> On Sun, Nov 16, 2014 at 4:51 PM, Mike Tancsa <mike@sentex.net> wrote:
>>> I don think a port multiplier is the same as a SAS expander and a SAS/SATA
>>> controller does not necessarily support port multipliers.
>>
>> You're absolutely correct. The SATA specifications if I recall
>> correctly, outline that providing support for port multipliers is
>> optional. Not to mention, the vast majority of chipsets that do have
>> support for port multipliers only provide the slower command-based
>> switching method [1], The LSI SAS 2008 chipset is not what you
>> typically find on the cheap consumer "fake RAID" cards which most
>> people have [2].
>>
>> Most people who purchase this card are purchasing it for SAS, true
>> RAID, performance, and reliability. Thus, LSI is only going to provide
>> support for the far superior SAS expander technology.
>
> If you go w/ SATA PMs, make sure you get a controller that supports
> FIS-base switching instead of command based switching... You'll get
> much better performance w/ NCQ support... As for if the LSI SAS 2008
> will support FBS, probably not, as it'll require the driver and/or
> firmware to be aware of how to handle things...
Performance was pretty good, but reliability was really poor for us.
Not sure if it was the cages, cables, or a firmware issue in the card
(SiI3132) or cages or a driver issue. But we would regularly see errors
such as
kernel: siisch0: Timeout on slot 18
kernel: siisch0: siis_timeout is 00040000 ss 001ff800 rs 001ff800 es
00000000 sts 80112000 serr 00000000
kernel: siisch0: ... waiting for slots 001bf800
kernel: siisch0: Timeout on slot 19
kernel: siisch2: Error while READ LOG EXT
kernel: (ada10:siisch2:0:2:0): WRITE_FPDMA_QUEUED. ACB: 61 ab 95 98 71
40 b6 00 00 00 00 00
kernel: (ada10:siisch2:0:2:0): CAM status: ATA Status Error
kernel: (ada10:siisch2:0:2:0): ATA status: 00 ()
kernel: (ada10:siisch2:0:2:0): RES: 00 00 00 00 00 00 00 00 00 00 00
kernel: (ada10:siisch2:0:2:0): Retrying command
kernel: siisch2: Error while READ LOG EXT
But after having swapped things far too many times, we switched to a
pair of LSI 9240-8i, and using the same disks (wiped and started fresh),
we have yet to get any of the old errors
prior to re-using the disks (there were no SMART errors logged), we did
a long test as well as wrote to every sector (no errors). Two weeks
later, still no errors. Even better performance too.
pool: zbackup1
state: ONLINE
scan: none requested
config:
NAME STATE READ WRITE CKSUM
zbackup1 ONLINE 0 0 0
raidz2-0 ONLINE 0 0 0
da0p1 ONLINE 0 0 0
da1p1 ONLINE 0 0 0
da2p1 ONLINE 0 0 0
da3p1 ONLINE 0 0 0
da4p1 ONLINE 0 0 0
da5p1 ONLINE 0 0 0
raidz2-2 ONLINE 0 0 0
da6 ONLINE 0 0 0
da7 ONLINE 0 0 0
da8 ONLINE 0 0 0
da9 ONLINE 0 0 0
da10 ONLINE 0 0 0
da11 ONLINE 0 0 0
logs
ada1p2 ONLINE 0 0 0
errors: No known data errors
In the end, the price really is not that much different for the hardware.
--
-------------------
Mike Tancsa, tel +1 519 651 3400
Sentex Communications, mike@sentex.net
Providing Internet services since 1994 www.sentex.net
Cambridge, Ontario Canada http://www.tancsa.com/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?546B6A54.1060403>