From owner-freebsd-fs@freebsd.org Tue Sep 29 19:01:01 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1E843A0AE23 for ; Tue, 29 Sep 2015 19:01:01 +0000 (UTC) (envelope-from rcv-cmlja0BoYXZva21vbi5jb20AQDE0NDM1NTI4NTg=@vfemail.net) Received: from smtp101-5.vfemail.net (eightfive.vfemail.net [96.30.253.185]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id E2E2A100D for ; Tue, 29 Sep 2015 19:01:00 +0000 (UTC) (envelope-from rcv-cmlja0BoYXZva21vbi5jb20AQDE0NDM1NTI4NTg=@vfemail.net) Received: (qmail 8066 invoked by uid 89); 29 Sep 2015 18:54:18 -0000 Received: by simscan 1.4.0 ppid: 8058, pid: 8061, t: 0.0813s scanners:none Received: from unknown (HELO d3d3MTEwQDE0NDM1NTI4NTg=) (cmlja0BoYXZva21vbi5jb21AMTQ0MzU1Mjg1OA==@MTcyLjE2LjEwMC45MkAxNDQzNTUyODU4) by 172.16.100.61 with ESMTPA; 29 Sep 2015 18:54:18 -0000 Date: Tue, 29 Sep 2015 13:54:17 -0500 Message-ID: <20150929135417.Horde.W2X_CSKI8gBxk1Df7JVOPQ1@www.vfemail.net> From: Rick Romero To: freebsd-fs@freebsd.org Subject: Re: Cannot replace broken hard drive with LSI HBA References: <1443447383.5271.66.camel@data-b104.adm.slu.se> <5609578E.1050606@physics.umn.edu> <560A4640.3030200@internetx.com> <560A9461.8090300@physics.umn.edu> <560A977C.1070102@internetx.com> <560AD2B9.5040706@fuckner.net> <560AD6C7.80800@internetx.com> In-Reply-To: <560AD6C7.80800@internetx.com> User-Agent: Internet Messaging Program (IMP) H5 (6.2.2) X-VFEmail-Originating-IP: MTIuMzEuMTAwLjE0Ng== X-VFEmail-AntiSpam: Notify admin@vfemail.net of any spam, and include VFEmail headers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8; format=flowed; DelSp=Yes Content-Transfer-Encoding: 8bit Content-Disposition: inline Content-Description: Plaintext Message X-Content-Filtered-By: Mailman/MimeDel 2.1.20 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2015 19:01:01 -0000 Quoting InterNetX - Juergen Gotteswinter : > Am 29.09.2015 um 20:04 schrieb Michael Fuckner: >> On 9/29/2015 3:51 PM, InterNetX - Juergen Gotteswinter wrote: >>> From my Experience using SATA Disks on SAS Controllers, no matter if >>> theres an Expander between or not or mixed, those Setups keep on beeing >>> flakey / unreliable. I might work under certain conditions, but its >>> nothing you can bet on. >>> >>> Garret Damore (Illumos Project) describes the problem more detailed here >>> >>> http://garrett.damore.org/2010/08/why-sas-sata-is-not-such-great-idea.html >>>   >> >> come on, the article is 5 years old, some things changed since then! >> >> - MUX Boards are unreliable and expensive- long time since I last saw >> those boards > > i am not talking about sata multiplexers, i am talking about sas > expanders. no one ever should think that sata multiplexers are a > reliable piece of hardware. they arent. > > this problems are design related, since Natacount on disks cant talk > native with sas controllers. they still exist and occour. like you can > see here. > > a few months ago i had a setup which had exactly what one whould expect > from sata/sas mix. bus resets, device hangs and so on. in the end, it > turned out that the indel dcs 3500 ssd wich where connected via a lsi > sas 2008 where the root cause of all. after they where hooked onto the > onboard sata ports, everything was fine. on the other side the intel dcs > 3700 sata ssd works fine in sas/sata mix. you see, its nothing you can > count on. stay with sas, or with sata and use proper controllers. I've seen both.  I have two nice Asus boxes with 12 4GB SATA drives in RAIDZ2, and they work wonderfully. OTOH, I recently built a new system with a 3 drive RAIDZ (using SSD), and it constantly blew chunks (timeouts etc). After going through multiple servers, drives,backplanes (both expanders and just generic hot-swap) and controllers, it turns out the latest LSI firmware is garbage.  You have to downgrade the LSI firmware to 'p17' (IIRC) - and preferrably if you're using ZFS, change it to IT firmware (for 'real' JBOD). http://lime-technology.com/forum/index.php?topic=12767.630 https://forums.freenas.org/index.php?threads/confused-about-that-lsi-card-join-the-crowd.11901/page-5 Rick