From owner-freebsd-fs@freebsd.org Tue Sep 29 18:29:21 2015 Return-Path: Delivered-To: freebsd-fs@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62C69A0C2EA for ; Tue, 29 Sep 2015 18:29:21 +0000 (UTC) (envelope-from jg@internetx.com) Received: from mx1.internetx.com (mx1.internetx.com [62.116.129.39]) by mx1.freebsd.org (Postfix) with ESMTP id E24331FA7 for ; Tue, 29 Sep 2015 18:29:20 +0000 (UTC) (envelope-from jg@internetx.com) Received: from localhost (localhost [127.0.0.1]) by mx1.internetx.com (Postfix) with ESMTP id C82361472007 for ; Tue, 29 Sep 2015 20:29:18 +0200 (CEST) X-Virus-Scanned: InterNetX GmbH amavisd-new at ix-mailer.internetx.de Received: from mx1.internetx.com ([62.116.129.39]) by localhost (ix-mailer.internetx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 8BQW6m3rm9FO for ; Tue, 29 Sep 2015 20:29:13 +0200 (CEST) Received: from [192.168.100.26] (pizza.internetx.de [62.116.129.3]) (using TLSv1 with cipher DHE-RSA-AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.internetx.com (Postfix) with ESMTPSA id 523FD1472006 for ; Tue, 29 Sep 2015 20:29:13 +0200 (CEST) Reply-To: jg@internetx.com Subject: Re: Cannot replace broken hard drive with LSI HBA References: <1443447383.5271.66.camel@data-b104.adm.slu.se> <5609578E.1050606@physics.umn.edu> <560A4640.3030200@internetx.com> <560A9461.8090300@physics.umn.edu> <560A977C.1070102@internetx.com> <560AD2B9.5040706@fuckner.net> To: "freebsd-fs@freebsd.org" From: InterNetX - Juergen Gotteswinter Message-ID: <560AD879.2010004@internetx.com> Date: Tue, 29 Sep 2015 20:29:13 +0200 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Sep 2015 18:29:21 -0000 Am 29.09.2015 um 20:25 schrieb Freddie Cash: > On Tue, Sep 29, 2015 at 11:04 AM, Michael Fuckner >wrote: > > On 9/29/2015 3:51 PM, InterNetX - Juergen Gotteswinter wrote: > > From my Experience using SATA Disks on SAS Controllers, no > matter if > theres an Expander between or not or mixed, those Setups keep on > beeing > flakey / unreliable. I might work under certain conditions, but its > nothing you can bet on. > > Garret Damore (Illumos Project) describes the problem more > detailed here > > http://garrett.damore.org/2010/08/why-sas-sata-is-not-such-great-idea.html > > > come on, the article is 5 years old, some things changed since then! > > - MUX Boards are unreliable and expensive- long time since I last > saw those boards > - SAS Disks are not just 10/15k high performance Disks anymore, most > Nearline Disks are available with native SAS interface as well > - if you pick the right disk there is no trouble using SATA Disks on > SAS Expanders or SAS Controllers (they should have R/V sensors, > optimized FW...). > - if you use desktop drives in a shelf with lets say 24 slots you > should not expect it to work ;-) > > > ​Why not? ;) > > We use desktop-class drives in our backups storage servers without any > issues. Even the monster boxes with 90 drives in them (2 JBODs of 45 > drives each) run without issues using desktop-class drives. > > We're using a mix of WD Black (1, 2, 4 TB), Toshiba (2 TB), and Seagate > (1, 2 TB). > > 2 systems using 24 drive bays. 2 systems using 90 drive bays. Plugged > into SuperMicro SAS expanders and LSI 9211-8i or 9211-8e (I think that's > the model number) controllers.​ All SAS2008 chipsets using mps(4) drivers. > > We're not looking for uber-performance and millions of IOps from these > systems, as the gigabit NIC is the bottleneck (rsync and zfs send both > saturate that link, but all operations still complete within the > allotted 8 hours window). > > We replace maybe 6-8 drives per year across all 4 systems; a little more > than that this year due to overheating in one location, but that's been > fixed. > > When a 2 TB desktop-class harddrive is $ 80 CDN in bulk, and we're only > replacing 8 drives per year (under warranty, of course), it just doesn't > make sense to spend the extra money on server-class, RAID-aware, > nearline, or SAS drives. :) > > ​If you ​are building a storage server that requires millions of IOps > with multiple 10 Gbps connections, then sure, desktop-class drives won't > cut it. But for everything else, they're fine. > > -- > Freddie Cash > fjwcash@gmail.com hello backplaze? :) sounds legit to me, since you dont seem to mix sata/sas