From owner-freebsd-stable@FreeBSD.ORG Wed Feb 10 19:39:53 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E45AA106566C for ; Wed, 10 Feb 2010 19:39:52 +0000 (UTC) (envelope-from korvus@comcast.net) Received: from qmta14.westchester.pa.mail.comcast.net (qmta14.westchester.pa.mail.comcast.net [76.96.59.212]) by mx1.freebsd.org (Postfix) with ESMTP id 8EFD88FC08 for ; Wed, 10 Feb 2010 19:39:52 +0000 (UTC) Received: from omta03.westchester.pa.mail.comcast.net ([76.96.62.27]) by qmta14.westchester.pa.mail.comcast.net with comcast id gFpU1d00K0bG4ec5EKfsxc; Wed, 10 Feb 2010 19:39:52 +0000 Received: from [10.0.0.51] ([71.199.122.142]) by omta03.westchester.pa.mail.comcast.net with comcast id gKfr1d00l34Sj4f3PKfsNG; Wed, 10 Feb 2010 19:39:52 +0000 Message-ID: <4B730B94.1050205@comcast.net> Date: Wed, 10 Feb 2010 14:40:04 -0500 From: Steve Polyack User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.7) Gecko/20100111 Lightning/1.0b1 Thunderbird/3.0.1 MIME-Version: 1.0 To: Dan Langille References: <4B6F9A8D.4050907@langille.org> <4B71490B.6030602@langille.org> <4B71AED5.4030002@wensing.org> <201002091949.o19JntPo009017@apollo.backplane.com> <4B723DF9.3070105@langille.org> In-Reply-To: <4B723DF9.3070105@langille.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: FreeBSD Stable Subject: Re: hardware for home use large storage X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 10 Feb 2010 19:39:53 -0000 On 2/10/2010 12:02 AM, Dan Langille wrote: > Trying to make sense of stuff I don't know about... > > Matthew Dillon wrote: >> >> AHCI on-motherboard with equivalent capabilities do not appear to be >> in wide distribution yet. Most AHCI chips can do NCQ to a single >> target (even a single target behind a PM), but not concurrently to >> multiple targets behind a port multiplier. Even though SATA >> bandwidth >> constraints might seem to make this a reasonable alternative it >> actually isn't because any seek heavy activity to multiple drives >> will be serialized and perform EXTREMELY poorly. Linear performance >> will be fine. Random performance will be horrible. > > Don't use a port multiplier and this goes away. I was hoping to avoid > a PM and using something like the Syba PCI Express SATA II 4 x Ports > RAID Controller seems to be the best solution so far. > > http://www.amazon.com/Syba-Express-Ports-Controller-SY-PEX40008/dp/B002R0DZWQ/ref=sr_1_22?ie=UTF8&s=electronics&qid=1258452902&sr=1-22 > Dan, I can personally vouch for these cards under FreeBSD. We have 3 of them in one system, with almost every port connected to a port multiplier (SiI5xxx PMs). Using the siis(4) driver on 8.0-RELEASE provides very good performance, and supports both NCQ and FIS-based switching (an essential for decent port-multiplier performance). One thing to consider, however, is that the card is only single-lane PCI-Express. The bandwidth available is only 2.5Gb/s (~312MB/sec, slightly less than that of the SATA-2 link spec), so if you have 4 high-performance drives connected, you may hit a bottleneck at the bus. I'd be particularly interested if anyone can find any similar Silicon Image SATA controllers with a PCI-E 4x or 8x interface ;) > >> >> It should be noted that while hotswap is supported with silicon >> image >> chipsets and port multiplier enclosures (which also use Sili >> chips in >> the enclosure), the hot-swap capability is not anywhere near as >> robust >> as you would find with a more costly commercial SAS setup. SI chips >> are very poorly made (this is the same company that went bust under >> another name a few years back due to shoddy chipsets), and have a >> lot >> of on-chip hardware bugs, but fortunately OSS driver writers (linux >> guys) have been able to work around most of them. So even though >> the >> chipset is a bit shoddy actual operation is quite good. However, >> this does mean you generally want to idle all activity on the >> enclosure >> to safely hot swap anything, not just the drive you are pulling out. >> I've done a lot of testing and hot-swapping an idle disk while other >> drives in the same enclosure are hot is not reliable (for a cheap >> port >> multiplier enclosure using a Sili chip inside, which nearly all do). > > I haven't had such bad experience as the above, but it is certainly a concern. Using ZFS we simply 'offline' the device, pull, replace with a new one, glabel, and zfs replace. It seems to work fine as long as nothing is accessing the device you are replacing (otherwise you will get a kernel panic a few minutes down the line). mav@FreeBSD.org has also committed a large patch set to 9-CURRENT which implements "proper" SATA/AHCI hot-plug support and error-recovery through CAM. -Steve Polyack