From owner-freebsd-scsi@FreeBSD.ORG Wed May 21 20:08:27 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14798106567C for ; Wed, 21 May 2008 20:08:25 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 9F0FD8FC19 for ; Wed, 21 May 2008 20:08:24 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local ([192.168.254.200]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id m4LK8HtU045239; Wed, 21 May 2008 14:08:17 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <48348131.3040602@samsco.org> Date: Wed, 21 May 2008 14:08:17 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Chris Dillon References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> <48336EA0.3050109@samsco.org> <20080521143051.17771kseoxrlhy7f@www.wolves.k12.mo.us> In-Reply-To: <20080521143051.17771kseoxrlhy7f@www.wolves.k12.mo.us> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.3 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 20:08:27 -0000 Chris Dillon wrote: > Quoting Scott Long : > >> For data reliability, you really don't want it enabled by default. The >> problem is that SATA/ATA performs so poorly without it that everyone >> turns it on and lives with the consequences. The tweak that I >> recommended puts it in line with what the FreeBSD ATA driver has been >> doing for years. > > Doesn't SATA NCQ solve this particular performance vs. reliability > problem since it safely allows multiple outstanding write requests? Of > course that means the SATA RAID controller would have to use NCQ on the > drives and would probably also need its own non-volatile cache. I've > always assumed this is how SCSI/SAS drives (with TCQ) perform as well as > they do without sacrificing data integrity. Yes and no. NCQ gets you 90% the way there, but the lack of an ordered tag operation in the NCQ protocol means that i/o streams can be starved, forcing you to do unpleasant i/o scheduling hacks. But yes, it helps quite a bit, and I have a prototype driver already working that supports NCQ and performs very well with write cache turned off. > > We recently bought a new HP DL380G5 server with a P800 SAS RAID > controller, MSA60 external drive shelf with 12 750GB SATA drives, > 11-drive RAID5 array w/ hot-spare (a few too many drives in a single > RAID5 array, I know, but I'm experimenting). The system is running > Windows Server 2K3 R2. Without telling the P800 to enable the SATA WC > (it has an option to do so, off by default), when doing a drag and drop > file copy of several very large files from the internal SAS array to the > external SATA array it writes 300MB/sec. I briefly enabled the > "Physical Drive Write Cache" on the controller just a few minutes ago > and ran another test and didn't notice any difference in write speed. I > can only assume from this that the P800 is using NCQ on the SATA drives. > The cache and queueing mechanism on most IOP raid cards will smooth over the performance problems with ATA/SATA, so your results aren't too surprising. Scott