From owner-freebsd-questions@FreeBSD.ORG Thu Sep 27 15:07:37 2012 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BE69E1065673 for ; Thu, 27 Sep 2012 15:07:37 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id 7ED878FC1E for ; Thu, 27 Sep 2012 15:07:37 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id q8RF7aI5032817 for ; Thu, 27 Sep 2012 11:07:36 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <50646BAE.20102@sentex.net> Date: Thu, 27 Sep 2012 11:07:26 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: freebsd-questions References: <5050E2EC.8070309@sentex.net> In-Reply-To: <5050E2EC.8070309@sentex.net> X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 Subject: Re: LSI 9750-4i (tws based cards) X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Sep 2012 15:07:37 -0000 On 9/12/2012 3:30 PM, Mike Tancsa wrote: > Does anyone have any experience with these cards ? We are looking for a > controller that has a little more gas than the twa based cards which > have been very reliable and stable for us on FreeBSD. I dont have any > experience with 3ware/LSI's cards that use the tws driver. Has anyone > used them yet ? For the archives... I ordered a 3ware 9750 4i card to test with and its quite fast! There is a small bug in the driver fixed now in HEAD as well as some cosmetic changes. But other than that it seems pretty solid. The same management interface as the twa and twe based cards. I ran a test box using a kernel with INVARIANTS and WITNESS with the card and 4 10k disks in raid 10. The card seems pretty zippy for the price. RW performance does seem to take advantage of the faster disk speeds 0{3w9750}# dd if=/dev/zero of=/mnt/test bs=1024k count=9000 9000+0 records in 9000+0 records out 9437184000 bytes transferred in 39.859600 secs (236760629 bytes/sec) 0{3w9750}# 0{3w9750}# umount /mnt 0{3w9750}# mount /dev/da0 /mnt 0{3w9750}# dd if=/mnt/test of=/dev/null bs=1024k 9000+0 records in 9000+0 records out 9437184000 bytes transferred in 27.887930 secs (338396720 bytes/sec) 0{3w9750}# For stress testing, I ran the disk.cfg component of http://people.freebsd.org/~pho/stress/index.html as well as random copies of dbench and bonnie as well as periodically accessing the disk while the stress scripts ran for 72hrs. The OS was netbooted, RELENG9 AMD64 0{3w9750}# tw_cli "/c0 show" Unit UnitType Status %RCmpl %V/I/M Stripe Size(GB) Cache AVrfy ------------------------------------------------------------------------------ u0 RAID-10 OK - - 256K 931.303 RiW ON VPort Status Unit Size Type Phy Encl-Slot Model ------------------------------------------------------------------------------ p0 OK u0 465.76 GB SATA 0 - WDC WD5002AALX-00J3 p1 OK u0 465.76 GB SATA 1 - WDC WD5002AALX-00J3 p2 OK u0 465.76 GB SATA 2 - WDC WD5002AALX-00J3 p3 OK u0 465.76 GB SATA 3 - WDC WD5002AALX-00J3 0{3w9750}# For some reason the card defaults legacy interrupts. Adding hw.tws.enable_msi=1 to /boot/loader.conf fixes that LSI 3ware device driver for SAS/SATA storage controllers, version: 10.80.00.003 tws0: port 0x4000-0x40ff mem 0xc2460000-0xc2463fff,0xc2400000-0xc243ffff irq 17 at device 0.0 on pci2 tws0: Using MSI tws0: Controller details: Model 9750-4i, 8 Phys, Firmware FH9X 5.12.00.007, BIOS BE9X 5.11.00.006 (probe65:tws0:0:65:0): INQUIRY. CDB: 12 0 0 0 24 0 (probe65:tws0:0:65:0): CAM status: Invalid Target ID (probe65:tws0:0:65:0): Error 22, Unretryable error da0 at tws0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 6000.000MB/s transfers da0: 953654MB (1953083392 512 byte sectors: 255H 63S/T 121573C) tws0@pci0:2:0:0: class=0x010400 card=0x000113c1 chip=0x101013c1 rev=0x05 hdr=0x00 vendor = '3ware Inc' device = '9750 SAS2/SATA-II RAID PCIe' class = mass storage subclass = RAID bar [10] = type I/O Port, range 32, base 0x4000, size 256, enabled bar [14] = type Memory, range 64, base 0xc2460000, size 16384, enabled bar [1c] = type Memory, range 64, base 0xc2400000, size 262144, enabled cap 01[50] = powerspec 3 supports D0 D1 D2 D3 current D0 cap 10[68] = PCI-Express 2 endpoint max data 128(4096) link x4(x8) cap 03[d0] = VPD cap 05[a8] = MSI supports 1 message, 64 bit enabled with 1 message ecap 0001[100] = AER 1 1 fatal 0 non-fatal 0 corrected ecap 0004[138] = unknown 1 In summary, we like the card on FreeBSD. We make heavy use of the older 3ware cards in our company on various platforms, so our staff are comfortable using the management tools to swap out dead drives. We will probably start to use these cards for customer builds in the future where they need faster IO. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/