From owner-freebsd-hardware@FreeBSD.ORG Mon May 27 11:06:46 2013 Return-Path: Delivered-To: freebsd-hardware@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D4ACF31D for ; Mon, 27 May 2013 11:06:46 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) by mx1.freebsd.org (Postfix) with ESMTP id AB01A6BE for ; Mon, 27 May 2013 11:06:46 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r4RB6kn7016020 for ; Mon, 27 May 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r4RB6koZ016018 for freebsd-hardware@FreeBSD.org; Mon, 27 May 2013 11:06:46 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 27 May 2013 11:06:46 GMT Message-Id: <201305271106.r4RB6koZ016018@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-hardware@FreeBSD.org Subject: Current problem reports assigned to freebsd-hardware@FreeBSD.org X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 May 2013 11:06:46 -0000 Note: to view an individual PR, use: http://www.freebsd.org/cgi/query-pr.cgi?pr=(number). The following is a listing of current problems submitted by FreeBSD users. These represent problem reports covering all versions including experimental development code and obsolete releases. S Tracker Resp. Description -------------------------------------------------------------------------------- f kern/156241 hardware [mfi] 'zfs send' does not prevents disks to suspend if 1 problem total. From owner-freebsd-hardware@FreeBSD.ORG Tue May 28 13:38:24 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 3629C72B; Tue, 28 May 2013 13:38:24 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id F3D46B54; Tue, 28 May 2013 13:38:23 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id r4SDcN0F049310; Tue, 28 May 2013 09:38:23 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <51A4B34F.7000505@sentex.net> Date: Tue, 28 May 2013 09:38:23 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: "freebsd-hardware@freebsd.org" Subject: Rocket Raid 622 in AHCI mode X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 13:38:24 -0000 I picked up an external PMP/Cage/RR6622 combo that claims to be Sata III. From the BIOS, the card implies it does indeed see the SATA III drives Pic of the BIOS here http://tancsa.com/rr.jpg ahci0: port 0x4090-0x4097,0x4080-0x4083,0x4070-0x4077,0x4060-0x4063,0x4050-0x405f mem 0xc2430000-0xc24307ff irq 16 at device 0.0 on pci1 ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS ahcich0: at channel 0 on ahci0 ahcich1: at channel 1 on ahci0 However, performance is abysmal and worse than using a regular sii based SATA II card. It sees the PMP and cage as pmp0 at ahcich1 bus 0 scbus1 target 15 lun 0 pmp0: ATA-0 device pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes) pmp0: 5 fan-out ports ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 ada1: ATA-8 SATA 3.x device ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada1: Command Queueing enabled ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada1: Previously was known as ad6 ada2 at ahcich1 bus 0 scbus1 target 1 lun 0 ada2: ATA-8 SATA 3.x device ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada2: Command Queueing enabled ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) ada2: Previously was known as ad7 ada3 at ahcich1 bus 0 scbus1 target 2 lun 0 ada3: ATA-8 SATA 3.x device ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada3: Command Queueing enabled ada3: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C) ada4 at ahcich1 bus 0 scbus1 target 3 lun 0 ada4: ATA-8 SATA 3.x device ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada4: Command Queueing enabled ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) ada5 at ahcich1 bus 0 scbus1 target 4 lun 0 ada5: ATA-8 SATA 3.x device ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) ada5: Command Queueing enabled ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) # camcontrol devlist at scbus1 target 0 lun 0 (ada1,pass2) at scbus1 target 1 lun 0 (ada2,pass3) at scbus1 target 2 lun 0 (ada3,pass4) at scbus1 target 3 lun 0 (ada4,pass5) at scbus1 target 4 lun 0 (ada5,pass6) at scbus1 target 15 lun 0 (pmp0,pass1) at scbus2 target 0 lun 0 (pass0,ada0) Writing to the individual disks seems fine, but when all the disks are engaged, its very slow # zpool create -f tank raidz ada1 ada2 ada3 ada4 ada5 # dd if=/dev/zero of=/tank/test1 bs=2048k count=10 10+0 records in 10+0 records out 20971520 bytes transferred in 60.370025 secs (347383 bytes/sec) # zpool export tank; zpool import tank # dd if=/tank/test1 of=/dev/null bs=2048k 10+0 records in 10+0 records out 20971520 bytes transferred in 7.678915 secs (2731052 bytes/sec) # for i in `jot 5 1`; do dd if=/dev/ada$i of=/dev/null count=200 bs=2048k;done 200+0 records in 200+0 records out 419430400 bytes transferred in 3.211699 secs (130594554 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 3.034111 secs (138238327 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 3.639701 secs (115237601 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 2.841386 secs (147614716 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 3.020611 secs (138856154 bytes/sec) # # for i in `jot 5 1`; do dd if=/dev/zero of=/dev/ada$i count=200 bs=2048k;done 200+0 records in 200+0 records out 419430400 bytes transferred in 3.285333 secs (127667539 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 3.043410 secs (137815945 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 3.611478 secs (116138154 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 2.830386 secs (148188414 bytes/sec) 200+0 records in 200+0 records out 419430400 bytes transferred in 2.987366 secs (140401412 bytes/sec) # Any ideas on how to fix this controller to make it work better with ZFS and get better speeds ? ahci0@pci0:1:0:0: class=0x010400 card=0x00011103 chip=0x06221103 rev=0x01 hdr=0x00 vendor = 'HighPoint Technologies, Inc.' class = mass storage subclass = RAID bar [10] = type I/O Port, range 32, base 0x4090, size 8, enabled bar [14] = type I/O Port, range 32, base 0x4080, size 4, enabled bar [18] = type I/O Port, range 32, base 0x4070, size 8, enabled bar [1c] = type I/O Port, range 32, base 0x4060, size 4, enabled bar [20] = type I/O Port, range 32, base 0x4050, size 16, enabled bar [24] = type Memory, range 32, base 0xc2430000, size 2048, enabled cap 01[40] = powerspec 3 supports D0 D3 current D0 cap 05[50] = MSI supports 1 message enabled with 1 message cap 10[70] = PCI-Express 2 legacy endpoint max data 128(512) link x1(x1) speed 5.0(5.0) ecap 0001[100] = AER 1 1 fatal 0 non-fatal 0 corrected ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-hardware@FreeBSD.ORG Tue May 28 16:33:11 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id DD09C40F for ; Tue, 28 May 2013 16:33:11 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ee0-f41.google.com (mail-ee0-f41.google.com [74.125.83.41]) by mx1.freebsd.org (Postfix) with ESMTP id 75136804 for ; Tue, 28 May 2013 16:33:11 +0000 (UTC) Received: by mail-ee0-f41.google.com with SMTP id d4so4947840eek.28 for ; Tue, 28 May 2013 09:33:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=cga4O5N8XEPZ2gLnqwwlr+FxsGEGwtIdP2m83keMtcM=; b=Hjwa83IbpIdKXGvNkFOnWBnDe+L/ScKQhqIEIDJEP1NiMaE/HCMoToe7O2No7ZYsNL j/rgI9zK5GxnqvblxdnF1tyJ6hUkFXBcH8o4uYR1pkWT1ezC4yuvgybSMNr+w8OXHrEK irR82J1HGZdAoZJ9F1gZyEbz50aSMEZJWzEik5sQLrHBCdomMMNbSbvwH/+Z0fUHz2Mz 7DFHBzpQcSafqjr4Do5603tws7YYTEAXLcGAkNG8bG7OXp8kjU6kqSd7hdKl8DbnCaB9 vEPrrEwkv5cIVhBVAN8QSamlNJ4SvKhrEVL7GVy0uOgqXKDWzhoR1gYUaW3Zlu/tksBr PHIA== X-Received: by 10.14.10.9 with SMTP id 9mr14990114eeu.75.1369758790449; Tue, 28 May 2013 09:33:10 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id s43sm48891875eem.13.2013.05.28.09.33.07 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 28 May 2013 09:33:09 -0700 (PDT) Sender: Alexander Motin Message-ID: <51A4DC42.30705@FreeBSD.org> Date: Tue, 28 May 2013 19:33:06 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Mike Tancsa Subject: Re: Rocket Raid 622 in AHCI mode References: <51A4B34F.7000505@sentex.net> In-Reply-To: <51A4B34F.7000505@sentex.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 16:33:11 -0000 On 28.05.2013 16:38, Mike Tancsa wrote: > I picked up an external PMP/Cage/RR6622 combo that claims to be Sata > III. From the BIOS, the card implies it does indeed see the SATA III drives > > Pic of the BIOS here > http://tancsa.com/rr.jpg I am not sure what does that number on photo mean, but I haven't even heard about any SATA3.x port multipliers yet. SiI3826 (as I can identify it from below) is not a new one and AFAIK only SATA 2.x. > ahci0: port > 0x4090-0x4097,0x4080-0x4083,0x4070-0x4077,0x4060-0x4063,0x4050-0x405f > mem 0xc2430000-0xc24307ff irq 16 at device 0.0 on pci1 > ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with FBS > ahcich0: at channel 0 on ahci0 > ahcich1: at channel 1 on ahci0 Yes, "2 6Gbps ports, Port Multiplier supported with FBS" is what this line of Marvell chip formally reports and it sounds great. I've tested it quite a long ago, but IIRC while it was fast enough with direct disk connection, it was quite slow working with port multipliers. > However, performance is abysmal and worse than using a regular sii based > SATA II card. SiI3124 still was not beaten. > It sees the PMP and cage as > > pmp0 at ahcich1 bus 0 scbus1 target 15 lun 0 > pmp0: ATA-0 device > pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes) > pmp0: 5 fan-out ports > ada1 at ahcich1 bus 0 scbus1 target 0 lun 0 > ada1: ATA-8 SATA 3.x device > ada1: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada1: Command Queueing enabled > ada1: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) > ada1: Previously was known as ad6 > ada2 at ahcich1 bus 0 scbus1 target 1 lun 0 > ada2: ATA-8 SATA 3.x device > ada2: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada2: Command Queueing enabled > ada2: 953869MB (1953525168 512 byte sectors: 16H 63S/T 16383C) > ada2: Previously was known as ad7 > ada3 at ahcich1 bus 0 scbus1 target 2 lun 0 > ada3: ATA-8 SATA 3.x device > ada3: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada3: Command Queueing enabled > ada3: 1430799MB (2930277168 512 byte sectors: 16H 63S/T 16383C) > ada4 at ahcich1 bus 0 scbus1 target 3 lun 0 > ada4: ATA-8 SATA 3.x device > ada4: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada4: Command Queueing enabled > ada4: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) > ada5 at ahcich1 bus 0 scbus1 target 4 lun 0 > ada5: ATA-8 SATA 3.x device > ada5: 300.000MB/s transfers (SATA 2.x, UDMA6, PIO 8192bytes) > ada5: Command Queueing enabled > ada5: 1907729MB (3907029168 512 byte sectors: 16H 63S/T 16383C) > > # camcontrol devlist > at scbus1 target 0 lun 0 (ada1,pass2) > at scbus1 target 1 lun 0 (ada2,pass3) > at scbus1 target 2 lun 0 (ada3,pass4) > at scbus1 target 3 lun 0 (ada4,pass5) > at scbus1 target 4 lun 0 (ada5,pass6) > at scbus1 target 15 lun 0 (pmp0,pass1) > at scbus2 target 0 lun 0 (pass0,ada0) > > Writing to the individual disks seems fine, but when all the disks are > engaged, its very slow > > # zpool create -f tank raidz ada1 ada2 ada3 ada4 ada5 > > # dd if=/dev/zero of=/tank/test1 bs=2048k count=10 > 10+0 records in > 10+0 records out > 20971520 bytes transferred in 60.370025 secs (347383 bytes/sec) > # zpool export tank; zpool import tank > # dd if=/tank/test1 of=/dev/null bs=2048k > 10+0 records in > 10+0 records out > 20971520 bytes transferred in 7.678915 secs (2731052 bytes/sec) ZFS in RAIDZ does simultaneous I/O to all disks, that is probably the most complicated case. Though I can hardly believe it can be that slow. Even without any FBS support in HBA I would expected at least 100MB/s / 5 = 20MB/s, but definitely not 350KB/s. > # for i in `jot 5 1`; do dd if=/dev/ada$i of=/dev/null count=200 > bs=2048k;done > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.211699 secs (130594554 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.034111 secs (138238327 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.639701 secs (115237601 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 2.841386 secs (147614716 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.020611 secs (138856154 bytes/sec) > # > > # for i in `jot 5 1`; do dd if=/dev/zero of=/dev/ada$i count=200 > bs=2048k;done > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.285333 secs (127667539 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.043410 secs (137815945 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 3.611478 secs (116138154 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 2.830386 secs (148188414 bytes/sec) > 200+0 records in > 200+0 records out > 419430400 bytes transferred in 2.987366 secs (140401412 bytes/sec) > # > > Any ideas on how to fix this controller to make it work better with ZFS > and get better speeds ? Good question. One of several about these Marvell controllers, caused by lack of any public documentation. :( -- Alexander Motin From owner-freebsd-hardware@FreeBSD.ORG Tue May 28 17:46:38 2013 Return-Path: Delivered-To: freebsd-hardware@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 1ECE71E6; Tue, 28 May 2013 17:46:38 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id DFED9AEB; Tue, 28 May 2013 17:46:37 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id r4SHkbaS090637; Tue, 28 May 2013 13:46:37 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <51A4ED7D.3010600@sentex.net> Date: Tue, 28 May 2013 13:46:37 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Alexander Motin Subject: Re: Rocket Raid 622 in AHCI mode References: <51A4B34F.7000505@sentex.net> <51A4DC42.30705@FreeBSD.org> In-Reply-To: <51A4DC42.30705@FreeBSD.org> X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 Cc: "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 17:46:38 -0000 On 5/28/2013 12:33 PM, Alexander Motin wrote: >> # >> >> Any ideas on how to fix this controller to make it work better with ZFS >> and get better speeds ? > > Good question. One of several about these Marvell controllers, caused by > lack of any public documentation. :( > OK, thanks for responding. I figured if anyone knew definitively, it would be you :) I will just use the 3124 for now. It has been pretty reliably for me to make these large storage appliances. Other than the 3124, do you recommend anything else ? I want to put together another 20TB storage appliance. Faster is nicer, but cost is a constraining factor. ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-hardware@FreeBSD.ORG Tue May 28 18:03:03 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id A9ADE928 for ; Tue, 28 May 2013 18:03:03 +0000 (UTC) (envelope-from mavbsd@gmail.com) Received: from mail-ea0-x235.google.com (mail-ea0-x235.google.com [IPv6:2a00:1450:4013:c01::235]) by mx1.freebsd.org (Postfix) with ESMTP id 40A03BF5 for ; Tue, 28 May 2013 18:03:03 +0000 (UTC) Received: by mail-ea0-f181.google.com with SMTP id a11so4770724eae.40 for ; Tue, 28 May 2013 11:03:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:message-id:date:from:user-agent:mime-version:to:cc:subject :references:in-reply-to:content-type:content-transfer-encoding; bh=RPQJv5KYqFfxi4eFfaGcDv831ysdFU926Qvf+KIe7ng=; b=O5GMprMt2EmwQbRTaJwc1tXlQ4jqaovRkA0EE25zKVPG/y1ik+L8UfH3aimb8GRXOy APJbb4F8Qr3XvQDidUeem9BRRJh6pXboftF/zgo9PpcDdhXTtbO9pNtk8+fBTa8H/m4r 3Kr11NiJm/X+NUi+J0L42WX27lGZ7u/ftedeLBvewBzDtFBrG89ggkIaWwNIIlo03nqU efi7UZOmTE/X0cIaDcAZ0BiWJsjVxecs1dVqUKZwaKKjKjR0wxhbAXLMk1dW6TORNg1c Pyvm/79vduW3qjAmNN4n3GoyjDDuque0Mm7mNPOhy2Zqw0DOUjRDDyO70xPXtVgEGrnb Lo+A== X-Received: by 10.14.7.66 with SMTP id 42mr15157950eeo.130.1369764182455; Tue, 28 May 2013 11:03:02 -0700 (PDT) Received: from mavbook.mavhome.dp.ua (mavhome.mavhome.dp.ua. [213.227.240.37]) by mx.google.com with ESMTPSA id y2sm49381332eeu.2.2013.05.28.11.03.00 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Tue, 28 May 2013 11:03:01 -0700 (PDT) Sender: Alexander Motin Message-ID: <51A4F152.6030202@FreeBSD.org> Date: Tue, 28 May 2013 21:02:58 +0300 From: Alexander Motin User-Agent: Mozilla/5.0 (X11; FreeBSD amd64; rv:17.0) Gecko/20130413 Thunderbird/17.0.5 MIME-Version: 1.0 To: Mike Tancsa Subject: Re: Rocket Raid 622 in AHCI mode References: <51A4B34F.7000505@sentex.net> <51A4DC42.30705@FreeBSD.org> <51A4ED7D.3010600@sentex.net> In-Reply-To: <51A4ED7D.3010600@sentex.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 May 2013 18:03:03 -0000 On 28.05.2013 20:46, Mike Tancsa wrote: > On 5/28/2013 12:33 PM, Alexander Motin wrote: >>> # >>> >>> Any ideas on how to fix this controller to make it work better with ZFS >>> and get better speeds ? >> >> Good question. One of several about these Marvell controllers, caused by >> lack of any public documentation. :( >> > > OK, thanks for responding. I figured if anyone knew definitively, it > would be you :) I will just use the 3124 for now. It has been pretty > reliably for me to make these large storage appliances. > > Other than the 3124, do you recommend anything else ? I want to put > together another 20TB storage appliance. Faster is nicer, but cost is a > constraining factor. Aside from these Marvell's I don't know any other AHCI HBA with FBS support. And HBAs without FBS are by definition are not suitable for simultaneous access to devices on PMP. Aside from AHCI, older Marvell's supported by mvs(4) like 88SX7042 can do FBS, but IIRC they are also not very fast and I don't trues them much either. -- Alexander Motin From owner-freebsd-hardware@FreeBSD.ORG Thu May 30 00:29:16 2013 Return-Path: Delivered-To: freebsd-hardware@FreeBSD.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id 072FA6F9; Thu, 30 May 2013 00:29:16 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id BB328133; Thu, 30 May 2013 00:29:15 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id r4U0TFCR090636; Wed, 29 May 2013 20:29:15 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <51A69D7F.3030605@sentex.net> Date: Wed, 29 May 2013 20:29:51 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: Alexander Motin Subject: Re: Rocket Raid 622 in AHCI mode References: <51A4B34F.7000505@sentex.net> <51A4DC42.30705@FreeBSD.org> In-Reply-To: <51A4DC42.30705@FreeBSD.org> X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 Cc: "freebsd-hardware@freebsd.org" X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 May 2013 00:29:16 -0000 On 5/28/2013 12:33 PM, Alexander Motin wrote: > On 28.05.2013 16:38, Mike Tancsa wrote: > > SiI3124 still was not beaten. > >> It sees the PMP and cage as >> >> pmp0 at ahcich1 bus 0 scbus1 target 15 lun 0 >> pmp0: ATA-0 device >> pmp0: 300.000MB/s transfers (SATA 2.x, NONE, PIO 8192bytes) >> pmp0: 5 fan-out ports >> Any ideas on how to fix this controller to make it work better with ZFS >> and get better speeds ? > > Good question. One of several about these Marvell controllers, caused by > lack of any public documentation. :( I tried to do a little more testing to see what might be up either with the cage, the card or the drives. It seems there is something odd about the first slot in the cage-- I have tried 2 cages, and they both show the same behaviour. ie. two cages (same make/model) and two of the rocket raid cards (same make and model). If I read/write to just the disk in slot zero, it works as expected. However, if I create some sort of multi-disk array with the disk in the first slot, its so slow, its almost broken. e.g. 0{mdttestbox}# gstripe stop data 0{mdttestbox}# gstripe label -v -s 131072 data ada0 ada1 Metadata value stored on ada0. Metadata value stored on ada1. Done. 0{mdttestbox}# newfs -U -O2 /dev/stripe/data > /dev/null 0{mdttestbox}# mount /dev/stripe/data /mnt 0{mdttestbox}# dd if=/dev/zero of=/mnt/test bs=4096k count=100 100+0 records in 100+0 records out 419430400 bytes transferred in 56.875768 secs (7374501 bytes/sec) 0{mdttestbox}# umount /mnt;mount /dev/stripe/data /mnt 0{mdttestbox}# dd if=/mnt/test of=/dev/null bs=4096k ^C17+0 records in 17+0 records out 71303168 bytes transferred in 237.424329 secs (300320 bytes/sec) 1{mdttestbox}# gstripe stop data gstripe: Cannot destroy device data (error=16). 1{mdttestbox}# umount /mnt ; mount /dev/stripe/data /mnt 0{mdttestbox}# umount /mnt Yet, if I use the other 4 disks, all works well 0{mdttestbox}# gstripe stop data 0{mdttestbox}# gstripe label -v -s 131072 data ada1 ada2 ada3 ada4 Metadata value stored on ada1. warning: ada2: only 1000204885504 bytes from 2000398933504 bytes used. Metadata value stored on ada2. warning: ada3: only 1000204885504 bytes from 1500301909504 bytes used. Metadata value stored on ada3. warning: ada4: only 1000204885504 bytes from 2000398933504 bytes used. Metadata value stored on ada4. Done. 0{mdttestbox}# newfs -U -O2 /dev/stripe/data > /dev/null 0{mdttestbox}# mount /dev/stripe/data /mnt 0{mdttestbox}# dd if=/dev/zero of=/mnt/test bs=4096k count=100 100+0 records in 100+0 records out 419430400 bytes transferred in 2.131533 secs (196774067 bytes/sec) 0{mdttestbox}# umount /mnt ; mount /dev/stripe/data /mnt 0{mdttestbox}# dd if=/mnt/test of=/dev/null bs=4096k 100+0 records in 100+0 records out 419430400 bytes transferred in 2.574856 secs (162894699 bytes/sec) 0{mdttestbox}# Although its a bit odd writes are faster than reads ? ZFS works as expected as well if I exclude the first slot. 1{mdttestbox}# zpool create stripe ada1 ada2 ada3 ada4 0{mdttestbox}# dd if=/dev/zero of=/stripe/test bs=4096k count=100 100+0 records in 100+0 records out 419430400 bytes transferred in 1.283134 secs (326879599 bytes/sec) 0{mdttestbox}# dd if=/dev/zero of=/stripe/test bs=4096k count=1000 1000+0 records in 1000+0 records out 4194304000 bytes transferred in 22.798638 secs (183971691 bytes/sec) 0{mdttestbox}# dd if=/stripe/test of=/dev/null bs=4096k 1000+0 records in 1000+0 records out 4194304000 bytes transferred in 0.472554 secs (8875820076 bytes/sec) 0{mdttestbox}# zpool export stripe 0{mdttestbox}# zpool import stripe 0{mdttestbox}# dd if=/stripe/test of=/dev/null bs=4096k 1000+0 records in 1000+0 records out 4194304000 bytes transferred in 23.068320 secs (181820956 bytes/sec) 0{mdttestbox}# Whats odd is that the SII card does not have this problem with 5 disks in the drive cage. Only the RR card does. 0{mdttestbox}# camcontrol devlist at scbus0 target 0 lun 0 (ada0,pass1) at scbus0 target 1 lun 0 (ada1,pass2) at scbus0 target 2 lun 0 (ada3,pass4) at scbus0 target 3 lun 0 (ada4,pass5) at scbus0 target 4 lun 0 (ada2,pass3) at scbus0 target 15 lun 0 (pmp0,pass0) at scbus2 target 0 lun 0 (pass6,ada5) 0{mdttestbox}# ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-hardware@FreeBSD.ORG Thu May 30 19:02:25 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id B25B3F8D for ; Thu, 30 May 2013 19:02:25 +0000 (UTC) (envelope-from lkchen@k-state.edu) Received: from ksu-out.merit.edu (ksu-out.merit.edu [207.75.117.132]) by mx1.freebsd.org (Postfix) with ESMTP id 7FC87E0A for ; Thu, 30 May 2013 19:02:25 +0000 (UTC) X-Merit-ExtLoop1: 1 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AikFAGmhp1HPS3TT/2dsb2JhbABWAxaCczBDgni/WRZ0giMBAQUjVgwCDRoCDRkCHTwGE4gNBwWpNoljiDUEgSKMNIFagjKBFAOYZ4p3hSCDK4FVNg X-IronPort-AV: E=Sophos;i="4.87,772,1363147200"; d="scan'208";a="229491002" X-MERIT-SOURCE: KSU Received: from ksu-sfpop-mailstore02.merit.edu ([207.75.116.211]) by sfpop-ironport02.merit.edu with ESMTP; 30 May 2013 15:02:24 -0400 Date: Thu, 30 May 2013 15:02:24 -0400 (EDT) From: "Lawrence K. Chen, P.Eng." To: Mike Tancsa Message-ID: <1489716432.21276182.1369940544405.JavaMail.root@k-state.edu> In-Reply-To: <51A4B34F.7000505@sentex.net> Subject: Re: Rocket Raid 622 in AHCI mode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [70.179.144.108] X-Mailer: Zimbra 7.2.2_GA_2852 (ZimbraWebClient - GC27 ([unknown])/7.2.2_GA_2852) Cc: freebsd-hardware@freebsd.org X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 May 2013 19:02:25 -0000 That matches my experience with rr622 as well. I replaced it with a Sil3132 card I had around, I also have a Sil3124 card in my system now. I've been thinking of replacing the cards with ASM1061 based ones....but things have been stable, so not sure I really want to go poking around inside my computer just to try different cards. Though I have plans to drop in another SSD to use as L2ARC in the future (had picked up a Velocity Solo x1 card a while back.... ) ----- Original Message ----- > I picked up an external PMP/Cage/RR6622 combo that claims to be Sata > III. From the BIOS, the card implies it does indeed see the SATA III > drives > > Pic of the BIOS here > http://tancsa.com/rr.jpg > > ahci0: port > 0x4090-0x4097,0x4080-0x4083,0x4070-0x4077,0x4060-0x4063,0x4050-0x405f > mem 0xc2430000-0xc24307ff irq 16 at device 0.0 on pci1 > ahci0: AHCI v1.00 with 2 6Gbps ports, Port Multiplier supported with > FBS > ahcich0: at channel 0 on ahci0 > ahcich1: at channel 1 on ahci0 > > > > However, performance is abysmal and worse than using a regular sii > based > SATA II card. > -- Who: Lawrence K. Chen, P.Eng. - W0LKC - Senior Unix Systems Administrator For: Enterprise Server Technologies (EST) -- & SafeZone Ally Snail: Computing and Telecommunications Services (CTS) Kansas State University, 109 East Stadium, Manhattan, KS 66506-3102 Phone: (785) 532-4916 - Fax: (785) 532-3515 - Email: lkchen@ksu.edu Web: http://www-personal.ksu.edu/~lkchen - Where: 11 Hale Library From owner-freebsd-hardware@FreeBSD.ORG Thu May 30 19:06:56 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id E41DEEE for ; Thu, 30 May 2013 19:06:56 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1-6.sentex.ca [IPv6:2607:f3e0:0:1::12]) by mx1.freebsd.org (Postfix) with ESMTP id B48E5E51 for ; Thu, 30 May 2013 19:06:56 +0000 (UTC) Received: from [192.168.43.26] (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.5/8.14.5) with ESMTP id r4UJ6s3b012095; Thu, 30 May 2013 15:06:54 -0400 (EDT) (envelope-from mike@sentex.net) Message-ID: <51A7A370.1080000@sentex.net> Date: Thu, 30 May 2013 15:07:28 -0400 From: Mike Tancsa Organization: Sentex Communications User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120428 Thunderbird/12.0.1 MIME-Version: 1.0 To: "Lawrence K. Chen, P.Eng." Subject: Re: Rocket Raid 622 in AHCI mode References: <1489716432.21276182.1369940544405.JavaMail.root@k-state.edu> In-Reply-To: <1489716432.21276182.1369940544405.JavaMail.root@k-state.edu> X-Enigmail-Version: 1.4.2 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.72 on 64.7.153.18 Cc: freebsd-hardware@freebsd.org X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 May 2013 19:06:56 -0000 On 5/30/2013 3:02 PM, Lawrence K. Chen, P.Eng. wrote: > > I've been thinking of replacing the cards with ASM1061 based ones....but things have been stable, so not sure I really want to go poking around inside my computer just to try different cards. Though I have plans to drop in another SSD to use as L2ARC in the future (had picked up a Velocity Solo x1 card a while back.... ) Hi, What does the ASM1061 attach as ? A generic AHCI controller ? ---Mike -- ------------------- Mike Tancsa, tel +1 519 651 3400 Sentex Communications, mike@sentex.net Providing Internet services since 1994 www.sentex.net Cambridge, Ontario Canada http://www.tancsa.com/ From owner-freebsd-hardware@FreeBSD.ORG Thu May 30 21:57:59 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id D9E83B37 for ; Thu, 30 May 2013 21:57:59 +0000 (UTC) (envelope-from lkchen@k-state.edu) Received: from ksu-out.merit.edu (ksu-out.merit.edu [207.75.117.132]) by mx1.freebsd.org (Postfix) with ESMTP id A7F68D23 for ; Thu, 30 May 2013 21:57:59 +0000 (UTC) X-Merit-ExtLoop1: 1 X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: AgEFAL7Kp1HPS3TT/2dsb2JhbABagwmDa7sng2sWdIIjAQEFI1YMDxgCAg0ZAlkGiCCpR4lkiAeBJpBAgRQDqH6DK4IL X-IronPort-AV: E=Sophos;i="4.87,773,1363147200"; d="scan'208";a="926522283" X-MERIT-SOURCE: KSU Received: from ksu-sfpop-mailstore02.merit.edu ([207.75.116.211]) by sfpop-ironport05.merit.edu with ESMTP; 30 May 2013 17:57:53 -0400 Date: Thu, 30 May 2013 17:57:53 -0400 (EDT) From: "Lawrence K. Chen, P.Eng." To: Mike Tancsa Message-ID: <2085034742.21347052.1369951073577.JavaMail.root@k-state.edu> In-Reply-To: <51A7A370.1080000@sentex.net> Subject: Re: Rocket Raid 622 in AHCI mode MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [70.179.144.108] X-Mailer: Zimbra 7.2.2_GA_2852 (ZimbraWebClient - GC27 ([unknown])/7.2.2_GA_2852) Cc: freebsd-hardware@freebsd.org X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 May 2013 21:57:59 -0000 ----- Original Message ----- > On 5/30/2013 3:02 PM, Lawrence K. Chen, P.Eng. wrote: > > > > I've been thinking of replacing the cards with ASM1061 based > > ones....but things have been stable, so not sure I really want to > > go poking around inside my computer just to try different cards. > > Though I have plans to drop in another SSD to use as L2ARC in the > > future (had picked up a Velocity Solo x1 card a while back.... ) > > Hi, > What does the ASM1061 attach as ? A generic AHCI controller ? > > ---Mike > > I would expect so, I got the idea of trying it from looking at /usr/src/sys/dev/ahci/ahci.c From owner-freebsd-hardware@FreeBSD.ORG Fri May 31 02:14:17 2013 Return-Path: Delivered-To: freebsd-hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 19D5F337 for ; Fri, 31 May 2013 02:14:17 +0000 (UTC) (envelope-from poyopoyo@puripuri.plala.or.jp) Received: from msa03a.plala.or.jp (msa03.plala.or.jp [IPv6:2400:7800:0:5010::3]) by mx1.freebsd.org (Postfix) with ESMTP id 8B5861FF for ; Fri, 31 May 2013 02:14:16 +0000 (UTC) Received: from i58-95-106-37.s02.a026.ap.plala.or.jp ([58.95.106.37]) by msa04b.plala.or.jp with ESMTP id <20130531021051.FZUA10966.msa04b.plala.or.jp@i58-95-106-37.s02.a026.ap.plala.or.jp> for ; Fri, 31 May 2013 11:10:51 +0900 Date: Fri, 31 May 2013 11:10:50 +0900 Message-ID: <861u8new5x.wl%poyopoyo@puripuri.plala.or.jp> From: poyopoyo@puripuri.plala.or.jp To: freebsd-hardware@freebsd.org Subject: Re: Rocket Raid 622 in AHCI mode In-Reply-To: <51A4DC42.30705@FreeBSD.org> References: <51A4B34F.7000505@sentex.net> <51A4DC42.30705@FreeBSD.org> Mail-Followup-To: freebsd-hardware@freebsd.org User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI/1.14.6 (Maruoka) FLIM/1.14.9 (=?ISO-2022-JP-2?B?R29qGyQoRCtXGyhC?=) APEL/10.8 Emacs/24.3 (amd64-portbld-freebsd10.0) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI 1.14.6 - "Maruoka") Content-Type: text/plain; charset=US-ASCII X-VirusScan: Outbound; msa04b; Fri, 31 May 2013 11:10:51 +0900 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 May 2013 02:14:17 -0000 At Tue, 28 May 2013 19:33:06 +0300, Alexander Motin wrote: > I am not sure what does that number on photo mean, but I haven't even > heard about any SATA3.x port multipliers yet. SiI3826 (as I can identify > it from below) is not a new one and AFAIK only SATA 2.x. http://www.highpoint-tech.com/USA_new/series_r750-specifications.htm http://www.marvell.com/storage/system-solutions/assets/Marvell-88SM97xx-PB.pdf Marvell is shipping 88SM97xx SATA 6Gbps port multiplier chip which can be seen on HighPoint Rocket 750 card, FYI. -- kuro