From owner-freebsd-scsi@FreeBSD.ORG Fri Nov 23 18:54:36 2012 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 50919582 for ; Fri, 23 Nov 2012 18:54:36 +0000 (UTC) (envelope-from h.schmalzbauer@omnilan.de) Received: from host.omnilan.net (s1.omnilan.net [62.245.232.135]) by mx1.freebsd.org (Postfix) with ESMTP id C4EC08FC15 for ; Fri, 23 Nov 2012 18:54:35 +0000 (UTC) Received: from titan.inop.wdn.omnilan.net (titan.inop.wdn.omnilan.net [172.21.3.1]) (authenticated bits=0) by host.omnilan.net (8.13.8/8.13.8) with ESMTP id qANIuNl5000787 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Fri, 23 Nov 2012 19:56:23 +0100 (CET) (envelope-from h.schmalzbauer@omnilan.de) Message-ID: <50AFC668.1040203@omnilan.de> Date: Fri, 23 Nov 2012 19:54:32 +0100 From: Harald Schmalzbauer Organization: OmniLAN User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; de-DE; rv:1.9.2.8) Gecko/20100906 Lightning/1.0b2 Thunderbird/3.1.2 MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Subject: cam SCSI negotiation issues (mpt in that case), only 3.300MB/s transfers X-Enigmail-Version: 1.1.2 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig7534BC35971E11E10FEA878F" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 23 Nov 2012 18:54:36 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig7534BC35971E11E10FEA878F Content-Type: text/plain; charset=ISO-8859-15 Content-Transfer-Encoding: quoted-printable Hello, (I was advised to post this also to freebsd-scsi@, but I'm not subscribed) I have a LSI20320IE, wich is a LSI1030 (U320-SCSI, supported by MPT) behind an PCI-X-PCIe bridge. I'm using that for two tape drives: LTO-3 and DAT72. My (solved, see below) first problem was, that passthrough of that HBA with ESXi5.0 didn't work. As soon as mpt was kldloaded, there was a heavy interrupt storm detected (>100k irq/s) and throttling occured. No matter what IRQ it was assigend and no matter if it was shared or not. The dmesg output hasn't shown any info about MSI usage, so I told ESXi to use legacy interrupts in host mode instead of msi. Didn't help anything. After I verified that this passthrough configuration works well on other OS, I tried out some things and found out that hint.mpt.0.msi_enable=3D"1" solves the interrupt storm problem, although dmesg output still is exactly the same: mpt0: port 0x4000-0x40ff mem 0xd6440000-0xd645ffff,0xd6420000-0xd643ffff irq 18 at device 0.0 on pci3 mpt0: MPI Version=3D1.2.15.0 Btw, I'm still curious what the sysctl "hw.mpt.0.role" means? With my LSI1030 it's "3", an onther 1068, it's "1" Now after I got passthrough working, I first tried to saturate my LTO-3. With the help of mbuffer (zfs send -R DATASET/filesystem@totape | mbuffer -s 1m -t -m 500M -P 80 -f -o /dev/nsa1) I saw 60-120MB/s transfer rates, avaeraging at 100MB/s at moderate interrupt load (~1k). So for so fine. But there are problems with the DAT72 drive. The SCSI negotiation seems to fail: sa0 at mpt0 bus 0 scbus3 target 0 lun 0 sa0: Removable Sequential Access SCSI-3 device sa0: 3.300MB/s transfers Now I read here http://lists.freebsd.org/pipermail/freebsd-current/2006-May/063603.html that this may be just cosmetic, but in my case I can't get more than ~3MB/s to/from the tape. Once again I changed the controller to another guest OS and got 5,5MB/s avarage write transfer rate with ntbakup (wich is sensible for the gained compression ratio). lsiutil.exe shows negotiation parameters with 80MB/s (40MT/s 16bit), which correponds to the drive's interface. Back in FreeBSD, I had no luck using "camcontrol negotiate sa0 -R 40" to alter the negotiation parameters. It always shows 3,300MB/s and seems to reflect reality. I'd higly appreciate any help, since I have no idea how this could be fixed. Slot-constraints limit my choice of SCSI-HBAs to PCIe ones... Thnaks, -Harry --------------enig7534BC35971E11E10FEA878F Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.18 (FreeBSD) iEYEARECAAYFAlCvxmkACgkQLDqVQ9VXb8jPeQCgphOuOOqzzk+EJgW0nyldoxPJ MRsAn23iSaU8zdoX/74IjDEuy6JyD9lj =Ovfx -----END PGP SIGNATURE----- --------------enig7534BC35971E11E10FEA878F--