From owner-freebsd-scsi@FreeBSD.ORG Tue Mar 25 08:12:21 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id DC9F048B; Tue, 25 Mar 2014 08:12:21 +0000 (UTC) Received: from cu01176a.smtpx.saremail.com (cu01176a.smtpx.saremail.com [195.16.150.151]) by mx1.freebsd.org (Postfix) with ESMTP id 7B25F91E; Tue, 25 Mar 2014 08:12:21 +0000 (UTC) Received: from [172.16.2.2] (izaro.sarenet.es [192.148.167.11]) by proxypop03.sare.net (Postfix) with ESMTPSA id 3FE289DCF24; Tue, 25 Mar 2014 09:12:13 +0100 (CET) Subject: Re: LSI - MR-Fusion controller driver patch and man page Mime-Version: 1.0 (Apple Message framework v1283) Content-Type: text/plain; charset=us-ascii From: Borja Marcos In-Reply-To: <20140324174519.GA30345@cisco.com> Date: Tue, 25 Mar 2014 09:12:09 +0100 Content-Transfer-Encoding: quoted-printable Message-Id: References: <20140107181139.GC2080@cisco.com> <20140124185356.GA28724@ambrisko.com> <20140124190047.GA34975@ambrisko.com> <9c3fd2b15e9b4c2cb967519a3b7f98ad@BN1PR07MB247.namprd07.prod.outlook.com> <20140318143738.GA65955@cisco.com> <20140320235534.GA92797@cisco.com> <20140321160954.GB99545@cisco.com> <5C32A3C7-B28B-4E69-9DF0-EE53181085F7@sarenet.es> <20140324174519.GA30345@cisco.com> To: Doug Ambrisko X-Mailer: Apple Mail (2.1283) X-Mailman-Approved-At: Tue, 25 Mar 2014 11:39:11 +0000 Cc: "scottl@netflix.com" , "Radford, Adam" , "sean_bruno@yahoo.com" , "Mankani, Krishnaraddi" , "dwhite@ixsystems.com" , "Maloy, Joe" , "jpaetzel@freebsd.org" , "freebsd-scsi@freebsd.org" , "Kenneth D. Merry" , "McConnell, Stephen" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 08:12:21 -0000 On Mar 24, 2014, at 6:45 PM, Doug Ambrisko wrote: > On Mon, Mar 24, 2014 at 09:03:43AM +0100, Borja Marcos wrote: > |=20 > | On Mar 21, 2014, at 5:09 PM, Doug Ambrisko wrote: > |=20 > | > Do you have a simple test case for that? There were issues in the = SCSI > | > command translation which should have been fixed via switching it = to use > | > the CAM translation code. Can you try it via a RAID volume of one = disk? > |=20 > | Yes, I tried all those possibilities. "syspd" devices, passthrough=20= > | (same result, corrruption) and when creating single-disk RAID0 = volumes=20 > | it worked, no corruption. >=20 > Okay, so you don't have and simple test case to show this. What are > you currently doing? I would think the syspd and the single disk = RAID0 > should be very similar. I'll take a look at that. Yes, sorry for not being so explicit. The server I tried has a backplane, with 23 disks fitted. At first I = assumed it would work, so I just created a raidz2 vdev and begun running benchmarks. That's where = I noticed that it was corrupting data badly. The disks were set as "JBOD" with the Invader's built-in configuration = tool and visible as mfisyspd drives. So I reduced it to using a single disk. Choosing one of them at random, = I did a newfs. Mounted, and executed bonnie++. I got a panic after some time. To avoid the inconvenience of = panics, I moved to ZFS (single disk vdev)=20 so that I could still detect corruption. The tests I did were: - Disk in JBOD/syspd mode, accessed through /dev/mfisyspd -> corruption - Disk in JBOD/syspd mode, passthrough enabled, accessed as /dev/daX -> = corruption - 1 disk RAID0 volume, accessed as /dev/mfid0 -> NO corruption And I understand that syspd and RAID0 should be really different. I = understand that it's a shallow layer doing little more than passing through commands? I should read the = driver thoroughly it seems... Pity there are no available documents. > | Just in case I tried with SSDs (Samsung EVO 840 and OCZ Vertex 4) = and=20 > | Seagate SAS disks. >=20 > So it seems generic. Size of disk might be useful. The Samsung are 1 TB, the OCZ are the 512 MB ones, and the Seagates are = 146 GB.=20 The SSDs (Samsung and OCZ) are SATA disks, the Seagate is SAS. The = Seagate was corrupted as well in the same test cases, so it doesn't seem to be linked to SATA-on-SAS or to = TRIM. TRIM, anyway, doesn't work on "mfisyspd" devices, which is a problem if you want to run ZFS on SSD = disks. > | Actually I think that converting passthrough disks to "mfisyspd" or = an > | equivalent is a bad idea, unless, of course, there's a compelling = reason > | I don't realize. For instance, if you are using SSDs you need access = to > | the TRIM command. > |=20 > | So, if I can vote, I vote with arms and legs for "da" devices in = case it's > | a passthough. >=20 > Nothing is planned to be removed from mfi. However, LSI would like = mrsas > to be used on newer cards. We've let people LSI know that people use > the pass through mode and having the logical volumes come up as = /dev/daX > and pass through would be confusing. Granted mrsas doesn't support = pass > through so that isn't a problem. Pass through is a critical feature when you are running ZFS. I think = it's a real problem if the new driver (the only one that will support newer cards, I guess) = won't support pass-through.=20 And of course I know that you can just buy making sure that you get HBAs = instead of RAID cards, but so many manufacturers bundle those RAID controllers in = fixed configurations that it wouldn't be practical. The ideal solution would be to have logical volumes come up as = /dev/mfid, /dev/mrsas, /dev/mfisyspd or whatever, and passthrough devices as real passthrough devices like = /dev/da. That would avoid confusion. So, please, LSI, consider it. It would really add to the versatility of = your cards and there's nothing to lose. > | It's not a pressing need, I installed a LSI2008 card and I connected = the=20 > | SAS backplane to it, so for now the Invader > | is a non-problem, but if it will help to run some tests with the mps=20= > | driver I can certainly do it. >=20 > The mps driver is a different thing, mrsas is LSI's replacement for = mfi > with newer cards. So if you could try syspd and single RAID 0 test > with mrsas that would be great to see if that shows the same pattern > of syspd failing with mfi and passing with single RAID 0. Again, > the logical volumes will show up as /dev/da* with no pass through > devices. If I can reproduce the problem then I can look into here > or others might be able to find the solution. I'll try, thanks :) Borja.