Date: Tue, 25 Mar 2014 11:42:47 +0000 From: "Desai, Kashyap" <Kashyap.Desai@lsi.com> To: Borja Marcos <borjam@sarenet.es>, Doug Ambrisko <ambrisko@cisco.com> Cc: "scottl@netflix.com" <scottl@netflix.com>, "Radford, Adam" <Adam.Radford@lsi.com>, "sean_bruno@yahoo.com" <sean_bruno@yahoo.com>, "Mankani, Krishnaraddi" <Krishnaraddi.Mankani@lsi.com>, "dwhite@ixsystems.com" <dwhite@ixsystems.com>, "Maloy, Joe" <Joe.Maloy@lsi.com>, "jpaetzel@freebsd.org" <jpaetzel@freebsd.org>, "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, "Kenneth D. Merry" <ken@kdm.org>, "McConnell, Stephen" <Stephen.McConnell@lsi.com> Subject: RE: LSI - MR-Fusion controller driver <mrsas> patch and man page Message-ID: <8bd5b88321704b49baaf4538c6941292@BN1PR07MB247.namprd07.prod.outlook.com> In-Reply-To: <AE6FAC75-D2EA-457F-8654-2ECD57EDAA13@sarenet.es> References: <e59396595152456dbcde63d48f70aa8f@BN1PR07MB247.namprd07.prod.outlook.com> <20140107181139.GC2080@cisco.com> <20140124185356.GA28724@ambrisko.com> <20140124190047.GA34975@ambrisko.com> <9c3fd2b15e9b4c2cb967519a3b7f98ad@BN1PR07MB247.namprd07.prod.outlook.com> <20140318143738.GA65955@cisco.com> <20140320235534.GA92797@cisco.com> <C698AC2A-06A7-4408-9790-20B69FAF31C6@sarenet.es> <20140321160954.GB99545@cisco.com> <5C32A3C7-B28B-4E69-9DF0-EE53181085F7@sarenet.es> <20140324174519.GA30345@cisco.com> <AE6FAC75-D2EA-457F-8654-2ECD57EDAA13@sarenet.es>
next in thread | previous in thread | raw e-mail | index | archive | help
> -----Original Message----- > From: Borja Marcos [mailto:borjam@sarenet.es] > Sent: Tuesday, March 25, 2014 1:42 PM > To: Doug Ambrisko > Cc: Desai, Kashyap; scottl@netflix.com; Radford, Adam; Kenneth D. Merry; > sean_bruno@yahoo.com; Mankani, Krishnaraddi; dwhite@ixsystems.com; > Maloy, Joe; jpaetzel@freebsd.org; freebsd-scsi@freebsd.org; McConnell, > Stephen > Subject: Re: LSI - MR-Fusion controller driver <mrsas> patch and man page >=20 >=20 > On Mar 24, 2014, at 6:45 PM, Doug Ambrisko wrote: >=20 > > On Mon, Mar 24, 2014 at 09:03:43AM +0100, Borja Marcos wrote: > > | > > | On Mar 21, 2014, at 5:09 PM, Doug Ambrisko wrote: > > | > > | > Do you have a simple test case for that? There were issues in the > > | > SCSI command translation which should have been fixed via > > | > switching it to use the CAM translation code. Can you try it via a= RAID > volume of one disk? > > | > > | Yes, I tried all those possibilities. "syspd" devices, passthrough > > | (same result, corrruption) and when creating single-disk RAID0 > > | volumes it worked, no corruption. > > > > Okay, so you don't have and simple test case to show this. What are > > you currently doing? I would think the syspd and the single disk > > RAID0 should be very similar. I'll take a look at that. >=20 > Yes, sorry for not being so explicit. >=20 > The server I tried has a backplane, with 23 disks fitted. At first I as= sumed it > would work, so I just created a raidz2 vdev and begun running benchmarks= . > That's where I noticed that it was corrupting data badly. >=20 > The disks were set as "JBOD" with the Invader's built-in configuration to= ol > and visible as mfisyspd drives. >=20 > So I reduced it to using a single disk. Choosing one of them at random, I= did a > newfs. Mounted, and executed > bonnie++. I got a panic after some time. To avoid the inconvenience of > bonnie++panics, I moved to ZFS (single disk vdev) > so that I could still detect corruption. >=20 > The tests I did were: >=20 > - Disk in JBOD/syspd mode, accessed through /dev/mfisyspd -> corruption > - Disk in JBOD/syspd mode, passthrough enabled, accessed as /dev/daX -> > corruption > - 1 disk RAID0 volume, accessed as /dev/mfid0 -> NO corruption >=20 > And I understand that syspd and RAID0 should be really different. I > understand that it's a shallow layer doing little more than passing throu= gh > commands? I should read the driver thoroughly it seems... Pity there are = no > available documents. >=20 > > | Just in case I tried with SSDs (Samsung EVO 840 and OCZ Vertex 4) > > | and Seagate SAS disks. > > > > So it seems generic. Size of disk might be useful. >=20 >=20 > The Samsung are 1 TB, the OCZ are the 512 MB ones, and the Seagates are > 146 GB. > The SSDs (Samsung and OCZ) are SATA disks, the Seagate is SAS. The Seagat= e > was corrupted as well in the same test cases, so it doesn't seem to be li= nked > to SATA-on-SAS or to TRIM. TRIM, anyway, doesn't work on "mfisyspd" > devices, which is a problem if you want to run ZFS on SSD disks. >=20 > > | Actually I think that converting passthrough disks to "mfisyspd" or > > | an equivalent is a bad idea, unless, of course, there's a compelling > > | reason I don't realize. For instance, if you are using SSDs you need > > | access to the TRIM command. > > | > > | So, if I can vote, I vote with arms and legs for "da" devices in > > | case it's a passthough. > > > > Nothing is planned to be removed from mfi. However, LSI would like > > mrsas to be used on newer cards. We've let people LSI know that > > people use the pass through mode and having the logical volumes come > > up as /dev/daX and pass through would be confusing. Granted mrsas > > doesn't support pass through so that isn't a problem. >=20 > Pass through is a critical feature when you are running ZFS. I think it's= a real > problem if the new driver (the only one that will support newer cards, I > guess) won't support pass-through. >=20 > And of course I know that you can just buy making sure that you get HBAs > instead of RAID cards, but so many manufacturers bundle those RAID > controllers in fixed configurations that it wouldn't be practical. >=20 > The ideal solution would be to have logical volumes come up as /dev/mfid, > /dev/mrsas, /dev/mfisyspd or whatever, and passthrough devices as real > passthrough devices like /dev/da. That would avoid confusion. >=20 > So, please, LSI, consider it. It would really add to the versatility of y= our cards > and there's nothing to lose. Borja: <mrsas> driver will attach Raid volume and JBOD (SysPD) to the CAM layer. = It is not good to expose hidden raid volume or what we called as pass-throu= gh device here to the OS for many reason.. Other than management things li= ke SMART monitor, we cannot/should not do file system IO on pass-through de= vices. With <mfi> it might be true that user always do file system IO on <= mfiX> deivce and consider /dev/daX as pass-through device... With <mrsas> a= ll device will be seen as <daX>. You cannot identify which will be a pass-t= hrough and which is configured device by LSI config utils. It is not a complex code change if pass-through device is required for <mrs= as>, but it is just a matter of no use and more error prone to expose devic= es as pass-through.=20 None of the LSI driver does this including <mps> and <mrsas> in FreeBSD + <= megaraid_sas> and <mpt2sas>/<mpt3sas> in Linux. If you can express what functionality you think it is missing, if there is = not pass-through device ? Are you doing ZFS (File system IO) on Pass-throu= gh device. ?=20 If yes, then why can't you create JBOD/SysPD for that purpose? >=20 > > | It's not a pressing need, I installed a LSI2008 card and I connected > > | the SAS backplane to it, so for now the Invader is a non-problem, > > | but if it will help to run some tests with the mps driver I can > > | certainly do it. > > > > The mps driver is a different thing, mrsas is LSI's replacement for > > mfi with newer cards. So if you could try syspd and single RAID 0 > > test with mrsas that would be great to see if that shows the same > > pattern of syspd failing with mfi and passing with single RAID 0. > > Again, the logical volumes will show up as /dev/da* with no pass > > through devices. If I can reproduce the problem then I can look into > > here or others might be able to find the solution. >=20 > I'll try, thanks :) >=20 >=20 >=20 >=20 >=20 > Borja. >=20 >=20 >=20
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?8bd5b88321704b49baaf4538c6941292>