From owner-freebsd-scsi@FreeBSD.ORG Tue Mar 25 11:42:50 2014 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5252BFDD; Tue, 25 Mar 2014 11:42:50 +0000 (UTC) Received: from na01-by2-obe.outbound.protection.outlook.com (mail-by2lp0239.outbound.protection.outlook.com [207.46.163.239]) (using TLSv1 with cipher AES128-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id C91CFEB9; Tue, 25 Mar 2014 11:42:49 +0000 (UTC) Received: from BN1PR07MB247.namprd07.prod.outlook.com (10.141.64.146) by BN1PR07MB438.namprd07.prod.outlook.com (10.141.63.13) with Microsoft SMTP Server (TLS) id 15.0.898.11; Tue, 25 Mar 2014 11:42:48 +0000 Received: from BN1PR07MB247.namprd07.prod.outlook.com ([169.254.7.243]) by BN1PR07MB247.namprd07.prod.outlook.com ([169.254.7.243]) with mapi id 15.00.0898.005; Tue, 25 Mar 2014 11:42:48 +0000 From: "Desai, Kashyap" To: Borja Marcos , Doug Ambrisko Subject: RE: LSI - MR-Fusion controller driver patch and man page Thread-Topic: LSI - MR-Fusion controller driver patch and man page Thread-Index: Ac7ykKZzGPABTpK8R3+D5UAv3TD3WQWOC9SAAHVVybAAG8S5gAAWATMAABupEoADWG5MAAAAPT6AClUK/9AACzY8AAB4EYUAABFKigAAELyuAACF5P6AABRP6YAAHkYYgAAGneGw Date: Tue, 25 Mar 2014 11:42:47 +0000 Message-ID: <8bd5b88321704b49baaf4538c6941292@BN1PR07MB247.namprd07.prod.outlook.com> References: <20140107181139.GC2080@cisco.com> <20140124185356.GA28724@ambrisko.com> <20140124190047.GA34975@ambrisko.com> <9c3fd2b15e9b4c2cb967519a3b7f98ad@BN1PR07MB247.namprd07.prod.outlook.com> <20140318143738.GA65955@cisco.com> <20140320235534.GA92797@cisco.com> <20140321160954.GB99545@cisco.com> <5C32A3C7-B28B-4E69-9DF0-EE53181085F7@sarenet.es> <20140324174519.GA30345@cisco.com> In-Reply-To: Accept-Language: en-US Content-Language: en-US X-MS-Has-Attach: X-MS-TNEF-Correlator: x-originating-ip: [192.19.224.250] x-forefront-prvs: 01613DFDC8 x-forefront-antispam-report: SFV:NSPM; SFS:(10019001)(6009001)(428001)(52314003)(51444003)(199002)(189002)(377454003)(24454002)(51704005)(13464003)(95416001)(20776003)(92566001)(83322001)(87936001)(81686001)(63696002)(85306002)(93136001)(54316002)(81342001)(77096001)(74366001)(77982001)(59766001)(80976001)(56816005)(76796001)(19580405001)(19580395003)(76576001)(79102001)(81816001)(86362001)(56776001)(94316002)(76482001)(87266001)(80022001)(95666003)(76786001)(65816001)(90146001)(97186001)(46102001)(2656002)(47736001)(81542001)(31966008)(54356001)(4396001)(74662001)(53806001)(74502001)(47446002)(85852003)(66066001)(49866001)(93516002)(33646001)(97336001)(47976001)(50986001)(83072002)(74876001)(51856001)(74706001)(69226001)(98676001)(74316001)(24736002); DIR:OUT; SFP:1102; SCL:1; SRVR:BN1PR07MB438; H:BN1PR07MB247.namprd07.prod.outlook.com; FPR:EEFFFA35.A0F293AD.F2D9517F.4EF8BDC2.206B1; MLV:sfv; PTR:InfoNoRecords; A:1; MX:1; LANG:en; received-spf: None (: lsi.com does not designate permitted sender hosts) Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-OriginatorOrg: lsi.com X-Mailman-Approved-At: Tue, 25 Mar 2014 11:52:14 +0000 Cc: "scottl@netflix.com" , "Radford, Adam" , "sean_bruno@yahoo.com" , "Mankani, Krishnaraddi" , "dwhite@ixsystems.com" , "Maloy, Joe" , "jpaetzel@freebsd.org" , "freebsd-scsi@freebsd.org" , "Kenneth D. Merry" , "McConnell, Stephen" X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.17 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Mar 2014 11:42:50 -0000 > -----Original Message----- > From: Borja Marcos [mailto:borjam@sarenet.es] > Sent: Tuesday, March 25, 2014 1:42 PM > To: Doug Ambrisko > Cc: Desai, Kashyap; scottl@netflix.com; Radford, Adam; Kenneth D. Merry; > sean_bruno@yahoo.com; Mankani, Krishnaraddi; dwhite@ixsystems.com; > Maloy, Joe; jpaetzel@freebsd.org; freebsd-scsi@freebsd.org; McConnell, > Stephen > Subject: Re: LSI - MR-Fusion controller driver patch and man page >=20 >=20 > On Mar 24, 2014, at 6:45 PM, Doug Ambrisko wrote: >=20 > > On Mon, Mar 24, 2014 at 09:03:43AM +0100, Borja Marcos wrote: > > | > > | On Mar 21, 2014, at 5:09 PM, Doug Ambrisko wrote: > > | > > | > Do you have a simple test case for that? There were issues in the > > | > SCSI command translation which should have been fixed via > > | > switching it to use the CAM translation code. Can you try it via a= RAID > volume of one disk? > > | > > | Yes, I tried all those possibilities. "syspd" devices, passthrough > > | (same result, corrruption) and when creating single-disk RAID0 > > | volumes it worked, no corruption. > > > > Okay, so you don't have and simple test case to show this. What are > > you currently doing? I would think the syspd and the single disk > > RAID0 should be very similar. I'll take a look at that. >=20 > Yes, sorry for not being so explicit. >=20 > The server I tried has a backplane, with 23 disks fitted. At first I as= sumed it > would work, so I just created a raidz2 vdev and begun running benchmarks= . > That's where I noticed that it was corrupting data badly. >=20 > The disks were set as "JBOD" with the Invader's built-in configuration to= ol > and visible as mfisyspd drives. >=20 > So I reduced it to using a single disk. Choosing one of them at random, I= did a > newfs. Mounted, and executed > bonnie++. I got a panic after some time. To avoid the inconvenience of > bonnie++panics, I moved to ZFS (single disk vdev) > so that I could still detect corruption. >=20 > The tests I did were: >=20 > - Disk in JBOD/syspd mode, accessed through /dev/mfisyspd -> corruption > - Disk in JBOD/syspd mode, passthrough enabled, accessed as /dev/daX -> > corruption > - 1 disk RAID0 volume, accessed as /dev/mfid0 -> NO corruption >=20 > And I understand that syspd and RAID0 should be really different. I > understand that it's a shallow layer doing little more than passing throu= gh > commands? I should read the driver thoroughly it seems... Pity there are = no > available documents. >=20 > > | Just in case I tried with SSDs (Samsung EVO 840 and OCZ Vertex 4) > > | and Seagate SAS disks. > > > > So it seems generic. Size of disk might be useful. >=20 >=20 > The Samsung are 1 TB, the OCZ are the 512 MB ones, and the Seagates are > 146 GB. > The SSDs (Samsung and OCZ) are SATA disks, the Seagate is SAS. The Seagat= e > was corrupted as well in the same test cases, so it doesn't seem to be li= nked > to SATA-on-SAS or to TRIM. TRIM, anyway, doesn't work on "mfisyspd" > devices, which is a problem if you want to run ZFS on SSD disks. >=20 > > | Actually I think that converting passthrough disks to "mfisyspd" or > > | an equivalent is a bad idea, unless, of course, there's a compelling > > | reason I don't realize. For instance, if you are using SSDs you need > > | access to the TRIM command. > > | > > | So, if I can vote, I vote with arms and legs for "da" devices in > > | case it's a passthough. > > > > Nothing is planned to be removed from mfi. However, LSI would like > > mrsas to be used on newer cards. We've let people LSI know that > > people use the pass through mode and having the logical volumes come > > up as /dev/daX and pass through would be confusing. Granted mrsas > > doesn't support pass through so that isn't a problem. >=20 > Pass through is a critical feature when you are running ZFS. I think it's= a real > problem if the new driver (the only one that will support newer cards, I > guess) won't support pass-through. >=20 > And of course I know that you can just buy making sure that you get HBAs > instead of RAID cards, but so many manufacturers bundle those RAID > controllers in fixed configurations that it wouldn't be practical. >=20 > The ideal solution would be to have logical volumes come up as /dev/mfid, > /dev/mrsas, /dev/mfisyspd or whatever, and passthrough devices as real > passthrough devices like /dev/da. That would avoid confusion. >=20 > So, please, LSI, consider it. It would really add to the versatility of y= our cards > and there's nothing to lose. Borja: driver will attach Raid volume and JBOD (SysPD) to the CAM layer. = It is not good to expose hidden raid volume or what we called as pass-throu= gh device here to the OS for many reason.. Other than management things li= ke SMART monitor, we cannot/should not do file system IO on pass-through de= vices. With it might be true that user always do file system IO on <= mfiX> deivce and consider /dev/daX as pass-through device... With a= ll device will be seen as . You cannot identify which will be a pass-t= hrough and which is configured device by LSI config utils. It is not a complex code change if pass-through device is required for , but it is just a matter of no use and more error prone to expose devic= es as pass-through.=20 None of the LSI driver does this including and in FreeBSD + <= megaraid_sas> and / in Linux. If you can express what functionality you think it is missing, if there is = not pass-through device ? Are you doing ZFS (File system IO) on Pass-throu= gh device. ?=20 If yes, then why can't you create JBOD/SysPD for that purpose? >=20 > > | It's not a pressing need, I installed a LSI2008 card and I connected > > | the SAS backplane to it, so for now the Invader is a non-problem, > > | but if it will help to run some tests with the mps driver I can > > | certainly do it. > > > > The mps driver is a different thing, mrsas is LSI's replacement for > > mfi with newer cards. So if you could try syspd and single RAID 0 > > test with mrsas that would be great to see if that shows the same > > pattern of syspd failing with mfi and passing with single RAID 0. > > Again, the logical volumes will show up as /dev/da* with no pass > > through devices. If I can reproduce the problem then I can look into > > here or others might be able to find the solution. >=20 > I'll try, thanks :) >=20 >=20 >=20 >=20 >=20 > Borja. >=20 >=20 >=20