Date: Thu, 20 Mar 2014 16:55:34 -0700 From: Doug Ambrisko <ambrisko@cisco.com> To: "Desai, Kashyap" <Kashyap.Desai@lsi.com> Cc: "scottl@netflix.com" <scottl@netflix.com>, "Radford, Adam" <Adam.Radford@lsi.com>, "Kenneth D. Merry" <ken@kdm.org>, "sean_bruno@yahoo.com" <sean_bruno@yahoo.com>, "Mankani, Krishnaraddi" <Krishnaraddi.Mankani@lsi.com>, "dwhite@ixsystems.com" <dwhite@ixsystems.com>, "Maloy, Joe" <Joe.Maloy@lsi.com>, "jpaetzel@freebsd.org" <jpaetzel@freebsd.org>, "freebsd-scsi@freebsd.org" <freebsd-scsi@freebsd.org>, "McConnell, Stephen" <Stephen.McConnell@lsi.com> Subject: Re: LSI - MR-Fusion controller driver <mrsas> patch and man page Message-ID: <20140320235534.GA92797@cisco.com> In-Reply-To: <20140318143738.GA65955@cisco.com> References: <08ba2a262fba45f687cdd3225f325110@BN1PR07MB247.namprd07.prod.outlook.com> <20140103211449.GA69721@cisco.com> <8c423414ecc2421fbace3eb9f386be91@BN1PR07MB247.namprd07.prod.outlook.com> <20140106182935.GC93278@cisco.com> <e59396595152456dbcde63d48f70aa8f@BN1PR07MB247.namprd07.prod.outlook.com> <20140107181139.GC2080@cisco.com> <20140124185356.GA28724@ambrisko.com> <20140124190047.GA34975@ambrisko.com> <9c3fd2b15e9b4c2cb967519a3b7f98ad@BN1PR07MB247.namprd07.prod.outlook.com> <20140318143738.GA65955@cisco.com>
next in thread | previous in thread | raw e-mail | index | archive | help
I kicked the tires. I did some tests with our home grown management
tool that uses the standard LSI ioctl format and that has worked okay.
I got a panic before when recreating a RAID. So things have improved.
Doing some performance tests, I seem to have tracked down how it
happens and why I didn't see a long time ago. Performance is okay
with mfi versus mrasas on ThunderBolt cards (ie. 9266-8i) but when
I used an Invader card (9361-8i) read performance is really bad.
I'm using 4, 300G, 2.5" SAS disk (Toshiba AL13SEB300) in a RAID 10
configuration. Write performance is fine, read performance is
really bad. With mfi performance is fine for read and write.
The mrsas write performance is ~302974Kbytes and ~7738Kbytes for
read. That is really bad. It can be seen with just a
dd if=/dev/da0 of=/dev/zero bs=1m
With mfi I see ~364128Kbytes writes and ~214817Kbytes reads. So
something is strange. The firmware on card is:
Name: LSI MegaRAID 9361-8i
Serial Number: SV33505325
Firmware BIOS 6.13.00.1_4.14.05.00_0x06010600
Firmware CTLR 5.01-0005
Firmware APP 4.210.50-3015
Firmware NVDT 3.1310.00-0062
Firmware BTBL 3.00.00.00-0009
which was the lastest when I looks on the LSI web site a month ago or so.
Any ideas of how to debug this?
I recently started testing with the Invader card so that is why I started
seeing performance issues. With ThunderBolt performance is okay and
about the same.
Thanks,
Doug A.
On Tue, Mar 18, 2014 at 07:37:38AM -0700, Doug Ambrisko wrote:
| Sounds good. I'll try it out and see how it works.
|
| I won't be able to make today's meeting.
|
| Thanks,
|
| Doug A.
|
| On Tue, Mar 18, 2014 at 09:23:23AM +0000, Desai, Kashyap wrote:
| | Hi,
| |
| | Please find the latest <mrsas> patch for upstream commit.
| |
| | - This patch include all latest and greatest feature supported by
| | <mrsas> driver released at LSI external site.
| | - mfi vs mrsas interopt is resolved using latest <mfi> commit which uses
| | tunable " hw.mfi.mrsas_enable".
| |
| | <mrsas> man page is still under re-work. I will be posting <mrsas> man
| | page as a follow up action item.
| |
| | Thanks, Kashyap
| |
| | > -----Original Message-----
| | > From: Doug Ambrisko [mailto:ambrisko@ambrisko.com]
| | > Sent: Saturday, January 25, 2014 12:31 AM
| | > To: Doug Ambrisko
| | > Cc: Desai, Kashyap; freebsd-scsi@freebsd.org; scottl@netflix.com;
| | > sean_bruno@yahoo.com; dwhite@ixsystems.com; jpaetzel@freebsd.org;
| | > Maloy, Joe; Mankani, Krishnaraddi; McConnell, Stephen; Saxena, Sumit;
| | > Radford, Adam; Kenneth D. Merry
| | > Subject: Re: LSI - MR-Fusion controller driver <mrsas> patch and man page
| | >
| | > On Fri, Jan 24, 2014 at 10:53:56AM -0800, Doug Ambrisko wrote:
| | > | On Tue, Jan 07, 2014 at 10:11:39AM -0800, Doug Ambrisko wrote:
| | > | [snip]
| | > | | Yes, we can probably make the minimal change to mfi to allow mrsas
| | > | | to optionally take over. That can probably be done the quickest.
| | > |
| | > | Here is the patch I propose to mfi(4) to allow mrsas(4) to optionally
| | > | take newer cards.
| | >
| | > I noticed that this patch is partially incomplete since I didn't have
| | > FLAGS_MRSAS added to all of the TBOLT ID's. I'll fix that in the commit.
| | >
| | > | Index: mfi_pci.c
| | > |
| | > ==========================================================
| | > =========
| | > | --- mfi_pci.c (revision 260231)
| | > | +++ mfi_pci.c (working copy)
| | > | @@ -112,6 +112,11 @@
| | > | SYSCTL_INT(_hw_mfi, OID_AUTO, msi, CTLFLAG_RDTUN, &mfi_msi, 0,
| | > | "Enable use of MSI interrupts");
| | > |
| | > | +static int mfi_mrsas_enable = 0;
| | > | +TUNABLE_INT("hw.mfi.mrsas_enable", &mfi_msi); SYSCTL_INT(_hw_mfi,
| | > | +OID_AUTO, mrsas_enable, CTLFLAG_RDTUN, &mfi_mrsas_enable,
| | > | + 0, "Allow mrasas to take newer cards");
| | > | +
| | > | struct mfi_ident {
| | > | uint16_t vendor;
| | > | uint16_t device;
| | > | @@ -127,11 +132,11 @@
| | > | {0x1000, 0x005b, 0x1028, 0x1f34, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Dell PERC H710P Mini (monolithics)"},
| | > | {0x1000, 0x005b, 0x1028, 0x1f35, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Dell PERC H710 Adapter"},
| | > | {0x1000, 0x005b, 0x1028, 0x1f37, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Dell PERC H710 Mini (blades)"},
| | > | - {0x1000, 0x005b, 0x1028, 0x1f38, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Dell PERC H710 Mini (monolithics)"},
| | > | - {0x1000, 0x005b, 0x8086, 0x9265, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Intel (R) RAID Controller RS25DB080"},
| | > | - {0x1000, 0x005b, 0x8086, 0x9285, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Intel (R) RAID Controller RS25NB008"},
| | > | - {0x1000, 0x005b, 0xffff, 0xffff, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "ThunderBolt"},
| | > | - {0x1000, 0x005d, 0xffff, 0xffff, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT, "Invader"},
| | > | + {0x1000, 0x005b, 0x1028, 0x1f38, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT| MFI_FLAGS_MRSAS, "Dell PERC H710 Mini
| | > (monolithics)"},
| | > | + {0x1000, 0x005b, 0x8086, 0x9265, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT| MFI_FLAGS_MRSAS, "Intel (R) RAID Controller
| | > RS25DB080"},
| | > | + {0x1000, 0x005b, 0x8086, 0x9285, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT| MFI_FLAGS_MRSAS, "Intel (R) RAID Controller
| | > RS25NB008"},
| | > | + {0x1000, 0x005b, 0xffff, 0xffff, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT| MFI_FLAGS_MRSAS, "ThunderBolt"},
| | > | + {0x1000, 0x005d, 0xffff, 0xffff, MFI_FLAGS_SKINNY|
| | > MFI_FLAGS_TBOLT|
| | > | +MFI_FLAGS_MRSAS, "Invader"},
| | > | {0x1000, 0x0060, 0x1028, 0xffff, MFI_FLAGS_1078, "Dell PERC 6"},
| | > | {0x1000, 0x0060, 0xffff, 0xffff, MFI_FLAGS_1078, "LSI MegaSAS
| | > 1078"},
| | > | {0x1000, 0x0071, 0xffff, 0xffff, MFI_FLAGS_SKINNY, "Drake Skinny"},
| | > | @@ -178,7 +183,13 @@
| | > |
| | > | if ((id = mfi_find_ident(dev)) != NULL) {
| | > | device_set_desc(dev, id->desc);
| | > | - return (BUS_PROBE_DEFAULT);
| | > | +
| | > | + /* give priority to mrsas if tunable set */
| | > | + TUNABLE_INT_FETCH("hw.mfi.mrsas_enable",
| | > &mfi_mrsas_enable);
| | > | + if ((id->flags & MFI_FLAGS_MRSAS) && mfi_mrsas_enable)
| | > | + return (BUS_PROBE_LOW_PRIORITY);
| | > | + else
| | > | + return (BUS_PROBE_DEFAULT);
| | > | }
| | > | return (ENXIO);
| | > | }
| | > | Index: mfivar.h
| | > |
| | > ==========================================================
| | > =========
| | > | --- mfivar.h (revision 260231)
| | > | +++ mfivar.h (working copy)
| | > | @@ -199,6 +199,7 @@
| | > | #define MFI_FLAGS_GEN2 (1<<6)
| | > | #define MFI_FLAGS_SKINNY (1<<7)
| | > | #define MFI_FLAGS_TBOLT (1<<8)
| | > | +#define MFI_FLAGS_MRSAS (1<<9)
| | > | // Start: LSIP200113393
| | > | bus_dma_tag_t verbuf_h_dmat;
| | > | bus_dmamap_t verbuf_h_dmamap;
| | > |
| | > | This creates a hw.mfi.mrsas_enable tunable to control it. The method
| | > | via hints wasn't the best since for one the unit index was being
| | > | abused a non-unit specfic option. It was also a little strange to
| | > | have mrsas hint be in mfi(4).
| | > |
| | > | Then we need a minor change to mrsas.c
| | > |
| | > |
| | > | --- ../mrsas.orig/mrsas.c 2014-01-03 11:30:28.000000000 -0800
| | > | +++ ./mrsas.c 2014-01-24 10:43:20.000000000 -0800
| | > | @@ -328,25 +328,11 @@ static struct mrsas_ident * mrsas_find_i static
| | > | int mrsas_probe(device_t dev) {
| | > | struct mrsas_ident *id;
| | > | - unsigned int force = 0, ivar;
| | > |
| | > | if ((id = mrsas_find_ident(dev)) != NULL) {
| | > | - if (id->device == 0x005b || id->device == 0x005d) {
| | > | - resource_int_value("mrsas", 0, "fusion_force", &ivar);
| | > | -
| | > | - if (ivar == 0 || ivar == 1)
| | > | - force = ivar;
| | > | -
| | > | - device_set_desc(dev, id->desc);
| | > | - if (force)
| | > | - return (BUS_PROBE_DEFAULT);
| | > | - //return (BUS_PROBE_SPECIFIC); //give priority to MFI driver
| | > | - else
| | > | - return (BUS_PROBE_LOW_PRIORITY);
| | > | - }
| | > | - else
| | > | - device_set_desc(dev, id->desc);
| | > | - return (BUS_PROBE_DEFAULT);
| | > | + device_set_desc(dev, id->desc);
| | > | + /* between BUS_PROBE_DEFAULT and BUS_PROBE_LOW_PRIORITY
| | > */
| | > | + return (-30);
| | > | }
| | > | return (ENXIO);
| | > |
| | > | So that its probe part way between mfi(4) results and then it doesn't
| | > | have to change.
| | > |
| | > | If no one has concerns then I'll check in the mfi(4) change.
| | >
| | > Thanks,
| | >
| | > Doug A.
| |
|
|
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20140320235534.GA92797>