From owner-freebsd-hardware@freebsd.org Mon Aug 1 13:30:09 2016 Return-Path: <owner-freebsd-hardware@freebsd.org> Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 816A2BA7F48; Mon, 1 Aug 2016 13:30:09 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu01176b.smtpx.saremail.com (cu01176b.smtpx.saremail.com [195.16.151.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 06F271D5B; Mon, 1 Aug 2016 13:30:08 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop01.sare.net (Postfix) with ESMTPSA id D14279DCCC2; Mon, 1 Aug 2016 15:29:58 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i From: Borja Marcos <borjam@sarenet.es> In-Reply-To: <20160801151203.14a7a67d@freyja.zeit4.iv.bundesimmobilien.de> Date: Mon, 1 Aug 2016 15:29:58 +0200 Cc: Jason Zhang <jasonzhang@cyphytech.com>, freebsd-performance@freebsd.org, freebsd-current@freebsd.org, freebsd-stable@freebsd.org, freebsd-hardware@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <0CA1A1F1-AFDD-4763-84C3-2FC059F44789@sarenet.es> References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> <B2239257-E136-4918-8FA3-1EE51BC50FB4@sarenet.es> <20160801084504.563c79cf@freyja.zeit4.iv.bundesimmobilien.de> <1519EC23-0DBC-4139-96F6-250EF872A14B@sarenet.es> <20160801151203.14a7a67d@freyja.zeit4.iv.bundesimmobilien.de> To: "O. Hartmann" <ohartman@zedat.fu-berlin.de> X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware <freebsd-hardware.freebsd.org> List-Unsubscribe: <https://lists.freebsd.org/mailman/options/freebsd-hardware>, <mailto:freebsd-hardware-request@freebsd.org?subject=unsubscribe> List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hardware/> List-Post: <mailto:freebsd-hardware@freebsd.org> List-Help: <mailto:freebsd-hardware-request@freebsd.org?subject=help> List-Subscribe: <https://lists.freebsd.org/mailman/listinfo/freebsd-hardware>, <mailto:freebsd-hardware-request@freebsd.org?subject=subscribe> X-List-Received-Date: Mon, 01 Aug 2016 13:30:09 -0000 > On 01 Aug 2016, at 15:12, O. Hartmann <ohartman@zedat.fu-berlin.de> = wrote: >=20 > First, thanks for responding so quickly. >=20 >> - The third option is to make the driver expose the SAS devices like = a HBA >> would do, so that they are visible to the CAM layer, and disks are = handled by >> the stock =E2=80=9Cda=E2=80=9D driver, which is the ideal solution.=20= >=20 > I didn't find any switch which offers me the opportunity to put the = PRAID > CP400i into a simple HBA mode. The switch is in the FreeBSD mfi driver, the loader tunable I mentioned, = regardless of what the card firmware does or pretends to do. It=E2=80=99s not visible doing a "sysctl -a=E2=80=9D, but it exists and = it=E2=80=99s unique even. It=E2=80=99s defined here: = https://svnweb.freebsd.org/base/stable/10/sys/dev/mfi/mfi_cam.c?revision=3D= 267084&view=3Dmarkup (line 93) >> In order to do it you need a couple of things. You need to set the = variable >> hw.mfi.allow_cam_disk_passthrough=3D1 and to load the mfip.ko module. >>=20 >> When booting installation media, enter command mode and use these = commands: >>=20 >> ----- >> set hw.mfi.allow_cam_disk_passthrough=3D1 >> load mfip >> boot >> =E2=80=94=E2=80=94=E2=80=94 >=20 > Well, I'm truly aware of this problemacy and solution (now), but I run = into a > henn-egg-problem, literally. As long as I can boot off of the = installation > medium, I have a kernel which deals with the setting. But the boot = medium is > supposed to be a SSD sitting with the PRAID CP400i controller itself! = So, I > never be able to boot off the system without crippling the ability to = have a > fullspeed ZFS configuration which I suppose to have with HBA mode, but = not > with any of the forced RAID modes offered by the controller.=20 Been there plenty of times, even argued quite strongly about the = advantages of ZFS against hardware based RAID 5 cards. :) I remember when the Dell salesmen couldn=E2=80=99t possibly = understand why I wanted a =E2=80=9Csoftware based RAID rather than a robust, hardware based solution=E2=80=9D :D=20 At worst, you can set up a simple boot from a thumb drive or, even = better, a SATADOM installed inside the server. I guess it will have SATA ports on the mainboard. That=E2=80=99s what I use to do. = FreeNAS uses a similar approach as well. And some modern servers also can boot from a SD card which you can use just to load the kernel. Depending on the number of disks you have, you can also sacrifice two to = set up a mirror with a =E2=80=9Cnomal=E2=80=9D boot system, and using the rest of the disks for ZFS. Actually I=E2=80=99ve got an old server I = set up in 2012. It has 16 disks, and I created a logical volume (mirror) with 2 disks for boot, the other 14 disks for ZFS. If I installed this server now I would do it different, booting off a = thumb drive. But I was younger and naiver :) Borja.