From owner-freebsd-stable@freebsd.org Mon Aug 1 06:45:23 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 67342BACD35; Mon, 1 Aug 2016 06:45:23 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from outpost1.zedat.fu-berlin.de (outpost1.zedat.fu-berlin.de [130.133.4.66]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 281C81A8F; Mon, 1 Aug 2016 06:45:22 +0000 (UTC) (envelope-from ohartman@zedat.fu-berlin.de) Received: from inpost2.zedat.fu-berlin.de ([130.133.4.69]) by outpost.zedat.fu-berlin.de (Exim 4.85) with esmtps (TLSv1.2:DHE-RSA-AES256-GCM-SHA384:256) (envelope-from ) id <1bU6yM-000mWX-2P>; Mon, 01 Aug 2016 08:45:10 +0200 Received: from p578a69f9.dip0.t-ipconnect.de ([87.138.105.249] helo=freyja.zeit4.iv.bundesimmobilien.de) by inpost2.zedat.fu-berlin.de (Exim 4.85) with esmtpsa (TLSv1.2:AES256-GCM-SHA384:256) (envelope-from ) id <1bU6yL-001051-PK>; Mon, 01 Aug 2016 08:45:09 +0200 Date: Mon, 1 Aug 2016 08:45:04 +0200 From: "O. Hartmann" To: Borja Marcos Cc: Jason Zhang , freebsd-performance@freebsd.org, freebsd-current@freebsd.org, freebsd-hardware@freebsd.org, freebsd-stable@freebsd.org Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i Message-ID: <20160801084504.563c79cf@freyja.zeit4.iv.bundesimmobilien.de> In-Reply-To: References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> Organization: FU Berlin X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.29; amd64-portbld-freebsd11.0) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: 87.138.105.249 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 01 Aug 2016 06:45:23 -0000 On Wed, 22 Jun 2016 08:58:08 +0200 Borja Marcos wrote: > > On 22 Jun 2016, at 04:08, Jason Zhang wrote: > >=20 > > Mark, > >=20 > > Thanks > >=20 > > We have same RAID setting both on FreeBSD and CentOS including cache > > setting. In FreeBSD, I enabled the write cache but the performance is = the > > same. =20 > >=20 > > We don=E2=80=99t use ZFS or UFS, and test the performance on the RAW GE= OM disk > > =E2=80=9Cmfidx=E2=80=9D exported by mfi driver. We observed the =E2=80= =9Cgstat=E2=80=9D result and found > > that the write latency is too high. When we =E2=80=9Cdd" the disk with= 8k, it is > > lower than 1ms, but it is 6ms on 64kb write. It seems that each single > > write operation is very slow. But I don=E2=80=99t know whether it is a = driver > > problem or not. =20 >=20 > There is an option you can use (I do it all the time!) to make the card > behave as a plain HBA so that the disks are handled by the =E2=80=9Cda=E2= =80=9D driver.=20 >=20 > Add this to /boot/loader.conf >=20 > hw.mfi.allow_cam_disk_passthrough=3D1 > mfip_load=3D=E2=80=9CYES" >=20 > And do the tests accessing the disks as =E2=80=9Cda=E2=80=9D. To avoid co= nfusions, it=E2=80=99s > better to make sure the disks are not part of a =E2=80=9Cjbod=E2=80=9D or= logical volume > configuration. >=20 >=20 >=20 >=20 > Borja. [...] How is this supposed to work when ALL disks (including boot device) are set= tled with the mfi (in our case, it is a Fujitsu CP400i, based upon LSI3008 and detected within FreeBSD 11-BETA and 12-CURRENT) controller itself? I did not find any solution to force the CP400i into a mode making itself acting as a HBA (we intend to use all drives with ZFS and let FreeBSD kernel/ZFS control everything). The boot device is a 256 GB Samsung SSD for enterprise use and putting the = UEFI load onto a EFI partition from 11-CURRENT-ALPHA4 is worse: dd takes up to almost a minute to put the image onto the SSD. The SSD active LED is blinki= ng alle the time indicating activity. Caches are off. I tried to enable the ca= che via the mfiutil command by 'mfiutil cache mfid0 enable', but it failed ... = It failed also on all other attached drives. I didn't further go into more investigations right now, since the experience with the EFI boot loader makes me suspect bad performance and that is harsh= so to speak. Glad to have found this thread anyway. I cross post this also to CURRENT as it might be an issue with CURRENT ... Kind regards, Oliver Hartmann