From owner-freebsd-hardware@freebsd.org Mon Jun 20 06:58:23 2016 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1847EA7B9D8; Mon, 20 Jun 2016 06:58:23 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EEC7C22B2; Mon, 20 Jun 2016 06:58:22 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-242-176.lns20.per4.internode.on.net [121.45.242.176]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u5K6w3mT086534 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 19 Jun 2016 23:58:06 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i To: Jason Zhang , freebsd-stable@freebsd.org, freebsd-performance@freebsd.org, freebsd-hardware@freebsd.org References: From: Julian Elischer Message-ID: <72fcce2c-2296-a9fe-bce2-d3f92caeb1d0@freebsd.org> Date: Mon, 20 Jun 2016 14:57:58 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 06:58:23 -0000 On 17/06/2016 3:16 PM, Jason Zhang wrote: > Hi, > > I am working on storage service based on FreeBSD. I look forward to a good result because many professional storage company use FreeBSD as its OS. But I am disappointed with the Bad performance. I tested the the performance of LSI MegaRAID 9260-8i and had the following bad result: > > 1. Test environment: > (1) OS: FreeBSD 10.0 release > (2) Memory: 16G > (3) RAID adapter: LSI MegaRAID 9260-8i > (4) Disks: 9 SAS hard drives (10000 rpm), performance is expected for each hard drive > (5) Test tools: fio with io-depth=1, thread num is 32 and block size is 64k or 1M > (6) RAID configuration: RAID 5, stripe size is 1M > > 2. Test result: > (1) write performance too bad: 20Mbytes/s throughput and 200 random write IOPS > (2) read performance is expected: 700Mbytes/s throughput and 1500 random read IOPS > > > I tested the same hardware configuration with CentOS linux and Linux's write performance is 5 times better than FreeBSD. > > > Anyone encountered the same performance problem? Does the mfi driver have performance issue or I should give up on FreeBSD? > > Unfortunatley issues related to performance can often be very specific. We use the LSI cards with great success under FreeBSD 8 in our product at work but it is impossible to say what is specifically wrong in your setup. Some years ago I did discover that fio needed to have completely different arguments to get good performance under FreeBSD, so please check that first. What does performance look like with a single large write stream? Also look at the handling of interrupts (systat -vmstat) to ensure that interrupts are being handled correctly. that can vary greatly from motherboard to motherboard and bios to bios. (even between revisions). Sometimes Linux will cope differently with these issues as they have better support from the motherboard makers themselves. (sometimes we cope better too). One final thought.. make sure you have partitioned your drives and filesyste,s so that all the block boundaries agree and line up. At on place I worked we found we had accidentally partitioned all our drives starting 63 sectors into the drive. That did NOT work well. :-) 8k raid stripe writes were always 2 writes (and sometimes a read) > > > 张京城 Jason > > 赛凡信息科技（厦门）有限公司 > Cyphy Technology (Xiamen) Co.Ltd. > 公司总部：厦门市软件园望海路55号A座901-904单元 > 研发总部：北京市东城区美术馆后街大取灯胡同2号 > 热线：4008798066 > 总机：0592-2936100 > 邮箱：jasonzhang@cyphytech.com > 公司网址：Http://www.cyphytech.com > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > > From owner-freebsd-hardware@freebsd.org Tue Jun 21 16:29:38 2016 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 14F26AC5D2C; Tue, 21 Jun 2016 16:29:38 +0000 (UTC) (envelope-from feld@feld.me) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id D30872652; Tue, 21 Jun 2016 16:29:37 +0000 (UTC) (envelope-from feld@feld.me) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id F2BED209F8; Tue, 21 Jun 2016 12:29:35 -0400 (EDT) Received: from web4 ([10.202.2.214]) by compute1.internal (MEProxy); Tue, 21 Jun 2016 12:29:36 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d=feld.me; h= content-transfer-encoding:content-type:date:from:in-reply-to :message-id:mime-version:references:subject:to:x-sasl-enc :x-sasl-enc; s=mesmtp; bh=61UxS9wiRoCeHyUNwMRvyGCMvN8=; b=rlWYXV i8bKgxO7Xy2PbSY4QEBnOKaxYVAoYJA6A/lxOpZ7ccch5GJNbM3wtFEiC50Hq11R aH44s/1nzN+IFnee4EYIiggmRW0AicqVoe89sJx46UpbhpvT6/k5UF50H0OhU+wy Y/cEe7ldHrq9iJYs0dXbbQ1lKYg8GcIkxmGDs= DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=61UxS9wiRoCeHyU NwMRvyGCMvN8=; b=mAk9UJXqQrwz6ahM3otjWbu95wwMwoRIz3EMmOqJjnFP+jx h8qbdanLJTF7L4MoUQL1YTK8l8raXPEGospfyJ2Fn+nVGfXK+/vkKhqs2agW0q04 WTLmmlhsPGmAFW/npz2EZCfR2W4aXlGvGsi1/CaJzBkm0pCWEQCBy2iLXFVc= Received: by mailuser.nyi.internal (Postfix, from userid 99) id C274FCC6CE; Tue, 21 Jun 2016 12:29:35 -0400 (EDT) Message-Id: <1466526575.2692982.644272073.33CF6186@webmail.messagingengine.com> X-Sasl-Enc: 45ij14uaCKBQGsRNnMoCKrYPwxodMPuwEiLasxUUZzoZ 1466526575 From: Mark Felder To: Jason Zhang , freebsd-stable@freebsd.org, freebsd-performance@freebsd.org, freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-eb8588c4 In-Reply-To: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i Date: Tue, 21 Jun 2016 11:29:35 -0500 X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 16:29:38 -0000 On Fri, Jun 17, 2016, at 02:17, Jason Zhang wrote: > Hi, > > I am working on storage service based on FreeBSD. I look forward to a > good result because many professional storage company use FreeBSD as its > OS. But I am disappointed with the Bad performance. I tested the the > performance of LSI MegaRAID 9260-8i and had the following bad result: > > 1. Test environment: > (1) OS: FreeBSD 10.0 release 10.0-RELEASE is no longer supported. Can you reproduce this on 10.3-RELEASE? -- Mark Felder feld@feld.me From owner-freebsd-hardware@freebsd.org Tue Jun 21 16:36:43 2016 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id A1EF1AC5FE2; Tue, 21 Jun 2016 16:36:43 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from out3-smtp.messagingengine.com (out3-smtp.messagingengine.com [66.111.4.27]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 7CA582AC8; Tue, 21 Jun 2016 16:36:43 +0000 (UTC) (envelope-from feld@FreeBSD.org) Received: from compute1.internal (compute1.nyi.internal [10.202.2.41]) by mailout.nyi.internal (Postfix) with ESMTP id 22E4B2092B; Tue, 21 Jun 2016 12:36:42 -0400 (EDT) Received: from web4 ([10.202.2.214]) by compute1.internal (MEProxy); Tue, 21 Jun 2016 12:36:42 -0400 DKIM-Signature: v=1; a=rsa-sha1; c=relaxed/relaxed; d= messagingengine.com; h=content-transfer-encoding:content-type :date:from:in-reply-to:message-id:mime-version:references :subject:to:x-sasl-enc:x-sasl-enc; s=smtpout; bh=QYn+6/JOvuakMzb KpvdFmEmnZuE=; b=p50kHpsAPgbhZ615Yfu56Y9RdrQQrzPOEdD+RnkCa/Ob1eM MhnWtyYJIaie7+J/+0u43NaISSP4Os/l6cn0uHw9fuvV490A7h0pQxC3DAJfrahA 8FUrnTc3Pqz6qxtIqLEWLFo4nGizeUcpUXDZISF33mqZjjttSn4uqltb5Sp0= Received: by mailuser.nyi.internal (Postfix, from userid 99) id F1955CC672; Tue, 21 Jun 2016 12:36:41 -0400 (EDT) Message-Id: <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> X-Sasl-Enc: k4/kYWm4awCS1xnjnT4ajGNVz/VSF8t/f/bie76/NtQk 1466527001 From: Mark Felder To: Jason Zhang , freebsd-stable@freebsd.org, freebsd-performance@freebsd.org, freebsd-hardware@freebsd.org MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Type: text/plain X-Mailer: MessagingEngine.com Webmail Interface - ajax-eb8588c4 Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i Date: Tue, 21 Jun 2016 11:36:41 -0500 In-Reply-To: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 21 Jun 2016 16:36:43 -0000 On Fri, Jun 17, 2016, at 02:17, Jason Zhang wrote: > Hi, > > I am working on storage service based on FreeBSD. I look forward to a > good result because many professional storage company use FreeBSD as its > OS. But I am disappointed with the Bad performance. I tested the the > performance of LSI MegaRAID 9260-8i and had the following bad result: > > 1. Test environment: > (1) OS: FreeBSD 10.0 release 10.0-RELEASE is no longer supported. Can you test this on 10.3-RELEASE? Have you confirmed that both servers are using identical RAID controller settings? It's possible the CentOS install has enabled write caching but it's disabled on your FreeBSD server. Are you using UFS or ZFS on FreeBSD? Do you have atime enabled? I believe CentOS is going to have "relatime" or "nodiratime" by default to mitigate the write penalty on each read access. We need more data :-) -- Mark Felder ports-secteam member feld@FreeBSD.org From owner-freebsd-hardware@freebsd.org Wed Jun 22 02:14:22 2016 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 379B3AC524F; Wed, 22 Jun 2016 02:14:22 +0000 (UTC) (envelope-from jasonzhang@cyphytech.com) Received: from m50-210.qiye.163.com (m50-210.qiye.163.com [123.125.50.210]) by mx1.freebsd.org (Postfix) with ESMTP id DED3F16A7; Wed, 22 Jun 2016 02:14:19 +0000 (UTC) (envelope-from jasonzhang@cyphytech.com) Received: from [192.168.120.81] (unknown [125.39.30.134]) by smtp6 (Coremail) with SMTP id RNOowEDp80Mt82lXOrMOBQ--.15S3; Wed, 22 Jun 2016 10:08:45 +0800 (CST) Content-Type: text/plain; charset=gb2312 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i From: Jason Zhang In-Reply-To: <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> Date: Wed, 22 Jun 2016 10:08:49 +0800 Cc: freebsd-stable@freebsd.org, freebsd-performance@freebsd.org, freebsd-hardware@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> To: Mark Felder X-Mailer: Apple Mail (2.3124) X-CM-TRANSID: RNOowEDp80Mt82lXOrMOBQ--.15S3 X-Coremail-Antispam: 1Uf129KBjDUn29KB7ZKAUJUUUUU529EdanIXcx71UUUUU7v73 VFW2AGmfu7bjvjm3AaLaJ3UbIYCTnIWIevJa73UjIFyTuYvjxUrXo7UUUUU X-Originating-IP: [125.39.30.134] X-CM-SenderInfo: pmdv00x2kd0wg6f11xp1whuxoofrz/1tbiQA2QHFbdGLa1zwAAse X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 02:14:22 -0000 Mark, Thanks We have same RAID setting both on FreeBSD and CentOS including cache = setting. In FreeBSD, I enabled the write cache but the performance is = the same. =20 We don=A1=AFt use ZFS or UFS, and test the performance on the RAW GEOM = disk =A1=B0mfidx=A1=B1 exported by mfi driver. We observed the = =A1=B0gstat=A1=B1 result and found that the write latency is too high. When we =A1=B0dd" the disk with 8k, it is lower than 1ms, = but it is 6ms on 64kb write. It seems that each single write operation = is very slow. But I don=A1=AFt know whether it is a driver problem or not. Jason > =D4=DA 2016=C4=EA6=D4=C222=C8=D5=A3=AC=C9=CF=CE=E712:36=A3=ACMark = Felder =D0=B4=B5=C0=A3=BA >=20 >=20 >=20 > On Fri, Jun 17, 2016, at 02:17, Jason Zhang wrote: >> Hi, >>=20 >> I am working on storage service based on FreeBSD. I look forward to = a >> good result because many professional storage company use FreeBSD as = its >> OS. But I am disappointed with the Bad performance. I tested the = the >> performance of LSI MegaRAID 9260-8i and had the following bad result: >>=20 >> 1. Test environment: >> (1) OS: FreeBSD 10.0 release >=20 > 10.0-RELEASE is no longer supported. Can you test this on = 10.3-RELEASE? >=20 > Have you confirmed that both servers are using identical RAID = controller > settings? It's possible the CentOS install has enabled write caching = but > it's disabled on your FreeBSD server. Are you using UFS or ZFS on > FreeBSD? Do you have atime enabled? I believe CentOS is going to have > "relatime" or "nodiratime" by default to mitigate the write penalty on > each read access. >=20 > We need more data :-) >=20 >=20 > --=20 > Mark Felder > ports-secteam member > feld@FreeBSD.org From owner-freebsd-hardware@freebsd.org Wed Jun 22 05:44:20 2016 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C364DAC5365; Wed, 22 Jun 2016 05:44:20 +0000 (UTC) (envelope-from d.eracledes@albourne.com) Received: from despam-hpn-0.albourne.com (ns-hpn-0.albourne.com [64.47.157.36]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "*.albourne.com", Issuer "SwissSign Server Silver CA 2014 - G22" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 865B0221E; Wed, 22 Jun 2016 05:44:20 +0000 (UTC) (envelope-from d.eracledes@albourne.com) X-Virus-Scanned: amavisd-new at despam-hpn-0.albourne.com Received: from mail-hpn-0.intern.albourne.com (mail-hpn-0.intern.albourne.com [192.168.96.24]) by despam-hpn-0.albourne.com (8.15.2/8.15.2/Albourne/DNSBL/grey) with ESMTP id u5M5SFGY027800; Wed, 22 Jun 2016 05:28:16 GMT Received: from localhost (localhost [127.0.0.1]) by mail-hpn-0.intern.albourne.com (Postfix) with ESMTP id 63916357DD5A5; Wed, 22 Jun 2016 05:28:15 +0000 (UTC) Received: from mail-hpn-0.intern.albourne.com ([127.0.0.1]) by localhost (mail-hpn-0.intern.albourne.com [127.0.0.1]) (amavisd-new, port 10032) with ESMTP id BI1zOc298Tgf; Wed, 22 Jun 2016 05:28:15 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mail-hpn-0.intern.albourne.com (Postfix) with ESMTP id 27A6F357DD5A3; Wed, 22 Jun 2016 05:28:15 +0000 (UTC) X-Quarantine-ID: Received: from mail-hpn-0.intern.albourne.com ([127.0.0.1]) by localhost (mail-hpn-0.intern.albourne.com [127.0.0.1]) (amavisd-new, port 10026) with ESMTP id AlXNKQbooAs8; Wed, 22 Jun 2016 05:28:15 +0000 (UTC) Received: from mail-hpn-0.intern.albourne.com (mail-hpn-0.intern.albourne.com [192.168.96.24]) by mail-hpn-0.intern.albourne.com (Postfix) with ESMTP id 0B3D5357DD3F6; Wed, 22 Jun 2016 05:28:15 +0000 (UTC) Content-Type: text/plain; charset="iso-8859-1" MIME-Version: 1.0 Content-Transfer-Encoding: quoted-printable X-Client-ID: 172709 X-Mailer: BlackBerry Email (10.3.2.2876) X-Mailer: Zimbra 8.0.5_GA_5839 (MobileSync - RIM-Passport-SQW100-1/10.3.2.2876) Message-ID: <20160622052814.5779540.99036.172709@albourne.com> Date: Wed, 22 Jun 2016 05:28:15 +0000 (UTC) Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i From: Doros Eracledes In-Reply-To: <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> Cc: freebsd-performance@freebsd.org, freebsd-stable@freebsd.org, freebsd-hardware@freebsd.org Thread-Topic: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i Thread-Index: gZYY2ta0yZK6SLr/gqj9E82KQYMYEA== X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 05:44:20 -0000 As a side note, we also use this controller with FreeBSD 10.1 but configure= d each drive as a JBOD and then created raidz zfs pools and that was much f= aster than to let the LSI do raid5.=A0 Best Doros From owner-freebsd-hardware@freebsd.org Wed Jun 22 07:05:03 2016 Return-Path: Delivered-To: freebsd-hardware@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DF723AC678A; Wed, 22 Jun 2016 07:05:03 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from cu1176c.smtpx.saremail.com (cu1176c.smtpx.saremail.com [195.16.148.151]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id A1F722730; Wed, 22 Jun 2016 07:05:03 +0000 (UTC) (envelope-from borjam@sarenet.es) Received: from [172.16.8.36] (izaro.sarenet.es [192.148.167.11]) by proxypop02.sare.net (Postfix) with ESMTPSA id 48E909DC9D4; Wed, 22 Jun 2016 08:58:08 +0200 (CEST) Content-Type: text/plain; charset=utf-8 Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\)) Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i From: Borja Marcos In-Reply-To: <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> Date: Wed, 22 Jun 2016 08:58:08 +0200 Cc: Mark Felder , freebsd-performance@freebsd.org, freebsd-stable@freebsd.org, freebsd-hardware@freebsd.org Content-Transfer-Encoding: quoted-printable Message-Id: References: <16CD100A-3BD0-47BA-A91E-F445E5DF6DBC@cyphytech.com> <1466527001.2694442.644278905.18E236CD@webmail.messagingengine.com> <1790833A-9292-4A46-B43C-BF41C7C801BE@cyphytech.com> To: Jason Zhang X-Mailer: Apple Mail (2.3124) X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 22 Jun 2016 07:05:04 -0000 > On 22 Jun 2016, at 04:08, Jason Zhang = wrote: >=20 > Mark, >=20 > Thanks >=20 > We have same RAID setting both on FreeBSD and CentOS including cache = setting. In FreeBSD, I enabled the write cache but the performance is = the same. =20 >=20 > We don=E2=80=99t use ZFS or UFS, and test the performance on the RAW = GEOM disk =E2=80=9Cmfidx=E2=80=9D exported by mfi driver. We observed = the =E2=80=9Cgstat=E2=80=9D result and found that the write latency > is too high. When we =E2=80=9Cdd" the disk with 8k, it is lower than = 1ms, but it is 6ms on 64kb write. It seems that each single write = operation is very slow. But I don=E2=80=99t know > whether it is a driver problem or not. There is an option you can use (I do it all the time!) to make the card = behave as a plain HBA so that the disks are handled by the =E2=80=9Cda=E2=80= =9D driver.=20 Add this to /boot/loader.conf hw.mfi.allow_cam_disk_passthrough=3D1 mfip_load=3D=E2=80=9CYES" And do the tests accessing the disks as =E2=80=9Cda=E2=80=9D. To avoid = confusions, it=E2=80=99s better to make sure the disks are not part of a = =E2=80=9Cjbod=E2=80=9D or logical volume configuration. Borja.