From owner-freebsd-performance@freebsd.org Mon Jun 20 06:58:23 2016 Return-Path: Delivered-To: freebsd-performance@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 1847EA7B9D8; Mon, 20 Jun 2016 06:58:23 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from vps1.elischer.org (vps1.elischer.org [204.109.63.16]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "vps1.elischer.org", Issuer "CA Cert Signing Authority" (not verified)) by mx1.freebsd.org (Postfix) with ESMTPS id EEC7C22B2; Mon, 20 Jun 2016 06:58:22 +0000 (UTC) (envelope-from julian@freebsd.org) Received: from Julian-MBP3.local (ppp121-45-242-176.lns20.per4.internode.on.net [121.45.242.176]) (authenticated bits=0) by vps1.elischer.org (8.15.2/8.15.2) with ESMTPSA id u5K6w3mT086534 (version=TLSv1.2 cipher=DHE-RSA-AES128-SHA bits=128 verify=NO); Sun, 19 Jun 2016 23:58:06 -0700 (PDT) (envelope-from julian@freebsd.org) Subject: Re: mfi driver performance too bad on LSI MegaRAID SAS 9260-8i To: Jason Zhang , freebsd-stable@freebsd.org, freebsd-performance@freebsd.org, freebsd-hardware@freebsd.org References: From: Julian Elischer Message-ID: <72fcce2c-2296-a9fe-bce2-d3f92caeb1d0@freebsd.org> Date: Mon, 20 Jun 2016 14:57:58 +0800 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 8bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.22 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 20 Jun 2016 06:58:23 -0000 On 17/06/2016 3:16 PM, Jason Zhang wrote: > Hi, > > I am working on storage service based on FreeBSD. I look forward to a good result because many professional storage company use FreeBSD as its OS. But I am disappointed with the Bad performance. I tested the the performance of LSI MegaRAID 9260-8i and had the following bad result: > > 1. Test environment: > (1) OS: FreeBSD 10.0 release > (2) Memory: 16G > (3) RAID adapter: LSI MegaRAID 9260-8i > (4) Disks: 9 SAS hard drives (10000 rpm), performance is expected for each hard drive > (5) Test tools: fio with io-depth=1, thread num is 32 and block size is 64k or 1M > (6) RAID configuration: RAID 5, stripe size is 1M > > 2. Test result: > (1) write performance too bad: 20Mbytes/s throughput and 200 random write IOPS > (2) read performance is expected: 700Mbytes/s throughput and 1500 random read IOPS > > > I tested the same hardware configuration with CentOS linux and Linux's write performance is 5 times better than FreeBSD. > > > Anyone encountered the same performance problem? Does the mfi driver have performance issue or I should give up on FreeBSD? > > Unfortunatley issues related to performance can often be very specific. We use the LSI cards with great success under FreeBSD 8 in our product at work but it is impossible to say what is specifically wrong in your setup. Some years ago I did discover that fio needed to have completely different arguments to get good performance under FreeBSD, so please check that first. What does performance look like with a single large write stream? Also look at the handling of interrupts (systat -vmstat) to ensure that interrupts are being handled correctly. that can vary greatly from motherboard to motherboard and bios to bios. (even between revisions). Sometimes Linux will cope differently with these issues as they have better support from the motherboard makers themselves. (sometimes we cope better too). One final thought.. make sure you have partitioned your drives and filesyste,s so that all the block boundaries agree and line up. At on place I worked we found we had accidentally partitioned all our drives starting 63 sectors into the drive. That did NOT work well. :-) 8k raid stripe writes were always 2 writes (and sometimes a read) > > > 张京城 Jason > > 赛凡信息科技（厦门）有限公司 > Cyphy Technology (Xiamen) Co.Ltd. > 公司总部：厦门市软件园望海路55号A座901-904单元 > 研发总部：北京市东城区美术馆后街大取灯胡同2号 > 热线：4008798066 > 总机：0592-2936100 > 邮箱：jasonzhang@cyphytech.com > 公司网址：Http://www.cyphytech.com > > > _______________________________________________ > freebsd-performance@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-performance > To unsubscribe, send any mail to "freebsd-performance-unsubscribe@freebsd.org" > >