From owner-freebsd-stable@FreeBSD.ORG Fri Jan 16 17:41:02 2015 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 737C0442 for ; Fri, 16 Jan 2015 17:41:02 +0000 (UTC) Received: from pit.databus.com (Databus-1-pt.tunnel.tserv4.nyc4.ipv6.he.net [IPv6:2001:470:1f06:80b::2]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 2E3041FD for ; Fri, 16 Jan 2015 17:41:02 +0000 (UTC) Received: by pit.databus.com (Postfix, from userid 202) id A5FBE54BD; Fri, 16 Jan 2015 12:41:00 -0500 (EST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=databus.com; s=20140217; t=1421430060; bh=JG6g0Wb9/jgXxjgXzIAgq+kkhshQLjjQM+kkp4aCnLk=; l=2549; h=Date:From:To:Cc:Subject:References:In-Reply-To; b=RHvbzy1Q9WPNWsg7f+RtlZlY9APSUh+38BUsdUh4BiHNMFKv0fydF6slbHAwncUJ+ OOUSME78w2exNDxq//jSvZPdcPZ8fB1+z+oMXZrp4Gpkk2UmHcn4ZYmuswg7Y/XH/Y Wy6sv6wndZjZhUZxgjnzmrh6FMDvBmHEFKdSkrBbks3ZvZw91I0B4aUry5p6EE/Ofh Cylh0C1r/2qJY+WuECKOAhFTtVxyWr9XoYiTYYYRCIE0x7nkYwsMOsmjvJH/KyNxOt 5gm+mPSUtyadoS/SEgJ3MHB9oZ/vjj8bWiTPj+M/tLdpjJ51Xs9bymPo1nmprEMZSP tRcP75ZDERcLg== Date: Fri, 16 Jan 2015 12:41:00 -0500 From: Barney Wolff To: Mihai Vintila Subject: Re: Poor performance on Intel P3600 NVME driver Message-ID: <20150116174100.GA9487@pit.databus.com> References: <54B7F769.40605@gmail.com> <20150115175927.GA19071@zxy.spb.ru> <54B945A8.10408@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <54B945A8.10408@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Cc: freebsd-stable@freebsd.org X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 16 Jan 2015 17:41:02 -0000 Apology if I've missed a mention, but are you sure there aren't zfs snapshots in existence, and things like access time updates differing between Linux and FreeBSD? On Fri, Jan 16, 2015 at 07:08:56PM +0200, Mihai Vintila wrote: > There are clearly issues with the controller driver on FreeBSD. Not only > zfs related: > I've did some testing under Centos 7 with ext4 > Command line used: iozone -Rb /root/output_iops.wks -O -i 0 -i > 1 -i 2 -e -+n -r4K -r 8K -r 32K -r 64K -r 128K -r 512K -s 8G > Time Resolution = 0.000001 seconds. > Processor cache size set to 1024 kBytes. > Processor cache line size set to 32 bytes. > File stride size set to 17 * record size. > random random bkwd record stride > kB reclen write rewrite read reread read > write read rewrite read fwrite frewrite fread freread > 8388608 4 156982 0 859809 0 656572 208156 > 8388608 8 82545 0 503549 0 435005 111282 > 8388608 32 21620 0 129740 0 135738 31361 > 8388608 64 11070 0 66027 0 71660 16175 > 8388608 128 5558 0 31800 0 35361 8066 > 8388608 512 1383 0 7520 0 8472 2068 > > > Same test on BSD with ufs > Command line used: iozone -Rb /root/output.wks -O -i 0 -i 1 -i > 2 -e -+n -r4K -r 8K -r 32K -r 64K -r 128K -s 1G > Time Resolution = 0.000001 seconds. > Processor cache size set to 1024 Kbytes. > Processor cache line size set to 32 bytes. > File stride size set to 17 * record size. > random random bkwd record stride > KB reclen write rewrite read reread read > write read rewrite read fwrite frewrite fread freread > 1048576 4 120642 0 368188 0 252302 41650 > 1048576 8 61871 0 253074 0 189851 35015 > 1048576 32 15925 0 89190 0 80004 19357 > 1048576 64 7958 0 47190 0 59120 10070 > 1048576 128 4024 0 23619 0 22435 5542 > > > ZFS test on BSD: > Command line used: iozone -Rb /root/output.wks -O -i 0 -i 1 -i > 2 -e -+n -r4K -s 1G > Time Resolution = 0.000001 seconds. > Processor cache size set to 1024 Kbytes. > Processor cache line size set to 32 bytes. > File stride size set to 17 * record size. > random random bkwd record stride > KB reclen write rewrite read reread read > write read rewrite read fwrite frewrite fread freread > 1048576 4 69600 0 99750 0 92007 49556 > > > > Perftest while write is close to what it should be, read takes a 4x penalty > nvmecontrol perftest -n 32 -o write -s 4096 -t30 nvme0ns1 > Threads: 32 Size: 4096 WRITE Time: 30 IO/s: 211294 MB/s: 825 > nvmecontrol perftest -n 32 -o read -s 4096 -t30 nvme0ns1 > Threads: 32 Size: 4096 READ Time: 30 IO/s: 221365 MB/s: 864