From owner-freebsd-questions@FreeBSD.ORG Wed Oct 23 19:30:56 2013 Return-Path: Delivered-To: freebsd-questions@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 13AC95A8 for ; Wed, 23 Oct 2013 19:30:56 +0000 (UTC) (envelope-from aurfalien@gmail.com) Received: from mail-pb0-x22a.google.com (mail-pb0-x22a.google.com [IPv6:2607:f8b0:400e:c01::22a]) (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id D8C6629ED for ; Wed, 23 Oct 2013 19:30:55 +0000 (UTC) Received: by mail-pb0-f42.google.com with SMTP id jt11so1179382pbb.1 for ; Wed, 23 Oct 2013 12:30:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:mime-version:content-type:from:in-reply-to:date:cc :message-id:references:to; bh=hjci2/Euax8jQaNPvqYAXPeDH9TiszyyE6+8TXghGqA=; b=JKajALdJNGeYNMTUlKWCwdVsnPaav6U9RG/UMEtxbIuee+n1R0S7XfEYtB4VNMiB1Z 6YLg2ofQAaCwNYv0v+iFwggKDm7aE1TMo1oFrQ7RxZInE8SXaA0FJNa2tNw8fhga0n62 aPsxVp0ASFpkEsXLnzTKyNNXaHisuiU/b3qzfojHvl3MSlWzGzIzDtgUwHwRLrFqWsAM FGPhkuqjw9Uck0uoHVs3Y6FPrIqAxOFmlRNDOP4uIvLOt+4jmuiidLDoHF7sRTHcSSKG H1Hhn6Jv9/FE+RIBewYAECjq9M5xrM6Loxih/gtDRflukeFkikoRa0JCoyvugNslTuiP yQOA== X-Received: by 10.66.136.167 with SMTP id qb7mr4950784pab.43.1382556654669; Wed, 23 Oct 2013 12:30:54 -0700 (PDT) Received: from briankrusicw.logan.tv ([64.17.255.138]) by mx.google.com with ESMTPSA id j9sm43193444paj.18.2013.10.23.12.30.53 for (version=TLSv1 cipher=ECDHE-RSA-RC4-SHA bits=128/128); Wed, 23 Oct 2013 12:30:53 -0700 (PDT) Subject: Re: FreeBSD, Centos and ZFS - SOLVED Mime-Version: 1.0 (Apple Message framework v1085) From: aurfalien In-Reply-To: Date: Wed, 23 Oct 2013 12:30:52 -0700 Message-Id: <66EE3A0C-8280-417C-99A7-5ADDC8E5E23D@gmail.com> References: <1381600575.18393.33219025.5D7B78D0@webmail.messagingengine.com> <8CA809B3-1692-4760-A63F-9D7451EB49BD@gmail.com> <525D143B.50202@gmail.com> <1382285088.2462.36238881.1AA6BB17@webmail.messagingengine.com> <25C055B9-4A19-46C8-B78E-AC8BD6E00CD9@gmail.com> <583CD1A7-A2F8-410E-B0C0-0C562833E024@gmail.com> To: Outback Dingo X-Mailer: Apple Mail (2.1085) Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: quoted-printable X-Content-Filtered-By: Mailman/MimeDel 2.1.14 Cc: freebsd-questions@freebsd.org X-BeenThere: freebsd-questions@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: User questions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Oct 2013 19:30:56 -0000 On Oct 23, 2013, at 12:11 PM, Outback Dingo wrote: >=20 >=20 >=20 > On Wed, Oct 23, 2013 at 1:53 PM, aurfalien = wrote: >=20 > On Oct 22, 2013, at 6:59 PM, Warren Block wrote: >=20 > > On Tue, 22 Oct 2013, aurfalien wrote: > > > >> > >> On Oct 20, 2013, at 9:04 AM, Mark Felder wrote: > >> > >>> On Sat, Oct 19, 2013, at 14:31, aurfalien wrote: > >>>> > >>>> ZoL comes with a little interesting feature; arc2 compression, = which does > >>>> seem to enhance performance all around. I disabled this in = CentOS to > >>>> level the playing field. > >>>> > >>> > >>> FreeBSD 10 also has L2ARC compression > >>> > >>> http://svnweb.freebsd.org/base?view=3Drevision&revision=3D251478 > >> > >> OMG, what a relief... the ~30% diff in performance was due to a = BIOS setting, in particular this one; > >> > >> Intel Turbo Boost Technology > >> Enhanced Intel SpeedStep Tech > >> > >> I disabled both. > > > > Enable them both and also enable powerd on FreeBSD in /etc/rc.conf = so it can take advantage of the first. > > > >> In fact, FreeBSD is showing a tad better now then CentOS. I have = files if any one is interested in viewing. > > > > With Turbo mode on, it may go even faster. >=20 > Yep, confirmed, got about ~1000 IOPS more by doing this (via fio). >=20 > So on to the other pots regarding tuning powerd. >=20 > - aurf >=20 > so curious what was the fio test on the command line....... =20 my fio file; [global] =20 description=3DEmulation of Intel IOmeter File Server Access Pattern =20 =20 [iometer] =20 rw=3Drandrw =20 rwmixread=3D75 =20 rwmixwrite=3D25 direct=3D0 =20 size=3D700g =20 ioengine=3Dposixaio fallocate=3Dnone runtime=3D900 random_distribution=3Dzipf:2.2 # IOMeter defines the server loads as the following: =20 # iodepth=3D1 Linear =20 # iodepth=3D4 Very Light =20 # iodepth=3D8 Light =20 # iodepth=3D64 Moderate =20 # iodepth=3D256 Heavy =20 iodepth=3D64 thread My results; iometer: (g=3D0): rw=3Drandrw, bs=3D4K-4K/4K-4K/4K-4K, = ioengine=3Dposixaio, iodepth=3D32 fio-2.1.1 Starting 1 thread Jobs: 1 (f=3D1): [m] [100.0% done] [102.4MB/34795KB/0KB /s] = [26.3K/8698/0 iops] [eta 00m:00s] iometer: (groupid=3D0, jobs=3D1): err=3D 0: pid=3D101195: Wed Oct 23 = 12:26:55 2013 Description : [Emulation of Intel IOmeter File Server Access Pattern] read : io=3D61266MB, bw=3D104561KB/s, iops=3D26140, runt=3D600001msec slat (usec): min=3D0, max=3D3643, avg=3D 7.73, stdev=3D13.20 clat (usec): min=3D7, max=3D707773, avg=3D707.63, stdev=3D1254.36 lat (usec): min=3D9, max=3D707777, avg=3D715.36, stdev=3D1254.22 clat percentiles (usec): | 1.00th=3D[ 32], 5.00th=3D[ 60], 10.00th=3D[ 93], = 20.00th=3D[ 149], | 30.00th=3D[ 213], 40.00th=3D[ 286], 50.00th=3D[ 394], = 60.00th=3D[ 548], | 70.00th=3D[ 788], 80.00th=3D[ 1128], 90.00th=3D[ 1720], = 95.00th=3D[ 2352], | 99.00th=3D[ 3824], 99.50th=3D[ 4512], 99.90th=3D[ 6240], = 99.95th=3D[ 7136], | 99.99th=3D[11968] bw (KB /s): min=3D 8755, max=3D112040, per=3D100.00%, = avg=3D104619.14, stdev=3D3609.80 write: io=3D20415MB, bw=3D34842KB/s, iops=3D8710, runt=3D600001msec slat (usec): min=3D1, max=3D3586, avg=3D 8.68, stdev=3D13.75 clat (usec): min=3D9, max=3D708187, avg=3D1436.86, stdev=3D2013.35 lat (usec): min=3D21, max=3D708192, avg=3D1445.54, stdev=3D2013.23 clat percentiles (usec): | 1.00th=3D[ 53], 5.00th=3D[ 96], 10.00th=3D[ 135], = 20.00th=3D[ 203], | 30.00th=3D[ 286], 40.00th=3D[ 418], 50.00th=3D[ 676], = 60.00th=3D[ 1080], | 70.00th=3D[ 1640], 80.00th=3D[ 2416], 90.00th=3D[ 3760], = 95.00th=3D[ 5088], | 99.00th=3D[ 8256], 99.50th=3D[ 9664], 99.90th=3D[12992], = 99.95th=3D[14528], | 99.99th=3D[19584] bw (KB /s): min=3D 2978, max=3D37584, per=3D100.00%, avg=3D34861.41, = stdev=3D1227.92 lat (usec) : 10=3D0.01%, 20=3D0.10%, 50=3D2.69%, 100=3D6.81%, = 250=3D23.36% lat (usec) : 500=3D20.88%, 750=3D10.72%, 1000=3D7.56% lat (msec) : 2=3D16.17%, 4=3D8.84%, 10=3D2.74%, 20=3D0.11%, 50=3D0.01%= lat (msec) : 100=3D0.01%, 250=3D0.01%, 750=3D0.01% cpu : usr=3D12.20%, sys=3D39.70%, ctx=3D10290014, majf=3D0, = minf=3D0 IO depths : 1=3D0.1%, 2=3D0.1%, 4=3D0.1%, 8=3D0.2%, 16=3D64.3%, = 32=3D35.5%, >=3D64=3D0.0% submit : 0=3D0.0%, 4=3D100.0%, 8=3D0.0%, 16=3D0.0%, 32=3D0.0%, = 64=3D0.0%, >=3D64=3D0.0% complete : 0=3D0.0%, 4=3D95.6%, 8=3D3.5%, 16=3D0.8%, 32=3D0.1%, = 64=3D0.0%, >=3D64=3D0.0% issued : total=3Dr=3D15684217/w=3D5226293/d=3D0, = short=3Dr=3D0/w=3D0/d=3D0 Run status group 0 (all jobs): READ: io=3D61266MB, aggrb=3D104561KB/s, minb=3D104561KB/s, = maxb=3D104561KB/s, mint=3D600001msec, maxt=3D600001msec WRITE: io=3D20415MB, aggrb=3D34841KB/s, minb=3D34841KB/s, = maxb=3D34841KB/s, mint=3D600001msec, maxt=3D600001msec - aurf=