From owner-freebsd-fs@FreeBSD.ORG Mon Mar 18 15:31:32 2013 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.FreeBSD.org [8.8.178.115]) by hub.freebsd.org (Postfix) with ESMTP id BAF8E8EB for ; Mon, 18 Mar 2013 15:31:32 +0000 (UTC) (envelope-from prvs=17892983bb=killing@multiplay.co.uk) Received: from mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) by mx1.freebsd.org (Postfix) with ESMTP id 60BD22C5 for ; Mon, 18 Mar 2013 15:31:32 +0000 (UTC) Received: from r2d2 ([46.65.172.4]) by mail1.multiplay.co.uk (mail1.multiplay.co.uk [85.236.96.23]) (MDaemon PRO v10.0.4) with ESMTP id md50002792921.msg for ; Mon, 18 Mar 2013 15:31:31 +0000 X-Spam-Processed: mail1.multiplay.co.uk, Mon, 18 Mar 2013 15:31:31 +0000 (not processed: message from valid local sender) X-MDDKIM-Result: neutral (mail1.multiplay.co.uk) X-MDRemoteIP: 46.65.172.4 X-Return-Path: prvs=17892983bb=killing@multiplay.co.uk X-Envelope-From: killing@multiplay.co.uk X-MDaemon-Deliver-To: freebsd-fs@freebsd.org Message-ID: <810E5C08C2D149DBAC94E30678234995@multiplay.co.uk> From: "Steven Hartland" To: "Davide D'Amico" , References: <514729BD.2000608@contactlab.com> Subject: Re: FreBSD 9.1 and ZFS v28 performances Date: Mon, 18 Mar 2013 15:31:51 -0000 MIME-Version: 1.0 Content-Type: text/plain; format=flowed; charset="iso-8859-1"; reply-type=response Content-Transfer-Encoding: 7bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2900.5931 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2900.6157 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 18 Mar 2013 15:31:32 -0000 ----- Original Message ----- From: "Davide D'Amico" To: Sent: Monday, March 18, 2013 2:50 PM Subject: FreBSD 9.1 and ZFS v28 performances > Hi all, > I'm trying to use ZFS on a DELL R720 with 2x6-core, 32GB ram, H710 > controller (no JBOD) and 15K rpm SAS HD: I will use it for a mysql 5.6 > server, so I am trying to use ZFS to get L2ARC and ZIL benefits. > > I created a RAID10 and used zpool to create a pool on top: > > # zpool create DATA mfid3 > # zpool add DATA cache mfid1 log mfid2 > > I have a question on zfs performances. Using: > > dd if=/dev/zero of=file.out bs=16k count=1M > > I cannot go faster than 400MB/s so I think I'm missing something; I > tried removing zil, removing l2arc but everything is still the same. > > Here my configuration details: > > OS: FreeBSD 9.1 amd64 GENERIC > > /boot/loader.conf > vfs.zfs.arc_min="4096M" > vfs.zfs.arc_max="15872M" > vm.kmem_size_max="64G" > vm.kmem_size="49152M" > vfs.zfs.write_limit_override=1073741824 > > /etc/sysctl.conf: > kern.ipc.somaxconn=32768 > kern.threads.max_threads_per_proc=16384 > kern.maxfiles=262144 > kern.maxfilesperproc=131072 > kern.ipc.nmbclusters=65536 > kern.corefile="/var/coredumps/%U.%N.%P.core" > vfs.zfs.prefetch_disable="1" > kern.maxvnodes=250000 > > mfiutil show volumes: > mfi0 Volumes: > Id Size Level Stripe State Cache Name > mfid0 ( 278G) RAID-1 64k OPTIMAL Disabled > mfid1 ( 118G) RAID-0 64k OPTIMAL Disabled > mfid2 ( 118G) RAID-0 64k OPTIMAL Disabled > mfid3 ( 1116G) RAID-10 64k OPTIMAL Disabled > > zpool status: > pool: DATA > state: ONLINE > scan: none requested > config: > > NAME STATE READ WRITE CKSUM > DATA ONLINE 0 0 0 > mfid3 ONLINE 0 0 0 > logs > mfid2 ONLINE 0 0 0 > cache > mfid1 ONLINE 0 0 0 > > errors: No known data errors > > zfs get all DATA > NAME PROPERTY VALUE SOURCE > DATA type filesystem - > DATA creation Mon Mar 18 13:41 2013 - > DATA used 53.0G - > DATA available 1.02T - > DATA referenced 53.0G - > DATA compressratio 1.00x - > DATA mounted yes - > DATA quota none default > DATA reservation none default > DATA recordsize 16K local > DATA mountpoint /DATA default > DATA sharenfs off default > DATA checksum on default > DATA compression off default > DATA atime off local > DATA devices on default > DATA exec on default > DATA setuid on default > DATA readonly off default > DATA jailed off default > DATA snapdir hidden default > DATA aclmode discard default > DATA aclinherit restricted default > DATA canmount on default > DATA xattr off temporary > DATA copies 1 default > DATA version 5 - > DATA utf8only off - > DATA normalization none - > DATA casesensitivity sensitive - > DATA vscan off default > DATA nbmand off default > DATA sharesmb off default > DATA refquota none default > DATA refreservation none default > DATA primarycache metadata local > DATA secondarycache all default > DATA usedbysnapshots 0 - > DATA usedbydataset 53.0G - > DATA usedbychildren 242K - > DATA usedbyrefreservation 0 - > DATA logbias latency default > DATA dedup off default > DATA mlslabel - > DATA sync standard default > DATA refcompressratio 1.00x - > DATA written 53.0G - > DATA zfs:zfs_nocacheflush 1 local > > > I'm using recordsize=16k because of mysql. > > I am trying to use sysbench (0.5, not in the ports yet) with oltp test > suite and my performances not so good. First off ideally you shouldn't use RAID controllers for ZFS, let it have the raw disks and use a JBOD controller e.g. mps not a HW RAID controller like mfi. HEAD has some significant changes for the mfi driver specifically:- http://svnweb.freebsd.org/base?view=revision&revision=247369 This fixes lots off bugs but also enables full queue support on TBOLT cards so if your mfi is a TBOLT card you may see some speed up in random IO, not that this would effect your test here. While having a separate ZIL disk is good, your benefits may well be limited if said disk is a traditional HD, better to look at enterprise SSD's for this. The same and them some applies to your L2ARC disks. Regards Steve ================================================ This e.mail is private and confidential between Multiplay (UK) Ltd. and the person or entity to whom it is addressed. In the event of misdirection, the recipient is prohibited from using, copying, printing or otherwise disseminating it or any information contained in it. In the event of misdirection, illegible or incomplete transmission please telephone +44 845 868 1337 or return the E.mail to postmaster@multiplay.co.uk.