From owner-freebsd-performance@FreeBSD.ORG Fri Mar 22 09:03:37 2013 Return-Path: Delivered-To: freebsd-performance@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by hub.freebsd.org (Postfix) with ESMTP id 78BDE8DD for ; Fri, 22 Mar 2013 09:03:37 +0000 (UTC) (envelope-from davide.damico@contactlab.com) Received: from mail2.shared.smtp.contactlab.it (mail2.shared.smtp.contactlab.it [93.94.37.7]) by mx1.freebsd.org (Postfix) with ESMTP id E92C9F95 for ; Fri, 22 Mar 2013 09:03:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; d=contactlab.it; s=clab1; c=relaxed/relaxed; q=dns/txt; i=@contactlab.it; t=1363943008; h=From:Subject:Date:To:MIME-Version:Content-Type; bh=33Ec4PVtCfSjIWVe2ofzodNqu7ovrjNMp/qt0dsf1qU=; b=pWaZHDRriYgDrEaQhhdUJlgz2o77KIpeWbBBl5779e04bBH/VT2Nuxhzy2GV7Dk3 JL+YCe+KYmnOkU0gaary2LQaEAmQ9poYnGuu0wv2lrE+BgvL7BGk/1Kx17jEQgHJ IT1lpXVYD+gyrIlbNDOn7UUemMS+BS35HkYyneVIbvw=; Received: from [213.92.90.12] ([213.92.90.12:53725] helo=mail3.tomato.it) by t.contactlab.it (envelope-from ) (ecelerity 3.5.1.37854 r(Momo-dev:3.5.1.0)) with ESMTP id D3/F9-24145-06E1C415; Fri, 22 Mar 2013 10:03:28 +0100 Received: from mx3-master.housing.tomato.lan ([172.16.7.55]) by mail3.tomato.it with smtp (Exim 4.80.1 (FreeBSD)) (envelope-from ) id 1UIxsm-000NYc-Kx for freebsd-performance@freebsd.org; Fri, 22 Mar 2013 10:03:28 +0100 Received: (qmail 90556 invoked by uid 89); 22 Mar 2013 09:03:28 -0000 Received: from localhost (HELO davepro.local) (127.0.0.1) by mx3-master.housing.tomato.lan with SMTP; 22 Mar 2013 09:03:28 -0000 Message-ID: <514C1E5F.8040504@contactlab.com> Date: Fri, 22 Mar 2013 10:03:27 +0100 From: Davide D'Amico User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:17.0) Gecko/20130307 Thunderbird/17.0.4 MIME-Version: 1.0 To: freebsd-performance@freebsd.org Subject: FreeBSD 9.1 vs CentOS 6.3 Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2013 09:03:37 -0000 Hi, I'm doing performance tests on a DELL R720, follows dmesg: Copyright (c) 1992-2012 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 9.1-RELEASE #0: Tue Mar 19 10:24:21 CET 2013 root:/usr/obj/usr/src/sys/R720 amd64 CPU: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz (2500.05-MHz K8-class CPU) Origin = "GenuineIntel" Id = 0x206d7 Family = 6 Model = 2d Stepping = 7 Features=0xbfebfbff Features2=0x1fbee3ff AMD Features=0x2c100800 AMD Features2=0x1 TSC: P-state invariant, performance statistics real memory = 34359738368 (32768 MB) avail memory = 33027436544 (31497 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs FreeBSD/SMP: 2 package(s) x 2 core(s) x 2 SMT threads cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 10 cpu3 (AP): APIC ID: 11 cpu4 (AP): APIC ID: 32 cpu5 (AP): APIC ID: 33 cpu6 (AP): APIC ID: 42 cpu7 (AP): APIC ID: 43 ioapic1: Changing APIC ID to 1 ioapic2: Changing APIC ID to 2 ioapic0 irqs 0-23 on motherboard ioapic1 irqs 32-55 on motherboard ioapic2 irqs 64-87 on motherboard kbd1 at kbdmux0 ctl: CAM Target Layer loaded acpi0: on motherboard acpi0: Power Button (fixed) acpi0: reservation of 0, 2000 (3) failed cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 cpu4: on acpi0 cpu5: on acpi0 cpu6: on acpi0 cpu7: on acpi0 atrtc0: port 0x70-0x7f irq 8 on acpi0 Event timer "RTC" frequency 32768 Hz quality 0 attimer0: port 0x40-0x5f irq 0 on acpi0 Timecounter "i8254" frequency 1193182 Hz quality 0 Event timer "i8254" frequency 1193182 Hz quality 100 Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 14318180 Hz quality 950 Event timer "HPET" frequency 14318180 Hz quality 550 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: irq 53 at device 1.0 on pci0 pci1: on pcib1 igb0: port 0xfc80-0xfc9f mem 0xdcb00000-0xdcbfffff,0xdcff0000-0xdcff3fff irq 38 at device 0.0 on pci1 igb0: Using MSIX interrupts with 9 vectors igb0: Ethernet address: bc:30:5b:ef:27:b8 [...] igb1: port 0xfca0-0xfcbf mem 0xdcc00000-0xdccfffff,0xdcff4000-0xdcff7fff irq 37 at device 0.1 on pci1 igb1: Using MSIX interrupts with 9 vectors igb1: Ethernet address: bc:30:5b:ef:27:b9 [...] igb2: port 0xfcc0-0xfcdf mem 0xdcd00000-0xdcdfffff,0xdcff8000-0xdcffbfff irq 36 at device 0.2 on pci1 igb2: Using MSIX interrupts with 9 vectors igb2: Ethernet address: bc:30:5b:ef:27:ba [...] igb3: port 0xfce0-0xfcff mem 0xdce00000-0xdcefffff,0xdcffc000-0xdcffffff irq 34 at device 0.3 on pci1 igb3: Using MSIX interrupts with 9 vectors igb3: Ethernet address: bc:30:5b:ef:27:bb [...] pcib2: irq 53 at device 2.0 on pci0 pci3: on pcib2 pcib3: irq 53 at device 2.2 on pci0 pci2: on pcib3 mfi0: port 0xec00-0xecff mem 0xddffc000-0xddffffff,0xddf80000-0xddfbffff irq 42 at device 0.0 on pci2 mfi0: Using MSI mfi0: Megaraid SAS driver Ver 4.23 mfi0: MaxCmd = 3f0 MaxSgl = 46 state = b73c03f0 pcib4: irq 53 at device 3.0 on pci0 pci4: on pcib4 pcib5: irq 53 at device 3.2 on pci0 pci5: on pcib5 pci0: at device 5.0 (no driver attached) pci0: at device 5.2 (no driver attached) pcib6: irq 16 at device 17.0 on pci0 pci6: on pcib6 pci0: at device 22.0 (no driver attached) pci0: at device 22.1 (no driver attached) ehci0: mem 0xdf8fe000-0xdf8fe3ff irq 23 at device 26.0 on pci0 usbus0: EHCI version 1.0 usbus0 on ehci0 pcib7: at device 28.0 on pci0 device_attach: pcib7 attach returned 6 pcib7: irq 19 at device 28.7 on pci0 pci8: on pcib7 pcib8: at device 0.0 on pci8 pci9: on pcib8 pcib9: at device 0.0 on pci9 pci10: on pcib9 pcib10: at device 0.0 on pci10 pci11: on pcib10 vgapci0: mem 0xd8000000-0xd8ffffff,0xdeffc000-0xdeffffff,0xde000000-0xde7fffff irq 19 at device 0.0 on pci11 pcib11: at device 1.0 on pci9 pci12: on pcib11 ehci1: mem 0xdf8ff000-0xdf8ff3ff irq 22 at device 29.0 on pci0 usbus1: EHCI version 1.0 usbus1 on ehci1 pcib12: at device 30.0 on pci0 pci13: on pcib12 isab0: at device 31.0 on pci0 isa0: on isab0 pcib13: on acpi0 pci64: on pcib13 pcib14: irq 85 at device 1.0 on pci64 pci65: on pcib14 pcib15: irq 85 at device 2.0 on pci64 pci66: on pcib15 pcib16: irq 85 at device 3.0 on pci64 pci67: on pcib16 pcib17: irq 85 at device 3.2 on pci64 pci68: on pcib17 pci64: at device 5.0 (no driver attached) pci64: at device 5.2 (no driver attached) pcib18: on acpi0 pci63: on pcib18 pci63: at device 8.0 (no driver attached) [...] pci63: at device 14.1 (no driver attached) [...] pci63: at device 19.1 (no driver attached) pci63: at device 19.4 (no driver attached) pci63: at device 19.5 (no driver attached) pci63: at device 19.6 (no driver attached) pcib19: on acpi0 pci127: on pcib19 [...] pci127: at device 14.1 (no driver attached) [...] pci127: at device 19.1 (no driver attached) pci127: at device 19.4 (no driver attached) pci127: at device 19.5 (no driver attached) pci127: at device 19.6 (no driver attached) uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 orm0: at iomem 0xc0000-0xc7fff,0xec000-0xeffff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 est0: on cpu0 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 226d00001e00 device_attach: est0 attach returned 6 p4tcc0: on cpu0 est1: on cpu1 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 226d00001e00 device_attach: est1 attach returned 6 p4tcc1: on cpu1 est2: on cpu2 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 226d00001e00 device_attach: est2 attach returned 6 p4tcc2: on cpu2 est3: on cpu3 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 224400001e00 device_attach: est3 attach returned 6 p4tcc3: on cpu3 est4: on cpu4 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 226d00001e00 device_attach: est4 attach returned 6 p4tcc4: on cpu4 est5: on cpu5 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 224400001e00 device_attach: est5 attach returned 6 p4tcc5: on cpu5 est6: on cpu6 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 224400001e00 device_attach: est6 attach returned 6 p4tcc6: on cpu6 est7: on cpu7 est: CPU supports Enhanced Speedstep, but is not recognized. est: cpu_vendor GenuineIntel, msr 224400001e00 device_attach: est7 attach returned 6 p4tcc7: on cpu7 mfi0: 6040 (417120269s/0x0020/info) - Shutdown command received from host mfi0: 6041 (boot + 4s/0x0020/info) - Firmware initialization started (PCI ID 005b/1000/1f34/1028) mfi0: 6042 (boot + 4s/0x0020/info) - Firmware version 3.130.05-1796 mfi0: 6043 (boot + 5s/0x0008/info) - Battery Present mfi0: 6044 (boot + 5s/0x0020/info) - Package version 21.1.0-0007 mfi0: 6045 (boot + 5s/0x0020/info) - Board Revision A01 mfi0: 6046 (boot + 6s/0x0008/info) - Battery temperature is normal mfi0: 6047 (boot + 6s/0x0008/info) - Current capacity of the battery is above threshold mfi0: 6048 (boot + 19s/0x0004/info) - Enclosure PD 20(c None/p1) communication restored mfi0: 6049 (boot + 19s/0x0002/info) - Inserted: Encl PD 20 mfi0: 6050 (boot + 19s/0x0002/info) - Inserted: PD 20(c None/p1) Info: enclPd=20, scsiType=d, portMap=00, sasAddr=5e4ae020a26cb500,0000000000000000 mfi0: 6051 (boot + 19s/0x0002/info) - Inserted: PD 00(e0x20/s0) mfi0: 6052 (boot + 19s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: enclPd=20, scsiType=0, portMap=04, sasAddr=5000c50054982339,0000000000000000 mfi0: 6053 (boot + 19s/0x0002/info) - Inserted: PD 01(e0x20/s1) mfi0: 6054 (boot + 19s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: enclPd=20, scsiType=0, portMap=05, sasAddr=5000c50054986c91,0000000000000000 mfi0: 6055 (boot + 19s/0x0002/info) - Inserted: PD 02(e0x20/s2) mfi0: 6056 (boot + 19s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: enclPd=20, scsiType=0, portMap=06, sasAddr=5000c50053da8f75,0000000000000000 mfi0: 6057 (boot + 19s/0x0002/info) - Inserted: PD 03(e0x20/s3) mfi0: 6058 (boot + 19s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: enclPd=20, scsiType=0, portMap=07, sasAddr=5000c50053c57eb5,0000000000000000 mfi0: 6059 (boot + 19s/0x0002/info) - Inserted: PD 04(e0x20/s4) mfi0: 6060 (boot + 19s/0x0002/info) - Inserted: PD 04(e0x20/s4) Info: enclPd=20, scsiType=0, portMap=00, sasAddr=5000c50053d364c5,0000000000000000 mfi0: 6061 (boot + 19s/0x0002/info) - Inserted: PD 05(e0x20/s5) mfi0: 6062 (boot + 19s/0x0002/info) - Inserted: PD 05(e0x20/s5) Info: enclPd=20, scsiType=0, portMap=03, sasAddr=5000c50053da2b41,0000000000000000 mfi0: 6063 (boot + 19s/0x0002/info) - Inserted: PD 06(e0x20/s6) mfi0: 6064 (boot + 19s/0x0002/info) - Inserted: PD 06(e0x20/s6) Info: enclPd=20, scsiType=0, portMap=02, sasAddr=4433221102000000,0000000000000000 mfi0: 6065 (boot + 19s/0x0002/WARN) - PD 06(e0x20/s6) is not a certified drive mfi0: 6066 (boot + 19s/0x0002/info) - Inserted: PD 07(e0x20/s7) mfi0: 6067 (boot + 19s/0x0002/info) - Inserted: PD 07(e0x20/s7) Info: enclPd=20, scsiType=0, portMap=01, sasAddr=4433221103000000,0000000000000000 mfi0: 6068 (boot + 19s/0x0002/WARN) - PD 07(e0x20/s7) is not a certified drive mfi0: 6069 (417120312s/0x0020/info) - Time established as 03/20/13 18:45:12; (32 seconds since power on) mfi0: 6070 (417120445s/0x0020/info) - Host driver is loaded and operational Timecounters tick every 1.000 msec usbus0: 480Mbps High Speed USB v2.0 usbus1: 480Mbps High Speed USB v2.0 mfid0 on mfi0 mfid0: 285568MB (584843264 sectors) RAID volume 'BASE' is optimal mfid1 on mfi0 mfid1: 121472MB (248774656 sectors) RAID volume 'L2ARC0' is optimal mfid2 on mfi0 mfid2: 121472MB (248774656 sectors) RAID volume 'ZIL0' is optimal mfid3 on mfi0 mfid3: 1143552MB (2341994496 sectors) RAID volume 'DATASTRIPE1' is optimal mfid4 on mfi0 mfid4: 1143552MB (2341994496 sectors) RAID volume 'DATASTRIPE2' is optimal ugen0.1: at usbus0 uhub0: on usbus0 ugen1.1: at usbus1 uhub1: on usbus1 uhub0: 2 ports with 2 removable, self powered uhub1: 2 ports with 2 removable, self powered ugen0.2: at usbus0 uhub2: on usbus0 ugen1.2: at usbus1 uhub3: on usbus1 uhub2: 6 ports with 6 removable, self powered uhub3: 8 ports with 8 removable, self powered ugen0.3: at usbus0 uhub4: on usbus0 SMP: AP CPU #1 Launched! SMP: AP CPU #7 Launched! SMP: AP CPU #3 Launched! SMP: AP CPU #6 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #5 Launched! SMP: AP CPU #4 Launched! Timecounter "TSC-low" frequency 9765827 Hz quality 1000 Root mount waiting for: usbus0 uhub4: 6 ports with 6 removable, self powered ugen0.4: at usbus0 ukbd0: on usbus0 kbd0 at ukbd0 Root mount waiting for: usbus0 Root mount waiting for: usbus0 ugen0.5: at usbus0 umass0: on usbus0 Trying to mount root from ufs:/dev/mfid0p3 [rw]... cd0 at umass-sim0 bus 0 scbus1 target 0 lun 0 cd0: Removable CD-ROM SCSI-0 device cd0: 40.000MB/s transfers cd0: Attempt to query device size failed: NOT READY, Medium not present da0 at umass-sim0 bus 0 scbus1 target 0 lun 1 da0: Removable Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: Attempt to query device size failed: NOT READY, Medium not present ZFS filesystem version 5 ZFS storage pool version 28 ums0: on usbus0 ums0: 3 buttons and [Z] coordinates ID=0 ums1: on usbus0 ums1: 3 buttons and [XYZ] coordinates ID=0 igb0: link state changed to UP With this loader.conf: vm.kmem_size_max="64G" vm.kmem_size="48G" vfs.zfs.arc_min="4096M" vfs.zfs.arc_max="15872M" vfs.zfs.txg.synctime="1" vfs.zfs.txg.timeout="5" vfs.zfs.write_limit_override="0" vfs.zfs.txg.write_limit_override="0" vfs.zfs.zil_disable="0" And with this sysctl.conf: security.bsd.see_other_uids=0 net.inet.tcp.blackhole=2 net.inet.udp.blackhole=1 net.inet.tcp.sendspace=65536 net.inet.tcp.recvspace=65536 net.link.ether.inet.log_arp_wrong_iface=0 kern.ipc.somaxconn=32768 kern.threads.max_threads_per_proc=16384 kern.maxfiles=262144 kern.maxfilesperproc=131072 kern.ipc.nmbclusters=65536 kern.corefile="/var/coredumps/%U.%N.%P.core" vfs.zfs.txg.timeout=5 vfs.zfs.l2arc_headroom=16 vfs.zfs.l2arc_write_max=26843546 vfs.zfs.l2arc_write_boost=26843546 vfs.zfs.vdev.min_pending=4 vfs.zfs.vdev.max_pending=64 vfs.zfs.l2arc_noprefetch=1 vfs.zfs.dedup.prefetch=0 vfs.zfs.prefetch_disable=1 I will use this server as a mysql-5.6 dbserver so I have a root partition using a hw raid1 and a /DATAZFS partition, follows configuration: root# mfiutil show drives mfi0 Physical Drives: 0 ( 279G) ONLINE SAS E1:S0 1 ( 279G) ONLINE SAS E1:S1 2 ( 558G) ONLINE SAS E1:S2 3 ( 558G) ONLINE SAS E1:S3 4 ( 558G) ONLINE SAS E1:S4 5 ( 558G) ONLINE SAS E1:S5 6 ( 119G) ONLINE SATA E1:S6 7 ( 119G) ONLINE SATA E1:S7 root# mfiutil show volumes mfi0 Volumes: Id Size Level Stripe State Cache Name mfid0 ( 278G) RAID-1 64k OPTIMAL Disabled mfid1 ( 118G) RAID-0 64k OPTIMAL Disabled mfid2 ( 118G) RAID-0 64k OPTIMAL Disabled mfid3 ( 1116G) RAID-1 64k OPTIMAL Disabled I have a /DATAZFS partition, too: root# zfs get all DATAZFS NAME PROPERTY VALUE SOURCE DATAZFS type filesystem - DATAZFS creation Thu Mar 21 10:45 2013 - DATAZFS used 54.2G - DATAZFS available 1.02T - DATAZFS referenced 54.2G - DATAZFS compressratio 1.00x - DATAZFS mounted yes - DATAZFS quota none default DATAZFS reservation none default DATAZFS recordsize 16K local DATAZFS mountpoint /DATAZFS default DATAZFS sharenfs off default DATAZFS checksum on default DATAZFS compression off default DATAZFS atime off local DATAZFS devices on default DATAZFS exec on default DATAZFS setuid on default DATAZFS readonly off default DATAZFS jailed off default DATAZFS snapdir hidden default DATAZFS aclmode discard default DATAZFS aclinherit restricted default DATAZFS canmount on default DATAZFS xattr off temporary DATAZFS copies 1 default DATAZFS version 5 - DATAZFS utf8only off - DATAZFS normalization none - DATAZFS casesensitivity sensitive - DATAZFS vscan off default DATAZFS nbmand off default DATAZFS sharesmb off default DATAZFS refquota none default DATAZFS refreservation none default DATAZFS primarycache all default DATAZFS secondarycache all default DATAZFS usedbysnapshots 0 - DATAZFS usedbydataset 54.2G - DATAZFS usedbychildren 1.41M - DATAZFS usedbyrefreservation 0 - DATAZFS logbias latency default DATAZFS dedup off local DATAZFS mlslabel - DATAZFS sync standard default DATAZFS refcompressratio 1.00x - DATAZFS written 54.2G - root# zpool status -v pool: DATAZFS state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM DATAZFS ONLINE 0 0 0 mfid3 ONLINE 0 0 0 logs mfid2 ONLINE 0 0 0 cache mfid1 ONLINE 0 0 0 errors: No known data errors root# mfiutil show firmware mfi0 Firmware Package Version: 21.1.0-0007 mfi0 Firmware Images: Name Version Date Time Status BIOS 5.31.01_4.12.05.00_0x05110000 6/20/2012 6/20/2012 active CTLR 4.00-0014 Aug 04 2011 12:49:17 active PCLI 05.00-03:#%00008 Feb 17 2011 14:03:12 active APP 3.130.05-1796 Jul 27 2012 15:58:31 active NVDT 2.1108.03-0094 Jun 22 2012 12:40:55 active BTBL 2.03.00.00-0004 Jul 27 2012 16:39:29 active BOOT 06.253.57.219 9/9/2010 15:32:25 active root# Then, I installed CentOS on the same hardware, same version of mysql (5.6.10-enterprise and 5.6.10-GA), and I tried an OLTP test (included in sysbench-0.5) executing 2000000 queries on 10000000 rows, with these results (transactions per seconds): BSD 9.1 amd64 ZFS (without tweaks): 1 thread: 2780 2 threads: 4371 4 threads: 6876 8 threads: 8202 12 threads: 8077 16 threads: 7780 24 threads: 7563 32 threads: 7632 48 threads: 7960 64 threads: 8062 BSD 9.1 amd64 ZFS (with tweaks): 1 thread: 2589 2 threads: 4679 4 threads: 6438 8 threads: 7073 12 threads: 7121 16 threads: 7227 24 threads: 7132 32 threads: 7273 48 threads: 7623 64 threads: 7672 CentOS 6.3 (ext4, without tweaks): 1 thread: 4163 2 threads: 7653 4 threads: 10905 8 threads: 12511 12 threads: 13556 16 threads: 14832 24 threads: 16270 32 threads: 16733 48 threads: 16925 64 threads: 16895 Well, any of you could ask "why zfs, please try ufs that is better": you are right, but changing ZFS with UFS doesn't change so much: http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016747.html So I hope I'm missing something because the performances of a standard CentOS installation on the same hardware performs much better than a 9.1 amd64 installation (I'm using the standard ULE scheduler included in the 9.1 release). Could anyone help me? Thanks, d.