Date: Fri, 22 Mar 2013 17:15:08 +0100 From: Davide D'Amico <davide.damico@contactlab.com> To: =?UTF-8?B?0JXQstCz0LXQvdC40Lkg0KXQvtGA0LrQuNC9?= <khorkin.eugeniy@gmail.com> Cc: freebsd-performance@freebsd.org Subject: Re: FreeBSD 9.1 vs CentOS 6.3 Message-ID: <ebfc033f-d8db-4162-86be-db013e3d7eb2@email.android.com> In-Reply-To: <CAF3xD3nDDUYB94TS9AUUQ=CPztB1S8mU4fRwNB5GupmK8MgwXg@mail.gmail.com> References: <CAF3xD3nDDUYB94TS9AUUQ=CPztB1S8mU4fRwNB5GupmK8MgwXg@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
Well, the I/O isn't the bottleneck (if you follow the link to freebsd-fs, you'll see iostats values) but it seems something related to cpu/scheduler or something else. Now I am trying vmware 5 on the same server and a vm with centos6: the vm outperforms freebsd with every concurrency from 1 to 48 threads in sysbench oltp tests. Thanks, d. "Евгений Хоркин" <khorkin.eugeniy@gmail.com> ha scritto: >Hi Davide! > >Are you sure that disk is the bottleneck in your test? >Does systat -vm 1 show 100% busy for disk ? > >Evgeny. > > >2013/3/22 Davide D'Amico <davide.damico@contactlab.com> > >> Hi, I'm doing performance tests on a DELL R720, follows dmesg: >> >> Copyright (c) 1992-2012 The FreeBSD Project. >> Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, >1994 >> The Regents of the University of California. All rights >reserved. >> FreeBSD is a registered trademark of The FreeBSD Foundation. >> FreeBSD 9.1-RELEASE #0: Tue Mar 19 10:24:21 CET 2013 >> root:/usr/obj/usr/src/sys/R720 amd64 >> CPU: Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz (2500.05-MHz K8-class >CPU) >> Origin = "GenuineIntel" Id = 0x206d7 Family = 6 Model = 2d >Stepping = >> 7 >> Features=0xbfebfbff<FPU,VME,**DE,PSE,TSC,MSR,PAE,MCE,CX8,** >> APIC,SEP,MTRR,PGE,MCA,CMOV,**PAT,PSE36,CLFLUSH,DTS,ACPI,** >> MMX,FXSR,SSE,SSE2,SS,HTT,TM,**PBE> >> Features2=0x1fbee3ff<SSE3,**PCLMULQDQ,DTES64,MON,DS_CPL,** >> VMX,SMX,EST,TM2,SSSE3,CX16,**xTPR,PDCM,PCID,DCA,SSE4.1,** >> SSE4.2,x2APIC,POPCNT,TSCDLT,**AESNI,XSAVE,OSXSAVE,AVX> >> AMD Features=0x2c100800<SYSCALL,**NX,Page1GB,RDTSCP,LM> >> AMD Features2=0x1<LAHF> >> TSC: P-state invariant, performance statistics >> real memory = 34359738368 (32768 MB) >> avail memory = 33027436544 (31497 MB) >> Event timer "LAPIC" quality 600 >> ACPI APIC Table: <DELL PE_SC3 > >> FreeBSD/SMP: Multiprocessor System Detected: 8 CPUs >> FreeBSD/SMP: 2 package(s) x 2 core(s) x 2 SMT threads >> cpu0 (BSP): APIC ID: 0 >> cpu1 (AP): APIC ID: 1 >> cpu2 (AP): APIC ID: 10 >> cpu3 (AP): APIC ID: 11 >> cpu4 (AP): APIC ID: 32 >> cpu5 (AP): APIC ID: 33 >> cpu6 (AP): APIC ID: 42 >> cpu7 (AP): APIC ID: 43 >> ioapic1: Changing APIC ID to 1 >> ioapic2: Changing APIC ID to 2 >> ioapic0 <Version 2.0> irqs 0-23 on motherboard >> ioapic1 <Version 2.0> irqs 32-55 on motherboard >> ioapic2 <Version 2.0> irqs 64-87 on motherboard >> kbd1 at kbdmux0 >> ctl: CAM Target Layer loaded >> acpi0: <DELL PE_SC3> on motherboard >> acpi0: Power Button (fixed) >> acpi0: reservation of 0, 2000 (3) failed >> cpu0: <ACPI CPU> on acpi0 >> cpu1: <ACPI CPU> on acpi0 >> cpu2: <ACPI CPU> on acpi0 >> cpu3: <ACPI CPU> on acpi0 >> cpu4: <ACPI CPU> on acpi0 >> cpu5: <ACPI CPU> on acpi0 >> cpu6: <ACPI CPU> on acpi0 >> cpu7: <ACPI CPU> on acpi0 >> atrtc0: <AT realtime clock> port 0x70-0x7f irq 8 on acpi0 >> Event timer "RTC" frequency 32768 Hz quality 0 >> attimer0: <AT timer> port 0x40-0x5f irq 0 on acpi0 >> Timecounter "i8254" frequency 1193182 Hz quality 0 >> Event timer "i8254" frequency 1193182 Hz quality 100 >> Timecounter "ACPI-fast" frequency 3579545 Hz quality 900 >> acpi_timer0: <24-bit timer at 3.579545MHz> port 0x808-0x80b on acpi0 >> hpet0: <High Precision Event Timer> iomem 0xfed00000-0xfed003ff on >acpi0 >> Timecounter "HPET" frequency 14318180 Hz quality 950 >> Event timer "HPET" frequency 14318180 Hz quality 550 >> pcib0: <ACPI Host-PCI bridge> port 0xcf8-0xcff on acpi0 >> pci0: <ACPI PCI bus> on pcib0 >> pcib1: <ACPI PCI-PCI bridge> irq 53 at device 1.0 on pci0 >> pci1: <ACPI PCI bus> on pcib1 >> igb0: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port >> 0xfc80-0xfc9f mem 0xdcb00000-0xdcbfffff,**0xdcff0000-0xdcff3fff irq >38 at >> device 0.0 on pci1 >> igb0: Using MSIX interrupts with 9 vectors >> igb0: Ethernet address: bc:30:5b:ef:27:b8 >> [...] >> igb1: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port >> 0xfca0-0xfcbf mem 0xdcc00000-0xdccfffff,**0xdcff4000-0xdcff7fff irq >37 at >> device 0.1 on pci1 >> igb1: Using MSIX interrupts with 9 vectors >> igb1: Ethernet address: bc:30:5b:ef:27:b9 >> [...] >> igb2: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port >> 0xfcc0-0xfcdf mem 0xdcd00000-0xdcdfffff,**0xdcff8000-0xdcffbfff irq >36 at >> device 0.2 on pci1 >> igb2: Using MSIX interrupts with 9 vectors >> igb2: Ethernet address: bc:30:5b:ef:27:ba >> [...] >> igb3: <Intel(R) PRO/1000 Network Connection version - 2.3.4> port >> 0xfce0-0xfcff mem 0xdce00000-0xdcefffff,**0xdcffc000-0xdcffffff irq >34 at >> device 0.3 on pci1 >> igb3: Using MSIX interrupts with 9 vectors >> igb3: Ethernet address: bc:30:5b:ef:27:bb >> [...] >> pcib2: <ACPI PCI-PCI bridge> irq 53 at device 2.0 on pci0 >> pci3: <ACPI PCI bus> on pcib2 >> pcib3: <ACPI PCI-PCI bridge> irq 53 at device 2.2 on pci0 >> pci2: <ACPI PCI bus> on pcib3 >> mfi0: <Dell PERC H710P Mini (monolithics)> port 0xec00-0xecff mem >> 0xddffc000-0xddffffff,**0xddf80000-0xddfbffff irq 42 at device 0.0 on >pci2 >> mfi0: Using MSI >> mfi0: Megaraid SAS driver Ver 4.23 >> mfi0: MaxCmd = 3f0 MaxSgl = 46 state = b73c03f0 >> pcib4: <ACPI PCI-PCI bridge> irq 53 at device 3.0 on pci0 >> pci4: <ACPI PCI bus> on pcib4 >> pcib5: <ACPI PCI-PCI bridge> irq 53 at device 3.2 on pci0 >> pci5: <ACPI PCI bus> on pcib5 >> pci0: <base peripheral> at device 5.0 (no driver attached) >> pci0: <base peripheral> at device 5.2 (no driver attached) >> pcib6: <PCI-PCI bridge> irq 16 at device 17.0 on pci0 >> pci6: <PCI bus> on pcib6 >> pci0: <simple comms> at device 22.0 (no driver attached) >> pci0: <simple comms> at device 22.1 (no driver attached) >> ehci0: <EHCI (generic) USB 2.0 controller> mem 0xdf8fe000-0xdf8fe3ff >irq >> 23 at device 26.0 on pci0 >> usbus0: EHCI version 1.0 >> usbus0 on ehci0 >> pcib7: <ACPI PCI-PCI bridge> at device 28.0 on pci0 >> device_attach: pcib7 attach returned 6 >> pcib7: <ACPI PCI-PCI bridge> irq 19 at device 28.7 on pci0 >> pci8: <ACPI PCI bus> on pcib7 >> pcib8: <PCI-PCI bridge> at device 0.0 on pci8 >> pci9: <PCI bus> on pcib8 >> pcib9: <PCI-PCI bridge> at device 0.0 on pci9 >> pci10: <PCI bus> on pcib9 >> pcib10: <PCI-PCI bridge> at device 0.0 on pci10 >> pci11: <PCI bus> on pcib10 >> vgapci0: <VGA-compatible display> mem 0xd8000000-0xd8ffffff,** >> 0xdeffc000-0xdeffffff,**0xde000000-0xde7fffff irq 19 at device 0.0 on >> pci11 >> pcib11: <PCI-PCI bridge> at device 1.0 on pci9 >> pci12: <PCI bus> on pcib11 >> ehci1: <EHCI (generic) USB 2.0 controller> mem 0xdf8ff000-0xdf8ff3ff >irq >> 22 at device 29.0 on pci0 >> usbus1: EHCI version 1.0 >> usbus1 on ehci1 >> pcib12: <PCI-PCI bridge> at device 30.0 on pci0 >> pci13: <PCI bus> on pcib12 >> isab0: <PCI-ISA bridge> at device 31.0 on pci0 >> isa0: <ISA bus> on isab0 >> pcib13: <ACPI Host-PCI bridge> on acpi0 >> pci64: <ACPI PCI bus> on pcib13 >> pcib14: <ACPI PCI-PCI bridge> irq 85 at device 1.0 on pci64 >> pci65: <ACPI PCI bus> on pcib14 >> pcib15: <ACPI PCI-PCI bridge> irq 85 at device 2.0 on pci64 >> pci66: <ACPI PCI bus> on pcib15 >> pcib16: <ACPI PCI-PCI bridge> irq 85 at device 3.0 on pci64 >> pci67: <ACPI PCI bus> on pcib16 >> pcib17: <ACPI PCI-PCI bridge> irq 85 at device 3.2 on pci64 >> pci68: <ACPI PCI bus> on pcib17 >> pci64: <base peripheral> at device 5.0 (no driver attached) >> pci64: <base peripheral> at device 5.2 (no driver attached) >> pcib18: <ACPI Host-PCI bridge> on acpi0 >> pci63: <ACPI PCI bus> on pcib18 >> pci63: <base peripheral> at device 8.0 (no driver attached) >> [...] >> pci63: <dasp> at device 14.1 (no driver attached) >> [...] >> pci63: <dasp> at device 19.1 (no driver attached) >> pci63: <dasp> at device 19.4 (no driver attached) >> pci63: <dasp> at device 19.5 (no driver attached) >> pci63: <base peripheral> at device 19.6 (no driver attached) >> pcib19: <ACPI Host-PCI bridge> on acpi0 >> pci127: <ACPI PCI bus> on pcib19 >> [...] >> pci127: <dasp> at device 14.1 (no driver attached) >> [...] >> pci127: <dasp> at device 19.1 (no driver attached) >> pci127: <dasp> at device 19.4 (no driver attached) >> pci127: <dasp> at device 19.5 (no driver attached) >> pci127: <base peripheral> at device 19.6 (no driver attached) >> uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 >> uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on >acpi0 >> orm0: <ISA Option ROMs> at iomem 0xc0000-0xc7fff,0xec000-**0xeffff on >isa0 >> sc0: <System console> at flags 0x100 on isa0 >> sc0: VGA <16 virtual consoles, flags=0x300> >> vga0: <Generic ISA VGA> at port 0x3c0-0x3df iomem 0xa0000-0xbffff on >isa0 >> est0: <Enhanced SpeedStep Frequency Control> on cpu0 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 226d00001e00 >> device_attach: est0 attach returned 6 >> p4tcc0: <CPU Frequency Thermal Control> on cpu0 >> est1: <Enhanced SpeedStep Frequency Control> on cpu1 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 226d00001e00 >> device_attach: est1 attach returned 6 >> p4tcc1: <CPU Frequency Thermal Control> on cpu1 >> est2: <Enhanced SpeedStep Frequency Control> on cpu2 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 226d00001e00 >> device_attach: est2 attach returned 6 >> p4tcc2: <CPU Frequency Thermal Control> on cpu2 >> est3: <Enhanced SpeedStep Frequency Control> on cpu3 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 224400001e00 >> device_attach: est3 attach returned 6 >> p4tcc3: <CPU Frequency Thermal Control> on cpu3 >> est4: <Enhanced SpeedStep Frequency Control> on cpu4 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 226d00001e00 >> device_attach: est4 attach returned 6 >> p4tcc4: <CPU Frequency Thermal Control> on cpu4 >> est5: <Enhanced SpeedStep Frequency Control> on cpu5 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 224400001e00 >> device_attach: est5 attach returned 6 >> p4tcc5: <CPU Frequency Thermal Control> on cpu5 >> est6: <Enhanced SpeedStep Frequency Control> on cpu6 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 224400001e00 >> device_attach: est6 attach returned 6 >> p4tcc6: <CPU Frequency Thermal Control> on cpu6 >> est7: <Enhanced SpeedStep Frequency Control> on cpu7 >> est: CPU supports Enhanced Speedstep, but is not recognized. >> est: cpu_vendor GenuineIntel, msr 224400001e00 >> device_attach: est7 attach returned 6 >> p4tcc7: <CPU Frequency Thermal Control> on cpu7 >> mfi0: 6040 (417120269s/0x0020/info) - Shutdown command received from >host >> mfi0: 6041 (boot + 4s/0x0020/info) - Firmware initialization started >(PCI >> ID 005b/1000/1f34/1028) >> mfi0: 6042 (boot + 4s/0x0020/info) - Firmware version 3.130.05-1796 >> mfi0: 6043 (boot + 5s/0x0008/info) - Battery Present >> mfi0: 6044 (boot + 5s/0x0020/info) - Package version 21.1.0-0007 >> mfi0: 6045 (boot + 5s/0x0020/info) - Board Revision A01 >> mfi0: 6046 (boot + 6s/0x0008/info) - Battery temperature is normal >> mfi0: 6047 (boot + 6s/0x0008/info) - Current capacity of the battery >is >> above threshold >> mfi0: 6048 (boot + 19s/0x0004/info) - Enclosure PD 20(c None/p1) >> communication restored >> mfi0: 6049 (boot + 19s/0x0002/info) - Inserted: Encl PD 20 >> mfi0: 6050 (boot + 19s/0x0002/info) - Inserted: PD 20(c None/p1) >Info: >> enclPd=20, scsiType=d, portMap=00, sasAddr=5e4ae020a26cb500,** >> 0000000000000000 >> mfi0: 6051 (boot + 19s/0x0002/info) - Inserted: PD 00(e0x20/s0) >> mfi0: 6052 (boot + 19s/0x0002/info) - Inserted: PD 00(e0x20/s0) Info: >> enclPd=20, scsiType=0, portMap=04, sasAddr=5000c50054982339,** >> 0000000000000000 >> mfi0: 6053 (boot + 19s/0x0002/info) - Inserted: PD 01(e0x20/s1) >> mfi0: 6054 (boot + 19s/0x0002/info) - Inserted: PD 01(e0x20/s1) Info: >> enclPd=20, scsiType=0, portMap=05, sasAddr=5000c50054986c91,** >> 0000000000000000 >> mfi0: 6055 (boot + 19s/0x0002/info) - Inserted: PD 02(e0x20/s2) >> mfi0: 6056 (boot + 19s/0x0002/info) - Inserted: PD 02(e0x20/s2) Info: >> enclPd=20, scsiType=0, portMap=06, sasAddr=5000c50053da8f75,** >> 0000000000000000 >> mfi0: 6057 (boot + 19s/0x0002/info) - Inserted: PD 03(e0x20/s3) >> mfi0: 6058 (boot + 19s/0x0002/info) - Inserted: PD 03(e0x20/s3) Info: >> enclPd=20, scsiType=0, portMap=07, sasAddr=5000c50053c57eb5,** >> 0000000000000000 >> mfi0: 6059 (boot + 19s/0x0002/info) - Inserted: PD 04(e0x20/s4) >> mfi0: 6060 (boot + 19s/0x0002/info) - Inserted: PD 04(e0x20/s4) Info: >> enclPd=20, scsiType=0, portMap=00, sasAddr=5000c50053d364c5,** >> 0000000000000000 >> mfi0: 6061 (boot + 19s/0x0002/info) - Inserted: PD 05(e0x20/s5) >> mfi0: 6062 (boot + 19s/0x0002/info) - Inserted: PD 05(e0x20/s5) Info: >> enclPd=20, scsiType=0, portMap=03, sasAddr=5000c50053da2b41,** >> 0000000000000000 >> mfi0: 6063 (boot + 19s/0x0002/info) - Inserted: PD 06(e0x20/s6) >> mfi0: 6064 (boot + 19s/0x0002/info) - Inserted: PD 06(e0x20/s6) Info: >> enclPd=20, scsiType=0, portMap=02, sasAddr=4433221102000000,** >> 0000000000000000 >> mfi0: 6065 (boot + 19s/0x0002/WARN) - PD 06(e0x20/s6) is not a >certified >> drive >> mfi0: 6066 (boot + 19s/0x0002/info) - Inserted: PD 07(e0x20/s7) >> mfi0: 6067 (boot + 19s/0x0002/info) - Inserted: PD 07(e0x20/s7) Info: >> enclPd=20, scsiType=0, portMap=01, sasAddr=4433221103000000,** >> 0000000000000000 >> mfi0: 6068 (boot + 19s/0x0002/WARN) - PD 07(e0x20/s7) is not a >certified >> drive >> mfi0: 6069 (417120312s/0x0020/info) - Time established as 03/20/13 >> 18:45:12; (32 seconds since power on) >> mfi0: 6070 (417120445s/0x0020/info) - Host driver is loaded and >operational >> Timecounters tick every 1.000 msec >> usbus0: 480Mbps High Speed USB v2.0 >> usbus1: 480Mbps High Speed USB v2.0 >> mfid0 on mfi0 >> mfid0: 285568MB (584843264 sectors) RAID volume 'BASE' is optimal >> mfid1 on mfi0 >> mfid1: 121472MB (248774656 sectors) RAID volume 'L2ARC0' is optimal >> mfid2 on mfi0 >> mfid2: 121472MB (248774656 sectors) RAID volume 'ZIL0' is optimal >> mfid3 on mfi0 >> mfid3: 1143552MB (2341994496 sectors) RAID volume 'DATASTRIPE1' is >optimal >> mfid4 on mfi0 >> mfid4: 1143552MB (2341994496 sectors) RAID volume 'DATASTRIPE2' is >optimal >> ugen0.1: <Intel> at usbus0 >> uhub0: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on >usbus0 >> ugen1.1: <Intel> at usbus1 >> uhub1: <Intel EHCI root HUB, class 9/0, rev 2.00/1.00, addr 1> on >usbus1 >> uhub0: 2 ports with 2 removable, self powered >> uhub1: 2 ports with 2 removable, self powered >> ugen0.2: <vendor 0x8087> at usbus0 >> uhub2: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr >2> on >> usbus0 >> ugen1.2: <vendor 0x8087> at usbus1 >> uhub3: <vendor 0x8087 product 0x0024, class 9/0, rev 2.00/0.00, addr >2> on >> usbus1 >> uhub2: 6 ports with 6 removable, self powered >> uhub3: 8 ports with 8 removable, self powered >> ugen0.3: <no manufacturer> at usbus0 >> uhub4: <no manufacturer Gadget USB HUB, class 9/0, rev 2.00/0.00, >addr 3> >> on usbus0 >> SMP: AP CPU #1 Launched! >> SMP: AP CPU #7 Launched! >> SMP: AP CPU #3 Launched! >> SMP: AP CPU #6 Launched! >> SMP: AP CPU #2 Launched! >> SMP: AP CPU #5 Launched! >> SMP: AP CPU #4 Launched! >> Timecounter "TSC-low" frequency 9765827 Hz quality 1000 >> Root mount waiting for: usbus0 >> uhub4: 6 ports with 6 removable, self powered >> ugen0.4: <Avocent> at usbus0 >> ukbd0: <Keyboard> on usbus0 >> kbd0 at ukbd0 >> Root mount waiting for: usbus0 >> Root mount waiting for: usbus0 >> ugen0.5: <Avocent> at usbus0 >> umass0: <SCSI Transparent Interface 0> on usbus0 >> Trying to mount root from ufs:/dev/mfid0p3 [rw]... >> cd0 at umass-sim0 bus 0 scbus1 target 0 lun 0 >> cd0: <iDRAC Virtual CD 0329> Removable CD-ROM SCSI-0 device >> cd0: 40.000MB/s transfers >> cd0: Attempt to query device size failed: NOT READY, Medium not >present >> da0 at umass-sim0 bus 0 scbus1 target 0 lun 1 >> da0: <iDRAC Virtual Floppy 0329> Removable Direct Access SCSI-0 >device >> da0: 40.000MB/s transfers >> da0: Attempt to query device size failed: NOT READY, Medium not >present >> ZFS filesystem version 5 >> ZFS storage pool version 28 >> ums0: <Mouse> on usbus0 >> ums0: 3 buttons and [Z] coordinates ID=0 >> ums1: <Mouse REL> on usbus0 >> ums1: 3 buttons and [XYZ] coordinates ID=0 >> igb0: link state changed to UP >> >> With this loader.conf: >> vm.kmem_size_max="64G" >> vm.kmem_size="48G" >> vfs.zfs.arc_min="4096M" >> vfs.zfs.arc_max="15872M" >> vfs.zfs.txg.synctime="1" >> vfs.zfs.txg.timeout="5" >> vfs.zfs.write_limit_override="**0" >> vfs.zfs.txg.write_limit_**override="0" >> vfs.zfs.zil_disable="0" >> >> And with this sysctl.conf: >> security.bsd.see_other_uids=0 >> net.inet.tcp.blackhole=2 >> net.inet.udp.blackhole=1 >> net.inet.tcp.sendspace=65536 >> net.inet.tcp.recvspace=65536 >> net.link.ether.inet.log_arp_**wrong_iface=0 >> kern.ipc.somaxconn=32768 >> kern.threads.max_threads_per_**proc=16384 >> kern.maxfiles=262144 >> kern.maxfilesperproc=131072 >> kern.ipc.nmbclusters=65536 >> kern.corefile="/var/coredumps/**%U.%N.%P.core" >> >> vfs.zfs.txg.timeout=5 >> vfs.zfs.l2arc_headroom=16 >> vfs.zfs.l2arc_write_max=**26843546 >> vfs.zfs.l2arc_write_boost=**26843546 >> vfs.zfs.vdev.min_pending=4 >> vfs.zfs.vdev.max_pending=64 >> vfs.zfs.l2arc_noprefetch=1 >> vfs.zfs.dedup.prefetch=0 >> vfs.zfs.prefetch_disable=1 >> >> I will use this server as a mysql-5.6 dbserver so I have a root >partition >> using a hw raid1 and a /DATAZFS partition, follows configuration: >> >> root# mfiutil show drives >> mfi0 Physical Drives: >> 0 ( 279G) ONLINE <SEAGATE ST3300657SS ES65> SAS E1:S0 >> 1 ( 279G) ONLINE <SEAGATE ST3300657SS ES65> SAS E1:S1 >> 2 ( 558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S2 >> 3 ( 558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S3 >> 4 ( 558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S4 >> 5 ( 558G) ONLINE <SEAGATE ST3600057SS ES65> SAS E1:S5 >> 6 ( 119G) ONLINE <OCZ-VERTEX4 1.4> SATA E1:S6 >> 7 ( 119G) ONLINE <OCZ-VERTEX4 1.4> SATA E1:S7 >> root# mfiutil show volumes >> mfi0 Volumes: >> Id Size Level Stripe State Cache Name >> mfid0 ( 278G) RAID-1 64k OPTIMAL Disabled <BASE> >> mfid1 ( 118G) RAID-0 64k OPTIMAL Disabled <L2ARC0> >> mfid2 ( 118G) RAID-0 64k OPTIMAL Disabled <ZIL0> >> mfid3 ( 1116G) RAID-1 64k OPTIMAL Disabled <DATA> >> >> I have a /DATAZFS partition, too: >> root# zfs get all DATAZFS >> NAME PROPERTY VALUE SOURCE >> DATAZFS type filesystem - >> DATAZFS creation Thu Mar 21 10:45 2013 - >> DATAZFS used 54.2G - >> DATAZFS available 1.02T - >> DATAZFS referenced 54.2G - >> DATAZFS compressratio 1.00x - >> DATAZFS mounted yes - >> DATAZFS quota none default >> DATAZFS reservation none default >> DATAZFS recordsize 16K local >> DATAZFS mountpoint /DATAZFS default >> DATAZFS sharenfs off default >> DATAZFS checksum on default >> DATAZFS compression off default >> DATAZFS atime off local >> DATAZFS devices on default >> DATAZFS exec on default >> DATAZFS setuid on default >> DATAZFS readonly off default >> DATAZFS jailed off default >> DATAZFS snapdir hidden default >> DATAZFS aclmode discard default >> DATAZFS aclinherit restricted default >> DATAZFS canmount on default >> DATAZFS xattr off temporary >> DATAZFS copies 1 default >> DATAZFS version 5 - >> DATAZFS utf8only off - >> DATAZFS normalization none - >> DATAZFS casesensitivity sensitive - >> DATAZFS vscan off default >> DATAZFS nbmand off default >> DATAZFS sharesmb off default >> DATAZFS refquota none default >> DATAZFS refreservation none default >> DATAZFS primarycache all default >> DATAZFS secondarycache all default >> DATAZFS usedbysnapshots 0 - >> DATAZFS usedbydataset 54.2G - >> DATAZFS usedbychildren 1.41M - >> DATAZFS usedbyrefreservation 0 - >> DATAZFS logbias latency default >> DATAZFS dedup off local >> DATAZFS mlslabel - >> DATAZFS sync standard default >> DATAZFS refcompressratio 1.00x - >> DATAZFS written 54.2G - >> >> root# zpool status -v >> pool: DATAZFS >> state: ONLINE >> scan: none requested >> config: >> >> NAME STATE READ WRITE CKSUM >> DATAZFS ONLINE 0 0 0 >> mfid3 ONLINE 0 0 0 >> logs >> mfid2 ONLINE 0 0 0 >> cache >> mfid1 ONLINE 0 0 0 >> >> errors: No known data errors >> >> root# mfiutil show firmware >> mfi0 Firmware Package Version: 21.1.0-0007 >> mfi0 Firmware Images: >> Name Version Date Time Status >> BIOS 5.31.01_4.12.05.00_0x05110000 6/20/2012 >> 6/20/2012 >> active >> CTLR 4.00-0014 Aug 04 2011 12:49:17 active >> PCLI 05.00-03:#%00008 Feb 17 2011 14:03:12 active >> APP 3.130.05-1796 Jul 27 2012 15:58:31 active >> NVDT 2.1108.03-0094 Jun 22 2012 12:40:55 active >> BTBL 2.03.00.00-0004 Jul 27 2012 16:39:29 active >> BOOT 06.253.57.219 9/9/2010 15:32:25 active >> root# >> >> Then, I installed CentOS on the same hardware, same version of mysql >> (5.6.10-enterprise and 5.6.10-GA), and I tried an OLTP test (included >in >> sysbench-0.5) executing 2000000 queries on 10000000 rows, with these >> results (transactions per seconds): >> >> BSD 9.1 amd64 ZFS (without tweaks): >> 1 thread: 2780 >> 2 threads: 4371 >> 4 threads: 6876 >> 8 threads: 8202 >> 12 threads: 8077 >> 16 threads: 7780 >> 24 threads: 7563 >> 32 threads: 7632 >> 48 threads: 7960 >> 64 threads: 8062 >> >> BSD 9.1 amd64 ZFS (with tweaks): >> 1 thread: 2589 >> 2 threads: 4679 >> 4 threads: 6438 >> 8 threads: 7073 >> 12 threads: 7121 >> 16 threads: 7227 >> 24 threads: 7132 >> 32 threads: 7273 >> 48 threads: 7623 >> 64 threads: 7672 >> >> CentOS 6.3 (ext4, without tweaks): >> 1 thread: 4163 >> 2 threads: 7653 >> 4 threads: 10905 >> 8 threads: 12511 >> 12 threads: 13556 >> 16 threads: 14832 >> 24 threads: 16270 >> 32 threads: 16733 >> 48 threads: 16925 >> 64 threads: 16895 >> >> >> Well, any of you could ask "why zfs, please try ufs that is better": >you >> are right, but changing ZFS with UFS doesn't change so much: >> >http://lists.freebsd.org/**pipermail/freebsd-fs/2013-**March/016747.html<http://lists.freebsd.org/pipermail/freebsd-fs/2013-March/016747.html>; >> >> So I hope I'm missing something because the performances of a >standard >> CentOS installation on the same hardware performs much better than a >9.1 >> amd64 installation (I'm using the standard ULE scheduler included in >the >> 9.1 release). >> >> Could anyone help me? >> >> Thanks, >> d. >> ______________________________**_________________ >> freebsd-performance@freebsd.**org ><freebsd-performance@freebsd.org>mailing list >> >http://lists.freebsd.org/**mailman/listinfo/freebsd-**performance<http://lists.freebsd.org/mailman/listinfo/freebsd-performance>; >> To unsubscribe, send any mail to "freebsd-performance-** >> unsubscribe@freebsd.org ><freebsd-performance-unsubscribe@freebsd.org>" >> -- d.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?ebfc033f-d8db-4162-86be-db013e3d7eb2>