From owner-freebsd-stable@FreeBSD.ORG Sat Aug 7 01:21:22 2010 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 310731065673 for ; Sat, 7 Aug 2010 01:21:22 +0000 (UTC) (envelope-from boydjd@jbip.net) Received: from mail-fx0-f54.google.com (mail-fx0-f54.google.com [209.85.161.54]) by mx1.freebsd.org (Postfix) with ESMTP id 884288FC0A for ; Sat, 7 Aug 2010 01:21:21 +0000 (UTC) Received: by fxm13 with SMTP id 13so4660601fxm.13 for ; Fri, 06 Aug 2010 18:21:20 -0700 (PDT) Received: by 10.223.122.194 with SMTP id m2mr13611078far.85.1281144080097; Fri, 06 Aug 2010 18:21:20 -0700 (PDT) MIME-Version: 1.0 Received: by 10.223.132.200 with HTTP; Fri, 6 Aug 2010 18:21:00 -0700 (PDT) From: Joshua Boyd Date: Fri, 6 Aug 2010 21:21:00 -0400 Message-ID: To: freebsd-stable Content-Type: text/plain; charset=ISO-8859-1 X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: 8-STABLE Slow Write Speeds on ESXI 4.0 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 07 Aug 2010 01:21:22 -0000 Hello, I'm experiencing slow write speeds on 8-STABLE running on an ESXI 4.0 server, despite whatever tunables I've thrown at it. Read speeds are slower than they should be, but acceptable. Note, this is a thick provisioned disk, not thin. Speeds on Windows hosts are as expected for an MD3000 DAS, 250MB/s or so. [root@git ~]# dd if=/dev/da0 of=/dev/null bs=1M count=500 500+0 records in 500+0 records out 524288000 bytes transferred in 3.304514 secs (158658118 bytes/sec [root@git ~]# dd if=/dev/zero of=/var/testfile bs=1M count=500 500+0 records in 500+0 records out 524288000 bytes transferred in 52.083421 secs (10066313 bytes/sec) Output of gstat during write: L(q) ops/s r/s kBps ms/r w/s kBps ms/w %busy Name 6 82 0 0 0.0 82 10520 82.5 100.7| da0 [root@git ~]# vmstat da0 2 procs memory page disk faults cpu r b w avm fre flt re pi po fr sr da0 in sy cs us sy id 0 0 0 664M 3684M 28 0 0 0 59 0 0 4 90 347 0 2 98 0 0 0 664M 3667M 1 0 0 0 44 0 149 152 120 1340 0 3 97 0 0 0 664M 3646M 0 0 0 0 0 0 163 166 123 1385 0 2 98 0 0 0 664M 3626M 0 0 0 0 0 0 160 162 119 1393 0 4 96 0 0 0 664M 3605M 1 0 0 0 1 0 164 165 156 1435 0 4 96 [root@git ~]# camcontrol tags da0 (pass0:mpt0:0:0:0): device openings: 127 [root@git ~]# camcontrol devlist at scbus0 target 0 lun 0 (da0,pass0) And now, for the rest of the information: [root@git ~]# uname -ar FreeBSD git.openfisma.org 8.1-PRERELEASE FreeBSD 8.1-PRERELEASE #0: Mon Jun 28 17:46:33 EDT 2010 root@git.openfisma.org:/usr/obj/usr/src/sys/GENERIC amd64 Copyright (c) 1992-2010 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 8.1-PRERELEASE #0: Mon Jun 28 17:46:33 EDT 2010 root@git.openfisma.org:/usr/obj/usr/src/sys/GENERIC amd64 Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Quad-Core AMD Opteron(tm) Processor 2350 (1999.81-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x100f23 Family = 10 Model = 2 Stepping = 3 Features=0x78bfbff Features2=0x80802001> AMD Features=0xea500800 AMD Features2=0x1e9 TSC: P-state invariant real memory = 4294967296 (4096 MB) avail memory = 4099297280 (3909 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs FreeBSD/SMP: 4 package(s) x 1 core(s) cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 MADT: Forcing active-low polarity and level trigger for SCI ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-safe" frequency 3579545 Hz quality 850 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 cpu1: on acpi0 cpu2: on acpi0 cpu3: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pcib1: at device 1.0 on pci0 pci1: on pcib1 isab0: at device 7.0 on pci0 isa0: on isab0 atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x10c0-0x10cf at device 7.1 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pci0: at device 7.3 (no driver attached) pci0: at device 7.7 (no driver attached) vgapci0: port 0x10d0-0x10df mem 0xd4000000-0xd7ffffff,0xd8000000-0xd87fffff irq 16 at device 15.0 on pci0 pcib2: at device 17.0 on pci0 pci2: on pcib2 em0: port 0x2000-0x203f mem 0xd8820000-0xd883ffff,0xd8800000-0xd880ffff irq 18 at device 0.0 on pci2 em0: Memory Access and/or Bus Master bits were not set! em0: [FILTER] em0: Ethernet address: 00:50:56:81:3c:c8 pcib3: at device 21.0 on pci0 pci3: on pcib3 mpt0: port 0x4000-0x40ff mem 0xd9c04000-0xd9c07fff,0xd9c10000-0xd9c1ffff irq 18 at device 0.0 on pci3 mpt0: [ITHREAD] mpt0: MPI Version=1.5.0.0 pcib4: at device 21.1 on pci0 pci4: on pcib4 pcib5: at device 21.2 on pci0 pci5: on pcib5 pcib6: at device 21.3 on pci0 pci6: on pcib6 pcib7: at device 21.4 on pci0 pci7: on pcib7 pcib8: at device 21.5 on pci0 pci8: on pcib8 pcib9: at device 21.6 on pci0 pci9: on pcib9 pcib10: at device 21.7 on pci0 pci10: on pcib10 pcib11: at device 22.0 on pci0 pci11: on pcib11 pcib12: at device 22.1 on pci0 pci12: on pcib12 pcib13: at device 22.2 on pci0 pci13: on pcib13 pcib14: at device 22.3 on pci0 pci14: on pcib14 pcib15: at device 22.4 on pci0 pci15: on pcib15 pcib16: at device 22.5 on pci0 pci16: on pcib16 pcib17: at device 22.6 on pci0 pci17: on pcib17 pcib18: at device 22.7 on pci0 pci18: on pcib18 pcib19: at device 23.0 on pci0 pci19: on pcib19 pcib20: at device 23.1 on pci0 pci20: on pcib20 pcib21: at device 23.2 on pci0 pci21: on pcib21 pcib22: at device 23.3 on pci0 pci22: on pcib22 pcib23: at device 23.4 on pci0 pci23: on pcib23 pcib24: at device 23.5 on pci0 pci24: on pcib24 pcib25: at device 23.6 on pci0 pci25: on pcib25 pcib26: at device 23.7 on pci0 pci26: on pcib26 pcib27: at device 24.0 on pci0 pci27: on pcib27 pcib28: at device 24.1 on pci0 pci28: on pcib28 pcib29: at device 24.2 on pci0 pci29: on pcib29 pcib30: at device 24.3 on pci0 pci30: on pcib30 pcib31: at device 24.4 on pci0 pci31: on pcib31 pcib32: at device 24.5 on pci0 pci32: on pcib32 pcib33: at device 24.6 on pci0 pci33: on pcib33 pcib34: at device 24.7 on pci0 pci34: on pcib34 acpi_acad0: on acpi0 acpi_button0: on acpi0 atrtc0: port 0x70-0x71 irq 8 on acpi0 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] psm0: irq 12 on atkbdc0 psm0: [GIANT-LOCKED] psm0: [ITHREAD] psm0: model IntelliMouse, device ID 3 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppc0: [ITHREAD] ppbus0: on ppc0 plip0: on ppbus0 plip0: [ITHREAD] lpt0: on ppbus0 lpt0: [ITHREAD] lpt0: Interrupt-driven port ppi0: on ppbus0 uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 uart0: [FILTER] uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0 uart1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] fd0: <1440-KB 3.5" drive> on fdc0 drive 0 orm0: at iomem 0xc0000-0xc7fff,0xc8000-0xc8fff,0xdc000-0xdffff,0xe0000-0xe3fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 acpi_throttle0: on cpu0 acpi_throttle1: on cpu1 acpi_throttle1: failed to attach P_CNT device_attach: acpi_throttle1 attach returned 6 acpi_throttle2: on cpu2 acpi_throttle2: failed to attach P_CNT device_attach: acpi_throttle2 attach returned 6 acpi_throttle3: on cpu3 acpi_throttle3: failed to attach P_CNT device_attach: acpi_throttle3 attach returned 6 Timecounters tick every 10.000 msec acd0: CDROM at ata1-master UDMA33 da0 at mpt0 bus 0 scbus0 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: 300.000MB/s transfers da0: Command Queueing enabled da0: 40960MB (83886080 512 byte sectors: 255H 63S/T 5221C) SMP: AP CPU #3 Launched! SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! Trying to mount root from ufs:/dev/da0s1a VMware memory control driver initialized [root@git /var/log]# mptutil show adapter mpt0 Adapter: Board Name: SAS3444 Board Assembly: Chip Name: C1068E Chip Revision: UNUSED RAID Levels: none -- Joshua Boyd JBipNet E-mail: boydjd@jbip.net http://www.jbip.net