From owner-freebsd-hardware@FreeBSD.ORG Mon Jan 26 14:34:14 2009 Return-Path: Delivered-To: hardware@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E2400106571A for ; Mon, 26 Jan 2009 14:34:14 +0000 (UTC) (envelope-from eirik.overby@modirum.com) Received: from mail1.netservant.fi (mail1.netservant.fi [213.250.93.2]) by mx1.freebsd.org (Postfix) with ESMTP id 46B078FC3C for ; Mon, 26 Jan 2009 14:34:14 +0000 (UTC) (envelope-from eirik.overby@modirum.com) Received: by mail1.netservant.fi (Postfix, from userid 1103) id 46C4A57EC2; Mon, 26 Jan 2009 16:15:06 +0200 (EET) X-Spam-Checker-Version: SpamAssassin 3.2.4 (2008-01-01) on mail1.netservant.fi X-Spam-Level: X-Spam-Status: No, score=-2.5 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.2.4 Received: from mail1.netservant.fi (localhost [127.0.0.1]) by mail1.netservant.fi (Postfix) with ESMTP id A7C5F56839 for ; Mon, 26 Jan 2009 16:15:04 +0200 (EET) Received: from notes.modirum.com (notes.modirum.com [213.250.93.103]) by mail1.netservant.fi (Postfix) with ESMTP id 9598021F86 for ; Mon, 26 Jan 2009 16:15:04 +0200 (EET) Received: from [192.168.3.200] ([193.213.86.45]) by notes.modirum.com (Lotus Domino Release 6.5.5FP2) with ESMTP id 2009012616150209-5181 ; Mon, 26 Jan 2009 16:15:02 +0200 Message-Id: From: =?ISO-8859-1?Q?Eirik_=D8verby?= To: hardware@freebsd.org Mime-Version: 1.0 (Apple Message framework v930.3) Date: Mon, 26 Jan 2009 15:14:56 +0100 X-Mailer: Apple Mail (2.930.3) X-MIMETrack: Itemize by SMTP Server on Helsinki/Modirum(Release 6.5.5FP2|October 23, 2006) at 26.01.2009 16:15:02, Serialize by Router on Helsinki/Modirum(Release 6.5.5FP2|October 23, 2006) at 26.01.2009 16:15:04, Serialize complete at 26.01.2009 16:15:04 Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes X-Virus-Scanned: ClamAV using ClamSMTP Cc: Subject: Areca ARC-1210 abysmal performance X-BeenThere: freebsd-hardware@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: General discussion of FreeBSD hardware List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 26 Jan 2009 14:34:16 -0000 Hi, I've just purchased a pile of Areca 1210 controllers, having seen that they should perform well with FreeBSD. Now having hooked up a pair of them to 4 WD 250gb SATA drives and configured them to RAID1+0, I see them perform very, very badly. Below is a typical test I run on newly created arrays, to see the sustained write speeds they can handle. It's nowhere near a real-world test, but I've found it to often reveal issues early on. As you can see, the Areca seems to accept a lump of data (filling its write cache) early on, then practically slows to a crawl, and for long stretches of time no data is written at all, before another burst is written followed by trickling, repeat ad infinitum. I've repeated this with HDD cache on and off, controller cache on and off, NCQ on and off and at both SATA150 and SATA300 speeds. When disabling the controller cache (setting it to write-through), I don't get the initial burst, but a slow trickle of data ~5-15 mbytes/sec. While this is going on, the system is basically unresponsive. There is nothing else going on, only sshd running. I've just updated the firmware to the latest as of today, however that didn't change anything. I'm on FreeBSD 7.1-RELEASE. Dmesg output below. Can anyone point me in the right direction here? Thanks, /Eirik [root@md-hh-play-01 /usr]# dd if=/dev/zero of=testfile bs=64k & iostat 1 [1] 889 tty da0 pass0 pass1 cpu tin tout KB/t tps MB/s KB/t tps MB/s KB/t tps MB/s us ni sy in id 1 98 61.81 76 4.60 0.00 0 0.00 0.00 0 0.00 0 0 0 0 99 0 231 64.00 3732 233.22 0.00 0 0.00 0.00 0 0.00 0 0 13 2 85 0 79 64.00 264 16.48 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 78 64.00 394 24.60 0.00 0 0.00 0.00 0 0.00 0 0 1 0 98 0 77 64.00 320 19.98 0.00 0 0.00 0.00 0 0.00 0 0 1 1 98 0 77 61.74 234 14.10 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 78 64.00 180 11.24 0.00 0 0.00 0.00 0 0.00 0 0 0 0 99 0 77 60.90 31 1.84 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 78 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 0 78 0.00 0 0.00 0.00 0 0.00 0.00 0 0.00 0 0 0 0 100 [root@md-hh-play-01 /usr]# vmstat -i interrupt total rate irq3: sio0 304870 198 irq4: sio1 2 0 irq10: ohci0+ 258071 167 irq14: ata0 58 0 cpu0: timer 3075904 1999 irq256: nfe0 2652 1 cpu1: timer 3067930 1994 cpu2: timer 3067899 1994 cpu3: timer 3067930 1994 Total 12845316 8351 DMESG: Copyright (c) 1992-2009 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD is a registered trademark of The FreeBSD Foundation. FreeBSD 7.1-RELEASE #0: Thu Jan 1 08:58:24 UTC 2009 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: Dual-Core AMD Opteron(tm) Processor 2218 (2600.02-MHz K8-class CPU) Origin = "AuthenticAMD" Id = 0x40f13 Stepping = 3 Features = 0x178bfbff < FPU ,VME ,DE ,PSE ,TSC ,MSR ,PAE ,MCE ,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CLFLUSH,MMX,FXSR,SSE,SSE2,HTT> Features2=0x2001 AMD Features=0xea500800 AMD Features2=0x1f Cores per package: 2 usable memory = 4280922112 (4082 MB) avail memory = 4115517440 (3924 MB) ACPI APIC Table: FreeBSD/SMP: Multiprocessor System Detected: 4 CPUs cpu0 (BSP): APIC ID: 0 cpu1 (AP): APIC ID: 1 cpu2 (AP): APIC ID: 2 cpu3 (AP): APIC ID: 3 ioapic0 irqs 0-23 on motherboard kbd1 at kbdmux0 ath_hal: 0.9.20.3 (AR5210, AR5211, AR5212, RF5111, RF5112, RF2413, RF5413) acpi0: on motherboard acpi0: [ITHREAD] acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 acpi_hpet0: iomem 0xfed00000-0xfed003ff on acpi0 Timecounter "HPET" frequency 25000000 Hz quality 900 acpi_button0: on acpi0 pcib0: port 0xcf8-0xcff on acpi0 pci0: on pcib0 pci0: at device 0.0 (no driver attached) isab0: port 0x1c00-0x1c7f at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) ohci0: mem 0xc8040000-0xc8040fff irq 10 at device 2.0 on pci0 ohci0: [GIANT-LOCKED] ohci0: [ITHREAD] usb0: OHCI version 1.0, legacy support usb0: SMM does not respond, resetting usb0: on ohci0 usb0: USB revision 1.0 uhub0: on usb0 uhub0: 10 ports with 10 removable, self powered ehci0: mem 0xc8041000-0xc80410ff irq 11 at device 2.1 on pci0 ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb1: EHCI version 1.0 usb1: companion controller, 10 ports each: usb0 usb1: on ehci0 usb1: USB revision 2.0 uhub1: on usb1 uhub1: 10 ports with 10 removable, self powered atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0x2480-0x248f at device 4.0 on pci0 ata0: on atapci0 ata0: [ITHREAD] ata1: on atapci0 ata1: [ITHREAD] pcib1: at device 6.0 on pci0 pci1: on pcib1 vgapci0: port 0x3000-0x30ff mem 0xd0000000-0xd7ffffff,0xc8100000-0xc810ffff irq 5 at device 4.0 on pci1 nfe0: port 0x2490-0x2497 mem 0xc8042000-0xc8042fff,0xc8041800-0xc80418ff,0xc8041400-0xc804140f irq 11 at device 8.0 on pci0 miibus0: on nfe0 e1000phy0: PHY 0 on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX- FDX, auto nfe0: Ethernet address: 00:e0:81:76:89:a8 nfe0: [FILTER] nfe0: [FILTER] nfe0: [FILTER] nfe0: [FILTER] nfe0: [FILTER] nfe0: [FILTER] nfe0: [FILTER] nfe0: [FILTER] nfe1: port 0x2498-0x249f mem 0xc8044000-0xc8044fff,0xc8043000-0xc80430ff,0xc8041c00-0xc8041c0f irq 10 at device 9.0 on pci0 miibus1: on nfe1 e1000phy1: PHY 1 on miibus1 e1000phy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX- FDX, auto nfe1: Ethernet address: 00:e0:81:76:89:a9 nfe1: [FILTER] nfe1: [FILTER] nfe1: [FILTER] nfe1: [FILTER] nfe1: [FILTER] nfe1: [FILTER] nfe1: [FILTER] nfe1: [FILTER] pcib2: at device 10.0 on pci0 pci2: on pcib2 pcib3: at device 13.0 on pci0 pci3: on pcib3 pcib4: at device 15.0 on pci0 pci4: on pcib4 pcib5: at device 0.0 on pci4 pci5: on pcib5 arcmsr0: mem 0xc8200000-0xc8200fff,0xd8000000-0xd83fffff irq 10 at device 14.0 on pci5 ARECA RAID ADAPTER0: Driver Version 1.20.00.15 2007-10-07 ARECA RAID ADAPTER0: FIRMWARE VERSION V1.46 2009-01-06 arcmsr0: [ITHREAD] pcib6: at device 0.2 on pci4 pci6: on pcib6 atkbdc0: port 0x60,0x64 irq 1 on acpi0 atkbd0: irq 1 on atkbdc0 kbd0 at atkbd0 atkbd0: [GIANT-LOCKED] atkbd0: [ITHREAD] sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: configured irq 3 not in bitmap of probed irqs 0 sio0: port may not be enabled sio0: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 flags 0x10 on acpi0 sio0: type 16550A sio0: [FILTER] sio1: configured irq 4 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1: configured irq 4 not in bitmap of probed irqs 0 sio1: port may not be enabled sio1: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 on acpi0 sio1: type 16550A sio1: [FILTER] fdc0: port 0x3f0-0x3f5,0x3f7 irq 6 drq 2 on acpi0 fdc0: [FILTER] ppc0: port 0x378-0x37f,0x778-0x77f irq 7 drq 3 on acpi0 ppc0: SMC-like chipset (ECP/EPP/PS2/NIBBLE) in COMPATIBLE mode ppc0: FIFO with 16/16/9 bytes threshold ppbus0: on ppc0 ppbus0: [ITHREAD] plip0: on ppbus0 plip0: WARNING: using obsoleted IFF_NEEDSGIANT flag lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 ppc0: [GIANT-LOCKED] ppc0: [ITHREAD] cpu0: on acpi0 acpi_throttle0: on cpu0 powernow0: on cpu0 cpu1: on acpi0 powernow1: on cpu1 cpu2: on acpi0 powernow2: on cpu2 cpu3: on acpi0 powernow3: on cpu3 ipmi0: on isa0 ipmi0: KCS mode found at io 0xca8 alignment 0x4 on isa orm0: at iomem 0xc0000-0xc8fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounters tick every 1.000 msec acd0: CDROM at ata0-slave UDMA33 Waiting 5 seconds for SCSI devices to settle ipmi0: IPMI device rev. 0, firmware rev. 0.2, version 2.0 ipmi0: Number of channels 5 ipmi0: Attached watchdog (probe16:arcmsr0:0:16:0): inquiry data fails comparison at DV1 step da0 at arcmsr0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da0: 476837MB (976562176 512 byte sectors: 255H 63S/T 60788C) SMP: AP CPU #1 Launched! SMP: AP CPU #2 Launched! SMP: AP CPU #3 Launched! Trying to mount root from ufs:/dev/da0s1a fuse4bsd: version 0.3.9-pre1, FUSE ABI 7.8 nfe0: link state changed to UP