From owner-freebsd-drivers@FreeBSD.ORG Fri Dec 1 18:14:42 2006 Return-Path: X-Original-To: freebsd-drivers@freebsd.org Delivered-To: freebsd-drivers@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [69.147.83.52]) by hub.freebsd.org (Postfix) with ESMTP id 5BE3F16A403; Fri, 1 Dec 2006 18:14:42 +0000 (UTC) (envelope-from freebsd@sopwith.solgatos.com) Received: from schitzo.solgatos.com (pool-71-245-104-192.ptldor.fios.verizon.net [71.245.104.192]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE67443CB4; Fri, 1 Dec 2006 18:14:24 +0000 (GMT) (envelope-from freebsd@sopwith.solgatos.com) Received: from schitzo.solgatos.com (localhost.home.localnet [127.0.0.1]) by schitzo.solgatos.com (8.13.8/8.13.6) with ESMTP id kB1IEdo4031252; Fri, 1 Dec 2006 10:14:39 -0800 Received: from sopwith.solgatos.com (uucp@localhost) by schitzo.solgatos.com (8.13.8/8.13.4/Submit) with UUCP id kB1IEd1q031249; Fri, 1 Dec 2006 10:14:39 -0800 Received: from localhost by sopwith.solgatos.com (8.8.8/6.24) id SAA07107; Fri, 1 Dec 2006 18:13:17 GMT Message-Id: <200612011813.SAA07107@sopwith.solgatos.com> To: freebsd-amd64@freebsd.org, freebsd-drivers@freebsd.org Date: Fri, 01 Dec 2006 10:13:17 +0000 From: Dieter Cc: Subject: nforce4 disk I/O X-BeenThere: freebsd-drivers@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: freebsd@sopwith.solgatos.com List-Id: Writing device drivers for FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Dec 2006 18:14:42 -0000 I thought I tested disk I/O in the past and got full write speed to 4 drives at once, but now I get a slowdown if writing to more than one disk at once. FreeBSD 6.0 FFS with soft updates nvidia nforce 4 ultra 4 SATA drives 1 PATA drive 1 PATA CD/DVD (idle) Doing dd from /dev/zero to 1 drive, I get about 7 MB/s. Doing dd from /dev/zero to 2 drives, I get about 8 MB/s total. Doing dd from /dev/zero to 3 drives, I get about 8-9 MB/s total. Reading from one drive doesn't hurt the read or write speed to a second drive. I'm also getting things like one process that is I/O bound writing to a disk will significantly slow down a CPU bound process writing to a different disk, and the CPU becomes over 50% idle. I wonder if there is contention for a lock that only happens when writing? Does the driver hold a lock while waiting for the disk to say "yes I wrote that sector" ? atapci0: port 0x1f0-0x1f7,0x3f6,0x170-0x177,0x376,0xe000-0xe00f at device 6.0 on pci0 ata0: on atapci0 ata1: on atapci0 atapci1: port 0x9f0-0x9f7,0xbf0-0xbf3,0x970-0x977,0xb70-0xb73,0xcc00-0xcc0f mem 0xfebfb000-0xfebfbfff irq 10 at device 7.0 on pci0 ata2: on atapci1 ata3: on atapci1 atapci2: port 0x9e0-0x9e7,0xbe0-0xbe3,0x960-0x967,0xb60-0xb63,0xb800-0xb80f mem 0xfebfa000-0xfebfafff irq 11 at device 8.0 on pci0 ata4: on atapci2 ata5: on atapci2 acd0: DVDR at ata0-master UDMA66 ad2: 305245MB at ata1-master UDMA100 ad4: 238475MB at ata2-master SATA150 ad6: 238475MB at ata3-master SATA150 ad8: 238475MB at ata4-master SATA150 ad10: 305245MB at ata5-master SATA150 cd0 at ata0 bus 0 target 0 lun 0