From owner-freebsd-performance@FreeBSD.ORG Mon Sep 5 00:18:39 2005 Return-Path: X-Original-To: freebsd-performance@freebsd.org Delivered-To: freebsd-performance@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4C7D616A41F for ; Mon, 5 Sep 2005 00:18:39 +0000 (GMT) (envelope-from jeff.tchang@gmail.com) Received: from rproxy.gmail.com (rproxy.gmail.com [64.233.170.205]) by mx1.FreeBSD.org (Postfix) with ESMTP id BCF4043D45 for ; Mon, 5 Sep 2005 00:18:38 +0000 (GMT) (envelope-from jeff.tchang@gmail.com) Received: by rproxy.gmail.com with SMTP id r35so763494rna for ; Sun, 04 Sep 2005 17:18:36 -0700 (PDT) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type; b=OpDzRTgOo32By6C3UTHJDRimm27T9BDGu0uZLmNvVAbhx8/NdS0QqpP0BSmfM1gB5EeZizIuXIDtLFsbkL3Nr39GzjhbqZFe7EZSI0aikOiwqGDtWBgLAeZCp0GuD6LFj4RkNlzDzDC/QsmCbDE6uAHpl8Xl++A7LKtF2QI0SdI= Received: by 10.38.10.54 with SMTP id 54mr30035rnj; Sun, 04 Sep 2005 17:18:35 -0700 (PDT) Received: by 10.39.1.43 with HTTP; Sun, 4 Sep 2005 17:18:35 -0700 (PDT) Message-ID: <63f9d26505090417183dff415e@mail.gmail.com> Date: Sun, 4 Sep 2005 20:18:35 -0400 From: Jeff Tchang To: freebsd-performance@freebsd.org Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: 3Ware 7500-4 Slow X-BeenThere: freebsd-performance@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Performance/tuning List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 05 Sep 2005 00:18:39 -0000 I have a 3Ware 7500-4 card. I am experiencing some sluggishness with the=20 RAID5 implementation. It has been running in a stable state for quite some= =20 time, however, I have always wondered if there was something I was not doin= g=20 because it is very slow. System Specs: pharos# dmesg | grep twe twe0: <3ware Storage Controller driver ver. 1.40.01.001 >=20 port 0xccd0-0xccdf mem 0xfe000000-0xfe7fffff,0xfe800400-0xfe80040f irq 2 at= =20 device 16.0 on pci0 twe0: 4 ports, Firmware FE7X 1.05.00.068 , BIOS BE7X=20 1.08.00.048 twed0: on twe0 twed0: 152636MB (312598784 sectors) Mounting root from ufs:/dev/twed0s1a uname -a FreeBSD 4.10-RELEASE FreeBSD 4.10-RELEASE #4: Sat Sep 3 15:53:44 EDT 2005= =20 i386 3ware CLI> info List of controllers ------------------- Controller 0: 7500-4 (4) 3ware CLI> info c0 Controller: c0 ------------- Driver: 1.40.01.001 Model: 7500-4 FW: FE7X 1.05.00.068 BIOS: BE7X 1.08.00.048 Monitor: ME7X 1.01.00.040 Serial #:=20 PCB: Rev3 PCHIP: 1.30-33 ACHIP: 3.20 # of units: 1 Unit 0: RAID 5 149.05 GB ( 312598784 blocks): OK # of ports: 4 Port 0: ST380020A 74.53 GB (156301488 blocks): OK(unit 0) Port 1: ST380020A 74.53 GB (156301488 blocks): OK(unit 0) Port 2: ST380020A 74.53 GB (156301488 blocks): OK(unit 0) 3ware CLI> info c0 u0 Unit /c0/u0 --------------------- Status: OK Unit Type: RAID 5 Stripe Size: 64k Size: 149.05 GB (312598784 blocks) # of subunits: 3 Subunit 0: CBOD: OK Physical Port: 0 Logical Port: 0 Subunit 1: CBOD: OK Physical Port: 1 Logical Port: 1 Subunit 2: CBOD: OK Physical Port: 2 Logical Port: 2 When I run systat -vmstat 1, I continually see twed0 running at 100% busy.= =20 In addition KB/t is between 1 and 10. tps is about 50. The dmesg says that the device is on irq2. The vmstat shows the interrupt a= s=20 mux irq2 and is about 300 to 500 calls. As far as I know, I am using the=20 most up to date firmware for the card. The computer itself is a dual 600 Mh= z=20 PIII (Dell). When I attempt to write many small files or remove a directory is when the= =20 slowness kicks in. Is this just something due to Raid5? Here is the output= =20 of Bonnie++: Version 1.93c ------Sequential Output------ --Sequential Input- --Random- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP pharos. 300M 53 94 16394 24 7163 14 121 95 35991 35 249.8 12 Latency 261ms 400ms 485ms 93237us 26252us 12216ms Version 1.93c ------Sequential Create------ --------Random Create-------- pharos -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 3835 45 28846 93 8255 85 4249 56 26604 84 4835 52 Latency 43557us 3422us 8544us 169ms 39158us 6936us