From owner-freebsd-stable@FreeBSD.ORG Mon Dec 15 19:00:23 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3205106567A for ; Mon, 15 Dec 2008 19:00:23 +0000 (UTC) (envelope-from mike@sentex.net) Received: from smarthost1.sentex.ca (smarthost1.sentex.ca [64.7.153.18]) by mx1.freebsd.org (Postfix) with ESMTP id 6BFBB8FC13 for ; Mon, 15 Dec 2008 19:00:23 +0000 (UTC) (envelope-from mike@sentex.net) Received: from lava.sentex.ca (pyroxene.sentex.ca [199.212.134.18]) by smarthost1.sentex.ca (8.14.3/8.14.3) with ESMTP id mBFJ0KoX052365; Mon, 15 Dec 2008 14:00:21 -0500 (EST) (envelope-from mike@sentex.net) Received: from mdt-xp.sentex.net (simeon.sentex.ca [192.168.43.27]) by lava.sentex.ca (8.13.8/8.13.3) with ESMTP id mBFJ0Jom084267 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Mon, 15 Dec 2008 14:00:19 -0500 (EST) (envelope-from mike@sentex.net) Message-Id: <200812151900.mBFJ0Jom084267@lava.sentex.ca> X-Mailer: QUALCOMM Windows Eudora Version 7.1.0.9 Date: Mon, 15 Dec 2008 14:00:26 -0500 To: Paul MacKenzie , freebsd-stable@freebsd.org From: Mike Tancsa In-Reply-To: <494698E4.2070406@cogeco.ca> References: <2515BCEE3A2F4CBA8FFB9F1C052924AA@jmlaptop> <4934CB77.30906@transactionware.com> <494698E4.2070406@cogeco.ca> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; format=flowed X-Scanned-By: MIMEDefang 2.64 on 64.7.153.18 Cc: Jan Mikkelsen Subject: Re: 7.1-PRERELEASE: arcmsr write performance problem X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 15 Dec 2008 19:00:23 -0000 At 12:50 PM 12/15/2008, Paul MacKenzie wrote: > > Any suggestions on where to look next? Are there obvious candidates? > >After weeks of working on this I now believe that anything that taxes >the writing to the hard drives causes the system CPU numbers to spike >through the roof (approx 80% usage) and the server grinds to a halt. And >I also see wide swings in the System CPU usage. It reminds me of the >QUOTA issue I had with 6.2 where the system usage was really high and it >was the QUOTA code that was broken. Hi, I dont have a box in the lab I can test a lot with right now, but I do have a couple of these cards in customer servers and write performance seems ok This is an AMD64 box, 8G RAM 7.1-PRERELEASE FreeBSD 7.1-PRERELEASE #2: Tue Sep 23 09:25:02 EDT 2008 running a lot of postgres queries right now [ns8]# dd if=/dev/zero of=/var/tmp/test count=1000 bs=1024k 1000+0 records in 1000+0 records out 1048576000 bytes transferred in 10.265211 secs (102148508 bytes/sec) [ns8]# which seems reasonable for raw write performance. If you bring up the little Areca web app, are there any errors about the array? When you created the raid sets, what params did you use ? The above is on Volume Capacity 320.0GB SCSI Ch/Id/Lun 0/0/0 Raid Level Raid 1+0 Stripe Size 64KBytes Block Size 512Bytes Member Disks 4 Cache Mode Write Back Tagged Queuing Enabled Volume State Normal [ns8]# vmstat -i interrupt total rate irq4: sio0 57065 0 irq17: em1 3989494045 554 irq18: arcmsr0 558098657 77 cpu0: timer 14381393929 2000 irq256: em0 22763077 3 cpu1: timer 14381384902 1999 Total 33333191675 4635 [ns8]# arcmsr0: mem 0xe8600000-0xe8600fff,0xe8000000-0xe83fffff irq 18 at device 14.0 on pci2 ARECA RAID ADAPTER0: Driver Version 1.20.00.15 2007-10-07 ARECA RAID ADAPTER0: FIRMWARE VERSION V1.43 2007-4-17 arcmsr0: [ITHREAD] ..... Waiting 5 seconds for SCSI devices to settle (probe16:arcmsr0:0:16:0): inquiry data fails comparison at DV1 step da0 at arcmsr0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 166.666MB/s transfers (83.333MHz DT, offset 32, 16bit) da0: 305175MB (624999424 512 byte sectors: 255H 63S/T 38904C) SMP: AP CPU #1 Launched!