From owner-freebsd-scsi@FreeBSD.ORG Mon Aug 27 11:08:35 2007 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 28F4716A50E for ; Mon, 27 Aug 2007 11:08:35 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id 1574513C465 for ; Mon, 27 Aug 2007 11:08:35 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.14.1/8.14.1) with ESMTP id l7RB8YEg020660 for ; Mon, 27 Aug 2007 11:08:34 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.1/8.14.1/Submit) id l7RB8Xt9020656 for freebsd-scsi@FreeBSD.org; Mon, 27 Aug 2007 11:08:33 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 27 Aug 2007 11:08:33 GMT Message-Id: <200708271108.l7RB8Xt9020656@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Aug 2007 11:08:35 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/40895 scsi wierd kernel / device driver bug o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o kern/60598 scsi wire down of scsi devices conflicts with config o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 o kern/81887 scsi [aac] Adaptec SCSI 2130S aac0: GetDeviceProbeInfo comm o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/93128 scsi [sym] FreeBSD 6.1 BETA 1 has problems with Symbios/LSI o kern/94838 scsi Kernel panic while mounting SD card with lock switch o o kern/99954 scsi [ahc] reading from DVD failes on 6.x (regression) o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks 15 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/23314 scsi [aic] aic driver fails to detect Adaptec 1520B unless o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce o kern/38828 scsi [feature request] DPT PM2012B/90 doesn't work o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/114597 scsi [sym] System hangs at SCSI bus reset with dual HBAs 6 problems total. From owner-freebsd-scsi@FreeBSD.ORG Tue Aug 28 09:31:40 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CA59B16A417 for ; Tue, 28 Aug 2007 09:31:40 +0000 (UTC) (envelope-from melnik@sovok.uplink.net.ua) Received: from sovok.uplink.net.ua (sovok.uplink.net.ua [195.3.204.34]) by mx1.freebsd.org (Postfix) with ESMTP id 8A83413C461 for ; Tue, 28 Aug 2007 09:31:40 +0000 (UTC) (envelope-from melnik@sovok.uplink.net.ua) Received: by sovok.uplink.net.ua (Postfix, from userid 1001) id 4C01D74694; Tue, 28 Aug 2007 12:08:34 +0300 (EEST) Date: Tue, 28 Aug 2007 12:08:34 +0300 From: Vladimir Melnik To: freebsd-scsi@freebsd.org Message-ID: <20070828090834.GE42596@sovok.uplink.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.13 (2006-08-11) Subject: mpt problem X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2007 09:31:40 -0000 Hello. We have a very stange problem with "LSILogic 1030 Ultra4 Adapter". There is an information about it: mpt0: port 0x4000-0x40ff mem 0xdc120000-0xdc13ffff,0xdc100000-0xdc11ffff irq 18 at device 3.0 on pci3 mpt0: [GIANT-LOCKED] mpt0: MPI Version=1.2.14.0 Everything works fine, but time to time disk operations hangs and we have this messages in log: mpt0: request 0xc4bd81f0:196 timed out for ccb 0xc4dedc00 (req->ccb 0xc4dedc00) mpt0: attempting to abort req 0xc4bd81f0:196 function 0 mpt0: completing timedout/aborted req 0xc4bd81f0:196 mpt0: abort of req 0xc4bd81f0:0 completed mpt0: request 0xc4bd8220:198 timed out for ccb 0xc4dedc00 (req->ccb 0xc4dedc00) mpt0: attempting to abort req 0xc4bd8220:198 function 0 mpt0: mpt_wait_req(1) timed out mpt0: mpt_recover_commands: abort timed-out. Resetting controller mpt0: mpt_cam_event: 0x0 mpt0: completing timedout/aborted req 0xc4bd8220:198 (da0:mpt0:0:0:0): lost device (da0:mpt0:0:0:0): Invalidating pack (da0:mpt0:0:0:0): Synchronize cache failed, status == 0x4a, scsi status == 0x0 (da0:mpt0:0:0:0): removing device entry SCSI-bus connected to Areca RAID-controller and this controller works fine with MS Windows. -- V.Melnik From owner-freebsd-scsi@FreeBSD.ORG Tue Aug 28 18:35:23 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 721D616A418 for ; Tue, 28 Aug 2007 18:35:23 +0000 (UTC) (envelope-from raul.rodriguez@estudiosms.es) Received: from llcc728-a.servidoresdns.net (llcc728-a.servidoresdns.net [82.223.191.109]) by mx1.freebsd.org (Postfix) with ESMTP id 2FAAB13C4B5 for ; Tue, 28 Aug 2007 18:35:23 +0000 (UTC) (envelope-from raul.rodriguez@estudiosms.es) Received: from mail.estudiosms.es (localhost.localdomain [127.0.0.1]) by llcc728-a.servidoresdns.net (Postfix) with ESMTP id C6EA317A2617 for ; Tue, 28 Aug 2007 20:13:42 +0200 (CEST) Received: from 200.55.130.195 by mail.estudiosms.es with HTTP; Tue, 28 Aug 2007 20:13:42 +0200 (CEST) Message-ID: <1632.200.55.130.195.1188324822.squirrel@mail.estudiosms.es> Date: Tue, 28 Aug 2007 20:13:42 +0200 (CEST) From: raul.rodriguez@estudiosms.es To: "freebsd-scsi@freebsd.org" User-Agent: SquirrelMail/1.4.8-4.0.1.el4 MIME-Version: 1.0 Content-Type: text/plain;charset=utf-8 Content-Transfer-Encoding: 8bit X-Priority: 3 (Normal) Importance: Normal Subject: Maxtor atlas 10k iv with AIC-7902 on FreeBSD X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2007 18:35:23 -0000 Hello Look. I have a problem with an Integris CathLab System. It only uses 18GBytes HDD. And nowadays you can figure how difficult is to get such as discs. Can you tell me a way to limit by Firmware the capacity of a brand new Disc let says 73 GBytes to just 18GBytes. By the way, and sure that’s the way Philips works when you order a new disc. Bye Raul From owner-freebsd-scsi@FreeBSD.ORG Tue Aug 28 19:43:25 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8521116A417 for ; Tue, 28 Aug 2007 19:43:25 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 08B0013C491 for ; Tue, 28 Aug 2007 19:43:24 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7SJUL4H026641; Tue, 28 Aug 2007 13:30:21 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D477C5.1040005@samsco.org> Date: Tue, 28 Aug 2007 13:30:13 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: raul.rodriguez@estudiosms.es References: <1632.200.55.130.195.1188324822.squirrel@mail.estudiosms.es> In-Reply-To: <1632.200.55.130.195.1188324822.squirrel@mail.estudiosms.es> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 8bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Tue, 28 Aug 2007 13:30:21 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: "freebsd-scsi@freebsd.org" Subject: Re: Maxtor atlas 10k iv with AIC-7902 on FreeBSD X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 28 Aug 2007 19:43:25 -0000 raul.rodriguez@estudiosms.es wrote: > Hello > Look. I have a problem with an Integris CathLab System. > It only uses 18GBytes HDD. And nowadays you can figure how difficult is to > get such as discs. Can you tell me a way to limit by Firmware the capacity > of a brand new Disc let says 73 GBytes to just 18GBytes. > By the way, and sure that’s the way Philips works when you order a new disc. > Bye > Raul > This is generally not considered a useful feature for most people, so it's not something that FreeBSD is naturally able to do =-) You'll need to hack src/sys/cam/scsi/scsi_da.c so that when it issues a READ_CAPACITY command, it ignores the results for your particular drive and instead substitutes a synthetic 18GB value. Scott From owner-freebsd-scsi@FreeBSD.ORG Wed Aug 29 20:50:16 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8A96916A418 for ; Wed, 29 Aug 2007 20:50:16 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.175]) by mx1.freebsd.org (Postfix) with ESMTP id DBF9213C4DA for ; Wed, 29 Aug 2007 20:50:15 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: by ug-out-1314.google.com with SMTP id a2so154316ugf for ; Wed, 29 Aug 2007 13:50:14 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=Yv13/Ll5SaRwwJChvF8aDAJgFPoWXzRTdLv3v0v9PX8KVThCWmL4YaN4e9ygjCAmA92JlZwUaJ2qVuyA6tuUXGzw47pR5KrkG/3yRSKpoL+RCTWlsFBr6Wq4/HDA1qRGOf7/ps0jPzIE9/e0IvVxNBocjz8ZByxTOOXThy/kk6Y= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=S85zDfzdXUuSg8dI0mLymonj7mA1jtuRWi5boiDvh762DY3HDHPs3nE6ixGArIaBKVWXL2Rt726qBf5iB6govWXoBsFDCFkFi/oPLE8r5n5ITiIVy0h/v5JfU3JTKwJnXQuvrifDYTTlhOFlamrK3VGCM98Uw7StMkhAbVxQxLE= Received: by 10.142.191.2 with SMTP id o2mr49105wff.1188416703034; Wed, 29 Aug 2007 12:45:03 -0700 (PDT) Received: by 10.142.101.15 with HTTP; Wed, 29 Aug 2007 12:45:03 -0700 (PDT) Message-ID: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> Date: Wed, 29 Aug 2007 16:45:03 -0300 From: "Lutieri G." To: "FREEBSD - SCSI - LIST" MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline Subject: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2007 20:50:16 -0000 Hi! I have a problem with my new server sun x4100(http://www.sun.com/servers/entry/x4100/specs.xml). Ffirmware is 6.14.04.00 retrivied from sun's site. I cannot get more than 3 MB/s when writing to it. #pciconf -lv --- CUT --- mpt0@pci2:3:0: class=0x010000 card=0x30601000 chip=0x00501000 rev=0x02 hdr=0x00 vendor = 'LSI Logic (Was: Symbios Logic, NCR)' class = mass storage subclass = SCSI --- CUT --- when i run blogbench, squid -z, or any other software systat return da0 with 3.0MB/s and 1005 busy. I have tried both with and without RAID configured through the controller and i get same results. Some times my dmesg return this message : Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 65 Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 37 Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 65 Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 65 Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 65 Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 62 Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 65 Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x04 Depth 65 Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Aug 29 15:22:47 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 Target 0x03 Depth 65 Is this a known issue ? tanx -- Lutieri G. B. From owner-freebsd-scsi@FreeBSD.ORG Wed Aug 29 21:13:52 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5EC9216A421 for ; Wed, 29 Aug 2007 21:13:52 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id BFBB513C45B for ; Wed, 29 Aug 2007 21:13:51 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7TLDikg033767; Wed, 29 Aug 2007 15:13:45 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D5E17F.3070403@samsco.org> Date: Wed, 29 Aug 2007 15:13:35 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: "Lutieri G." References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> In-Reply-To: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Wed, 29 Aug 2007 15:13:45 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2007 21:13:52 -0000 You forgot to say which version of FreeBSD you are using. Scott Lutieri G. wrote: > Hi! > > I have a problem with my new server sun > x4100(http://www.sun.com/servers/entry/x4100/specs.xml). > > Ffirmware is 6.14.04.00 retrivied from sun's site. > > I cannot get more than 3 MB/s when writing to it. > > #pciconf -lv > --- CUT --- > mpt0@pci2:3:0: class=0x010000 card=0x30601000 chip=0x00501000 rev=0x02 hdr=0x00 > vendor = 'LSI Logic (Was: Symbios Logic, NCR)' > class = mass storage > subclass = SCSI > --- CUT --- > > > when i run blogbench, squid -z, or any other software systat return > da0 with 3.0MB/s and 1005 busy. > > I have tried both with and without RAID configured through the > controller and i get same results. > > Some times my dmesg return this message : > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 65 > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 37 > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 65 > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 65 > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 65 > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 62 > Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 65 > Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x04 Depth 65 > Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > Aug 29 15:22:47 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > Target 0x03 Depth 65 > > > Is this a known issue ? > > tanx > From owner-freebsd-scsi@FreeBSD.ORG Wed Aug 29 21:16:55 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A159916A555 for ; Wed, 29 Aug 2007 21:16:54 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.173]) by mx1.freebsd.org (Postfix) with ESMTP id 0327213C45B for ; Wed, 29 Aug 2007 21:16:51 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: by ug-out-1314.google.com with SMTP id a2so159099ugf for ; Wed, 29 Aug 2007 14:16:50 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=sgxxJepu7mXGz0uqoKoYjdU27dxMtG33CKvChMCoDcBjsf82VUBME6oPySxC+kB5oMdfYwtjigJB0oNp3YBpqN37YyemS4XurVeEEzmYNLa8/eiOVpAJccoQNXSmRa65Gbr6hKbiBHduQXWIrixAFZShmMmpRkM9yncGCD5gW3I= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=NZ8F/6UU1hG9ayx6W9ZNizZJdeZn+Y+EuRv+GFjdC/aWHC0WMQSiF8VN83iLtIn/N5mgBCrCRgwT8BuFckg1HzDSKuliqOo2434HDHmDnd2EtHKiIx5SB3P9cgOZ46MEOjxZNI/zJPyVtuR9F+4Mi+J1Rb44+JNKwgHcqeZSfHs= Received: by 10.143.165.13 with SMTP id s13mr52361wfo.1188422208477; Wed, 29 Aug 2007 14:16:48 -0700 (PDT) Received: by 10.142.101.15 with HTTP; Wed, 29 Aug 2007 14:16:48 -0700 (PDT) Message-ID: <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> Date: Wed, 29 Aug 2007 18:16:48 -0300 From: "Lutieri G." To: "FREEBSD - SCSI - LIST" In-Reply-To: <46D5E17F.3070403@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2007 21:16:55 -0000 Sorry. # uname -a FreeBSD sd.xyz.com.br 6.2-STABLE FreeBSD 6.2-STABLE #0: Wed Aug 29 10:26:18 BRT 2007 root@sd.xyz.com.br:/usr/src/sys/amd64/compile/LGB amd64 Yesterday i've downloaded kernel from cvsup16.FreeBSD.org and compile it. 2007/8/29, Scott Long : > You forgot to say which version of FreeBSD you are using. > > Scott > > > Lutieri G. wrote: > > Hi! > > > > I have a problem with my new server sun > > x4100(http://www.sun.com/servers/entry/x4100/specs.xml). > > > > Ffirmware is 6.14.04.00 retrivied from sun's site. > > > > I cannot get more than 3 MB/s when writing to it. > > > > #pciconf -lv > > --- CUT --- > > mpt0@pci2:3:0: class=0x010000 card=0x30601000 chip=0x00501000 rev=0x02 hdr=0x00 > > vendor = 'LSI Logic (Was: Symbios Logic, NCR)' > > class = mass storage > > subclass = SCSI > > --- CUT --- > > > > > > when i run blogbench, squid -z, or any other software systat return > > da0 with 3.0MB/s and 1005 busy. > > > > I have tried both with and without RAID configured through the > > controller and i get same results. > > > > Some times my dmesg return this message : > > > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 65 > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 37 > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 65 > > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 65 > > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 65 > > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 62 > > Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 65 > > Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x04 Depth 65 > > Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > Aug 29 15:22:47 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 > > Target 0x03 Depth 65 > > > > > > Is this a known issue ? > > > > tanx > > > > -- Att. Lutieri G. B. From owner-freebsd-scsi@FreeBSD.ORG Wed Aug 29 21:31:43 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7E1CD16A417 for ; Wed, 29 Aug 2007 21:31:43 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 12F7C13C46E for ; Wed, 29 Aug 2007 21:31:42 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7TLVddf033861; Wed, 29 Aug 2007 15:31:39 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D5E5B1.207@samsco.org> Date: Wed, 29 Aug 2007 15:31:29 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: "Lutieri G." References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> In-Reply-To: <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Wed, 29 Aug 2007 15:31:39 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2007 21:31:43 -0000 Try adding the following line to /boot/loader.conf: hw.mpt.enable_sata_wc=1 Scott Lutieri G. wrote: > Sorry. > > # uname -a > FreeBSD sd.xyz.com.br 6.2-STABLE FreeBSD 6.2-STABLE #0: Wed Aug 29 > 10:26:18 BRT 2007 > root@sd.xyz.com.br:/usr/src/sys/amd64/compile/LGB amd64 > > Yesterday i've downloaded kernel from cvsup16.FreeBSD.org and compile it. > > > 2007/8/29, Scott Long : >> You forgot to say which version of FreeBSD you are using. >> >> Scott >> >> >> Lutieri G. wrote: >>> Hi! >>> >>> I have a problem with my new server sun >>> x4100(http://www.sun.com/servers/entry/x4100/specs.xml). >>> >>> Ffirmware is 6.14.04.00 retrivied from sun's site. >>> >>> I cannot get more than 3 MB/s when writing to it. >>> >>> #pciconf -lv >>> --- CUT --- >>> mpt0@pci2:3:0: class=0x010000 card=0x30601000 chip=0x00501000 rev=0x02 hdr=0x00 >>> vendor = 'LSI Logic (Was: Symbios Logic, NCR)' >>> class = mass storage >>> subclass = SCSI >>> --- CUT --- >>> >>> >>> when i run blogbench, squid -z, or any other software systat return >>> da0 with 3.0MB/s and 1005 busy. >>> >>> I have tried both with and without RAID configured through the >>> controller and i get same results. >>> >>> Some times my dmesg return this message : >>> >>> Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 65 >>> Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 37 >>> Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:15 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 65 >>> Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 65 >>> Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 65 >>> Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:17 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 62 >>> Aug 29 15:22:18 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 65 >>> Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x04 Depth 65 >>> Aug 29 15:22:29 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> Aug 29 15:22:47 sdfirewall kernel: mpt0: QUEUE FULL EVENT: Bus 0x00 >>> Target 0x03 Depth 65 >>> >>> >>> Is this a known issue ? >>> >>> tanx >>> >> > > From owner-freebsd-scsi@FreeBSD.ORG Wed Aug 29 22:07:02 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 50E2516A46C for ; Wed, 29 Aug 2007 22:07:02 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.171]) by mx1.freebsd.org (Postfix) with ESMTP id C18EB13C478 for ; Wed, 29 Aug 2007 22:07:01 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: by ug-out-1314.google.com with SMTP id a2so166783ugf for ; Wed, 29 Aug 2007 15:07:00 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=CqyHPCvqAgDUsBULEccrmSnchAU7w9811oxyINdM/pT+ye3HINCmLiIfuBI7jMyXty4vLPS3fy9KwXeIP0r+8azHimWdULJfvu+5edYD7zrVuDgs47EahqbknrEgqfc0v6HSGKD4ggURH33OdqZ339clpFD527zMKEctY8sWJUU= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=g+zjP8s56OE9UQYi5uSki+sOg9UBRNbKaz+s7sz0HmfhnFmIlMFk3n02y7IWzN5Ml/vleS+qStO/8CPzS1MZ/9yKHBvzA8EJHGaT0wNsquAv8L3qqGeSoDe6PKNePbsPVNC7NAgrj5jkLr8l39/ujdRxfNH4s97BYxguk1PiZUk= Received: by 10.142.76.4 with SMTP id y4mr57890wfa.1188425218772; Wed, 29 Aug 2007 15:06:58 -0700 (PDT) Received: by 10.142.101.15 with HTTP; Wed, 29 Aug 2007 15:06:58 -0700 (PDT) Message-ID: <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> Date: Wed, 29 Aug 2007 19:06:58 -0300 From: "Lutieri G." To: "Scott Long" In-Reply-To: <46D5E5B1.207@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 29 Aug 2007 22:07:02 -0000 I've make a test with dd command: # time dd if=/dev/zero of=./8gbfile bs=1024k count=8192 8192+0 records in 8192+0 records out 8589934592 bytes transferred in 155.653213 secs (55186362 bytes/sec) 0.007u 25.129s 2:35.69 16.1% 55+6039k 117+68628io 0pf+0w in other terminal i ran iostat while dd were running and I get this: # iostat -I 1 tty da0 pass0 cpu tin tout KB/t xfrs MB KB/t xfrs MB us ni sy in id 1 61 25.23 52958 1305.07 0.00 0 0.00 0 0 8 0 91 0 184 127.48 434 54.03 0.00 0 0.00 0 0 5 0 95 0 61 127.49 440 54.78 0.00 0 0.00 0 0 5 0 95 0 61 127.75 445 55.52 0.00 0 0.00 0 0 5 0 95 0 61 127.49 442 55.03 0.00 0 0.00 0 0 4 0 96 0 61 127.49 436 54.28 0.00 0 0.00 0 0 5 0 95 0 61 125.27 425 51.99 0.00 0 0.00 0 0 5 1 94 0 61 118.14 393 45.34 0.00 0 0.00 0 0 3 0 97 average 54MB/s with or without hw.mpt.enable_sata_wc seted in loader.conf file. is it a normal speed for this adapter?! -- Att. Lutieri G. B. From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 03:48:16 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 8982016A417 for ; Thu, 30 Aug 2007 03:48:16 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 5210E13C45E for ; Thu, 30 Aug 2007 03:48:11 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (r74-193-81-203.pfvlcmta01.grtntx.tl.dh.suddenlink.net [74.193.81.203]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l7U3Ijct074340; Wed, 29 Aug 2007 22:18:47 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <46D63710.1020103@freebsd.org> Date: Wed, 29 Aug 2007 22:18:40 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: "Lutieri G." References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> In-Reply-To: <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 03:48:16 -0000 Lutieri G. wrote: > I've make a test with dd command: > > # time dd if=/dev/zero of=./8gbfile bs=1024k count=8192 > 8192+0 records in > 8192+0 records out > 8589934592 bytes transferred in 155.653213 secs (55186362 bytes/sec) > 0.007u 25.129s 2:35.69 16.1% 55+6039k 117+68628io 0pf+0w > > in other terminal i ran iostat while dd were running and I get this: > > # iostat -I 1 > tty da0 pass0 cpu > tin tout KB/t xfrs MB KB/t xfrs MB us ni sy in id > 1 61 25.23 52958 1305.07 0.00 0 0.00 0 0 8 0 91 > 0 184 127.48 434 54.03 0.00 0 0.00 0 0 5 0 95 > 0 61 127.49 440 54.78 0.00 0 0.00 0 0 5 0 95 > 0 61 127.75 445 55.52 0.00 0 0.00 0 0 5 0 95 > 0 61 127.49 442 55.03 0.00 0 0.00 0 0 4 0 96 > 0 61 127.49 436 54.28 0.00 0 0.00 0 0 5 0 95 > 0 61 125.27 425 51.99 0.00 0 0.00 0 0 5 1 94 > 0 61 118.14 393 45.34 0.00 0 0.00 0 0 3 0 97 > > average 54MB/s with or without hw.mpt.enable_sata_wc seted in loader.conf file. > > is it a normal speed for this adapter?! > > I'm confused - you said in your first post you were getting 3MB/s, where above you show something like 55MB/s. You didn't say what kind of disks, or how many, the configuration, etc - so it's hard to answer much. The 55MB/s seems pretty decent for many hard drives in a sequential use state (which is what dd tests really). Your errors before were probably caused because your queue depth is set to 255 (or 256?) and the adapter can't do that many. You should use camcontrol to reduce it, to maybe 32. See the camcontrol man page for the right usage. It's something that needs setting on every boot, so a startup file is a good place for it maybe. Eric From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 03:58:00 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 024D516A41A for ; Thu, 30 Aug 2007 03:58:00 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 9B6A213C45D for ; Thu, 30 Aug 2007 03:57:59 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7U3vUKI035379; Wed, 29 Aug 2007 21:57:31 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D64020.3000503@samsco.org> Date: Wed, 29 Aug 2007 21:57:20 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: Eric Anderson References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> In-Reply-To: <46D63710.1020103@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Wed, 29 Aug 2007 21:57:31 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 03:58:00 -0000 Eric Anderson wrote: > Lutieri G. wrote: >> I've make a test with dd command: >> >> # time dd if=/dev/zero of=./8gbfile bs=1024k count=8192 >> 8192+0 records in >> 8192+0 records out >> 8589934592 bytes transferred in 155.653213 secs (55186362 bytes/sec) >> 0.007u 25.129s 2:35.69 16.1% 55+6039k 117+68628io 0pf+0w >> >> in other terminal i ran iostat while dd were running and I get this: >> >> # iostat -I 1 >> tty da0 pass0 cpu >> tin tout KB/t xfrs MB KB/t xfrs MB us ni sy in id >> 1 61 25.23 52958 1305.07 0.00 0 0.00 0 0 8 0 91 >> 0 184 127.48 434 54.03 0.00 0 0.00 0 0 5 0 95 >> 0 61 127.49 440 54.78 0.00 0 0.00 0 0 5 0 95 >> 0 61 127.75 445 55.52 0.00 0 0.00 0 0 5 0 95 >> 0 61 127.49 442 55.03 0.00 0 0.00 0 0 4 0 96 >> 0 61 127.49 436 54.28 0.00 0 0.00 0 0 5 0 95 >> 0 61 125.27 425 51.99 0.00 0 0.00 0 0 5 1 94 >> 0 61 118.14 393 45.34 0.00 0 0.00 0 0 3 0 97 >> >> average 54MB/s with or without hw.mpt.enable_sata_wc seted in >> loader.conf file. >> >> is it a normal speed for this adapter?! >> >> > > > I'm confused - you said in your first post you were getting 3MB/s, where > above you show something like 55MB/s. > > You didn't say what kind of disks, or how many, the configuration, etc - > so it's hard to answer much. The 55MB/s seems pretty decent for many > hard drives in a sequential use state (which is what dd tests really). > > Your errors before were probably caused because your queue depth is set > to 255 (or 256?) and the adapter can't do that many. You should use > camcontrol to reduce it, to maybe 32. See the camcontrol man page for > the right usage. It's something that needs setting on every boot, so a > startup file is a good place for it maybe. > > Eric > > Well, if he's using SATA (which I kinda assumed originally without asking) then queue depth isn't going to matter; the MPT driver has no interaction with how SATA NCQ operates, if he even has a rev of the LSI chip that supports NCQ at all. If he's using SAS, then queue depth will only be a minor factor, CAM is pretty good at autosizing the depth with minimal impact. Now, if he's using SAS disks then the boot tunable that I gave him will indeed have no impact at all. I believe that the Sun 4100 uses 2.5" disks, whether SATA or SAS. 54MB/s is not all that bad for disks of this size. It's pretty close to what I would expect, actually. Scott From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 12:03:30 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 820B816A468 for ; Thu, 30 Aug 2007 12:03:30 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: from wr-out-0506.google.com (wr-out-0506.google.com [64.233.184.236]) by mx1.freebsd.org (Postfix) with ESMTP id 0E73E13C4B0 for ; Thu, 30 Aug 2007 12:03:29 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: by wr-out-0506.google.com with SMTP id 70so298972wra for ; Thu, 30 Aug 2007 05:02:44 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=YqDigTau5U3UyrusZBkjfx4uM8FnlpapOHpxa/QrQlBbnQlJlP0OQPzz/UjWfWD7wQLl3nG24O7LPgOtuu4aVRfTA+It3KTWlbAO51qXqAdBpUo5fl+zBWAb/axEHM1gtyVsZATpURISFBBLsPLi/M6Mh6CW1AYwSfDgtjgR0JY= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=RsFZaPjqrKLkJ3Q5CF0JfugYtxHWfZVfH1KXGoRdHvHbJ2FO9zibH+RKu+652bAPY2ce5SpScNeQxMo9rvGZQzc6FMU+UxUkW7c7bVCjeI62OwmTpLaoPbkwNmKn1VstrGS+nAQwoIPQ/IdQNE+EM8QFxAuOmc5N7me3AKRzCHM= Received: by 10.142.215.5 with SMTP id n5mr19544wfg.1188475363919; Thu, 30 Aug 2007 05:02:43 -0700 (PDT) Received: by 10.142.101.15 with HTTP; Thu, 30 Aug 2007 05:02:43 -0700 (PDT) Message-ID: <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> Date: Thu, 30 Aug 2007 09:02:43 -0300 From: "Lutieri G." To: "Eric Anderson" In-Reply-To: <46D63710.1020103@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 12:03:30 -0000 2007/8/30, Eric Anderson : > > I'm confused - you said in your first post you were getting 3MB/s, where > above you show something like 55MB/s. Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i thinked that 3MB/s is the maximum speed. But i was wrong... > > You didn't say what kind of disks, or how many, the configuration, etc - > so it's hard to answer much. The 55MB/s seems pretty decent for many > hard drives in a sequential use state (which is what dd tests really). > SAS disks. Seagate, i don't know what is the right model of disks. Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with squid cache and maybe should be a problem related with disks. But... i'm investigating and discharging problems. > Your errors before were probably caused because your queue depth is set > to 255 (or 256?) and the adapter can't do that many. You should use > camcontrol to reduce it, to maybe 32. See the camcontrol man page for > the right usage. It's something that needs setting on every boot, so a > startup file is a good place for it maybe. > Is there any way of get the right number to reduce?! > Eric -- Regards Lutieri G. B. From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 12:58:08 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2FBA516A469; Thu, 30 Aug 2007 12:58:08 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id B5D3E13C4A5; Thu, 30 Aug 2007 12:58:07 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7UCvlSl038016; Thu, 30 Aug 2007 06:57:48 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D6BEC0.1050104@samsco.org> Date: Thu, 30 Aug 2007 06:57:36 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: "Lutieri G." References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> In-Reply-To: <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Thu, 30 Aug 2007 06:57:48 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 12:58:08 -0000 Lutieri G. wrote: > 2007/8/30, Eric Anderson : >> I'm confused - you said in your first post you were getting 3MB/s, where >> above you show something like 55MB/s. > Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i > thinked that 3MB/s is the maximum speed. But i was wrong... %busy is a completely useless number for a anything but untagged, uncached disk subsystems. It's only an indirect measure of latency, and there are better tools for measuring latency (gstat). >> You didn't say what kind of disks, or how many, the configuration, etc - >> so it's hard to answer much. The 55MB/s seems pretty decent for many >> hard drives in a sequential use state (which is what dd tests really). >> > SAS disks. Seagate, i don't know what is the right model of disks. > > Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with > squid cache and maybe should be a problem related with disks. But... > i'm investigating and discharging problems. > > >> Your errors before were probably caused because your queue depth is set >> to 255 (or 256?) and the adapter can't do that many. You should use >> camcontrol to reduce it, to maybe 32. See the camcontrol man page for >> the right usage. It's something that needs setting on every boot, so a >> startup file is a good place for it maybe. >> > Is there any way of get the right number to reduce?! > If you're seeing erratic performance in production _AND_ you're seeing lots of accompanying messages on the console about tag depth jumping around, you can use camcontrol to force the depth to a lower number of you're choosing. This kind of problem is pretty rare, though. Scott From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 13:40:44 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 78F4916A420 for ; Thu, 30 Aug 2007 13:40:44 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 5605313C47E for ; Thu, 30 Aug 2007 13:40:44 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (209-163-168-124.static.twtelecom.net [209.163.168.124]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l7UDeINA013907; Thu, 30 Aug 2007 08:40:21 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <46D6C8C2.3000504@freebsd.org> Date: Thu, 30 Aug 2007 08:40:18 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Scott Long References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <46D64020.3000503@samsco.org> In-Reply-To: <46D64020.3000503@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 13:40:44 -0000 Scott Long wrote: > Eric Anderson wrote: >> Lutieri G. wrote: >>> I've make a test with dd command: >>> >>> # time dd if=/dev/zero of=./8gbfile bs=1024k count=8192 >>> 8192+0 records in >>> 8192+0 records out >>> 8589934592 bytes transferred in 155.653213 secs (55186362 bytes/sec) >>> 0.007u 25.129s 2:35.69 16.1% 55+6039k 117+68628io 0pf+0w >>> >>> in other terminal i ran iostat while dd were running and I get this: >>> >>> # iostat -I 1 >>> tty da0 pass0 cpu >>> tin tout KB/t xfrs MB KB/t xfrs MB us ni sy in id >>> 1 61 25.23 52958 1305.07 0.00 0 0.00 0 0 8 0 91 >>> 0 184 127.48 434 54.03 0.00 0 0.00 0 0 5 0 95 >>> 0 61 127.49 440 54.78 0.00 0 0.00 0 0 5 0 95 >>> 0 61 127.75 445 55.52 0.00 0 0.00 0 0 5 0 95 >>> 0 61 127.49 442 55.03 0.00 0 0.00 0 0 4 0 96 >>> 0 61 127.49 436 54.28 0.00 0 0.00 0 0 5 0 95 >>> 0 61 125.27 425 51.99 0.00 0 0.00 0 0 5 1 94 >>> 0 61 118.14 393 45.34 0.00 0 0.00 0 0 3 0 97 >>> >>> average 54MB/s with or without hw.mpt.enable_sata_wc seted in >>> loader.conf file. >>> >>> is it a normal speed for this adapter?! >>> >>> >> >> >> I'm confused - you said in your first post you were getting 3MB/s, >> where above you show something like 55MB/s. >> >> You didn't say what kind of disks, or how many, the configuration, etc >> - so it's hard to answer much. The 55MB/s seems pretty decent for >> many hard drives in a sequential use state (which is what dd tests >> really). >> >> Your errors before were probably caused because your queue depth is >> set to 255 (or 256?) and the adapter can't do that many. You should >> use camcontrol to reduce it, to maybe 32. See the camcontrol man page >> for the right usage. It's something that needs setting on every boot, >> so a startup file is a good place for it maybe. >> >> Eric >> >> > > Well, if he's using SATA (which I kinda assumed originally without > asking) then queue depth isn't going to matter; the MPT driver has no > interaction with how SATA NCQ operates, if he even has a rev of the > LSI chip that supports NCQ at all. If he's using SAS, then queue > depth will only be a minor factor, CAM is pretty good at autosizing > the depth with minimal impact. Now, if he's using SAS disks then > the boot tunable that I gave him will indeed have no impact at all. > > I believe that the Sun 4100 uses 2.5" disks, whether SATA or SAS. > 54MB/s is not all that bad for disks of this size. It's pretty close > to what I would expect, actually. If he's using a SAS Seagate 15k rpm 2.5" drive, he could see much better than 55MB/s. I have an LSI (PCI-X 133, model 1064 as he does) with some Seagate 15k RPM drives, and I can get 100MB/s. Tests were done on FreeBSD 7-CURRENT, with write caching enabled on the drive. Here's some of the numbers: 1k block read size: 6390033 bytes/sec 2k block read size: 12257594 bytes/sec 4k block read size: 23071775 bytes/sec 8k block read size: 41858586 bytes/sec 16k block read size: 71242623 bytes/sec 32k block read size: 108019212 bytes/sec 64k block read size: 108043811 bytes/sec 128k block read size: 108881295 bytes/sec 256k block read size: 108509827 bytes/sec 512k block read size: 108670186 bytes/sec 1024k block read size: 108553724 bytes/sec 1k block write size: 6199645 bytes/sec 2k block write size: 12169083 bytes/sec 4k block write size: 23246388 bytes/sec 8k block write size: 42133751 bytes/sec 16k block write size: 70968273 bytes/sec 32k block write size: 108306742 bytes/sec 64k block write size: 108142985 bytes/sec 128k block write size: 109156720 bytes/sec 256k block write size: 109164252 bytes/sec 512k block write size: 109795665 bytes/sec 1024k block write size: 109202660 bytes/sec Doing latency tests on /dev/da0 /dev/da0 512 # sectorsize 73407820800 # mediasize in bytes (68G) 143374650 # mediasize in sectors 8924 # Cylinders according to firmware. 255 # Heads according to firmware. 63 # Sectors according to firmware. Seek times: Full stroke: 250 iter in 1.943102 sec = 7.772 msec Half stroke: 250 iter in 1.567395 sec = 6.270 msec Quarter stroke: 500 iter in 1.671074 sec = 3.342 msec Short forward: 400 iter in 1.159763 sec = 2.899 msec Short backward: 400 iter in 1.014000 sec = 2.535 msec Seq outer: 2048 iter in 0.322013 sec = 0.157 msec Seq inner: 2048 iter in 0.493810 sec = 0.241 msec Transfer rates: outside: 102400 kbytes in 0.968299 sec = 105752 kbytes/sec middle: 102400 kbytes in 1.088158 sec = 94104 kbytes/sec inside: 102400 kbytes in 1.361127 sec = 75232 kbytes/sec Doing read tests on /dev/da0 READ: 1k block size: 102400+0 records in 102400+0 records out 104857600 bytes transferred in 16.409555 secs (6390033 bytes/sec) READ: 2k block size: 51200+0 records in 51200+0 records out 104857600 bytes transferred in 8.554501 secs (12257594 bytes/sec) READ: 4k block size: 25600+0 records in 25600+0 records out 104857600 bytes transferred in 4.544843 secs (23071775 bytes/sec) READ: 8k block size: 12800+0 records in 12800+0 records out 104857600 bytes transferred in 2.505044 secs (41858586 bytes/sec) READ: 16k block size: 6400+0 records in 6400+0 records out 104857600 bytes transferred in 1.471838 secs (71242623 bytes/sec) READ: 32k block size: 3200+0 records in 3200+0 records out 104857600 bytes transferred in 0.970731 secs (108019212 bytes/sec) READ: 64k block size: 1600+0 records in 1600+0 records out 104857600 bytes transferred in 0.970510 secs (108043811 bytes/sec) READ: 128k block size: 800+0 records in 800+0 records out 104857600 bytes transferred in 0.963045 secs (108881295 bytes/sec) READ: 256k block size: 400+0 records in 400+0 records out 104857600 bytes transferred in 0.966342 secs (108509827 bytes/sec) READ: 512k block size: 200+0 records in 200+0 records out 104857600 bytes transferred in 0.964916 secs (108670186 bytes/sec) READ: 1024k block size: 100+0 records in 100+0 records out 104857600 bytes transferred in 0.965951 secs (108553724 bytes/sec) Doing write tests on /dev/da0 WRITE: 1k block size: 102400+0 records in 102400+0 records out 104857600 bytes transferred in 16.913484 secs (6199645 bytes/sec) WRITE: 2k block size: 51200+0 records in 51200+0 records out 104857600 bytes transferred in 8.616722 secs (12169083 bytes/sec) WRITE: 4k block size: 25600+0 records in 25600+0 records out 104857600 bytes transferred in 4.510705 secs (23246388 bytes/sec) WRITE: 8k block size: 12800+0 records in 12800+0 records out 104857600 bytes transferred in 2.488684 secs (42133751 bytes/sec) WRITE: 16k block size: 6400+0 records in 6400+0 records out 104857600 bytes transferred in 1.477528 secs (70968273 bytes/sec) WRITE: 32k block size: 3200+0 records in 3200+0 records out 104857600 bytes transferred in 0.968154 secs (108306742 bytes/sec) WRITE: 64k block size: 1600+0 records in 1600+0 records out 104857600 bytes transferred in 0.969620 secs (108142985 bytes/sec) WRITE: 128k block size: 800+0 records in 800+0 records out 104857600 bytes transferred in 0.960615 secs (109156720 bytes/sec) WRITE: 256k block size: 400+0 records in 400+0 records out 104857600 bytes transferred in 0.960549 secs (109164252 bytes/sec) WRITE: 512k block size: 200+0 records in 200+0 records out 104857600 bytes transferred in 0.955025 secs (109795665 bytes/sec) WRITE: 1024k block size: 100+0 records in 100+0 records out 104857600 bytes transferred in 0.960211 secs (109202660 bytes/sec) Tests complete. Eric From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 13:48:40 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CBDC616A418; Thu, 30 Aug 2007 13:48:40 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 7A7B613C459; Thu, 30 Aug 2007 13:48:40 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7UDmCcl038375; Thu, 30 Aug 2007 07:48:12 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D6CA91.3080707@samsco.org> Date: Thu, 30 Aug 2007 07:48:01 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: Eric Anderson References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <46D64020.3000503@samsco.org> <46D6C8C2.3000504@freebsd.org> In-Reply-To: <46D6C8C2.3000504@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Thu, 30 Aug 2007 07:48:13 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 13:48:40 -0000 Eric Anderson wrote: > Scott Long wrote: >> Eric Anderson wrote: >>> Lutieri G. wrote: >>>> I've make a test with dd command: >>>> >>>> # time dd if=/dev/zero of=./8gbfile bs=1024k count=8192 >>>> 8192+0 records in >>>> 8192+0 records out >>>> 8589934592 bytes transferred in 155.653213 secs (55186362 bytes/sec) >>>> 0.007u 25.129s 2:35.69 16.1% 55+6039k 117+68628io 0pf+0w >>>> >>>> in other terminal i ran iostat while dd were running and I get this: >>>> >>>> # iostat -I 1 >>>> tty da0 pass0 cpu >>>> tin tout KB/t xfrs MB KB/t xfrs MB us ni sy in id >>>> 1 61 25.23 52958 1305.07 0.00 0 0.00 0 0 8 0 91 >>>> 0 184 127.48 434 54.03 0.00 0 0.00 0 0 5 0 95 >>>> 0 61 127.49 440 54.78 0.00 0 0.00 0 0 5 0 95 >>>> 0 61 127.75 445 55.52 0.00 0 0.00 0 0 5 0 95 >>>> 0 61 127.49 442 55.03 0.00 0 0.00 0 0 4 0 96 >>>> 0 61 127.49 436 54.28 0.00 0 0.00 0 0 5 0 95 >>>> 0 61 125.27 425 51.99 0.00 0 0.00 0 0 5 1 94 >>>> 0 61 118.14 393 45.34 0.00 0 0.00 0 0 3 0 97 >>>> >>>> average 54MB/s with or without hw.mpt.enable_sata_wc seted in >>>> loader.conf file. >>>> >>>> is it a normal speed for this adapter?! >>>> >>>> >>> >>> >>> I'm confused - you said in your first post you were getting 3MB/s, >>> where above you show something like 55MB/s. >>> >>> You didn't say what kind of disks, or how many, the configuration, >>> etc - so it's hard to answer much. The 55MB/s seems pretty decent >>> for many hard drives in a sequential use state (which is what dd >>> tests really). >>> >>> Your errors before were probably caused because your queue depth is >>> set to 255 (or 256?) and the adapter can't do that many. You should >>> use camcontrol to reduce it, to maybe 32. See the camcontrol man >>> page for the right usage. It's something that needs setting on every >>> boot, so a startup file is a good place for it maybe. >>> >>> Eric >>> >>> >> >> Well, if he's using SATA (which I kinda assumed originally without >> asking) then queue depth isn't going to matter; the MPT driver has no >> interaction with how SATA NCQ operates, if he even has a rev of the >> LSI chip that supports NCQ at all. If he's using SAS, then queue >> depth will only be a minor factor, CAM is pretty good at autosizing >> the depth with minimal impact. Now, if he's using SAS disks then >> the boot tunable that I gave him will indeed have no impact at all. >> >> I believe that the Sun 4100 uses 2.5" disks, whether SATA or SAS. >> 54MB/s is not all that bad for disks of this size. It's pretty close >> to what I would expect, actually. > > If he's using a SAS Seagate 15k rpm 2.5" drive, he could see much better > than 55MB/s. I have an LSI (PCI-X 133, model 1064 as he does) with some > Seagate 15k RPM drives, and I can get 100MB/s. Tests were done on > FreeBSD 7-CURRENT, with write caching enabled on the drive. Here's some > of the numbers: > Ah, I didn't know that Seagate made 15k drives in the 2.5" form factor, I was expecting just 7200 or 10k. Scott From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 13:52:00 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7ED9F16A418 for ; Thu, 30 Aug 2007 13:52:00 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 4F44D13C465 for ; Thu, 30 Aug 2007 13:52:00 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (209-163-168-124.static.twtelecom.net [209.163.168.124]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l7UDpjux030920; Thu, 30 Aug 2007 08:51:48 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <46D6CB71.4030707@freebsd.org> Date: Thu, 30 Aug 2007 08:51:45 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Scott Long References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> In-Reply-To: <46D6BEC0.1050104@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 13:52:00 -0000 Scott Long wrote: > Lutieri G. wrote: >> 2007/8/30, Eric Anderson : >>> I'm confused - you said in your first post you were getting 3MB/s, where >>> above you show something like 55MB/s. >> Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i >> thinked that 3MB/s is the maximum speed. But i was wrong... > > %busy is a completely useless number for a anything but untagged, > uncached disk subsystems. It's only an indirect measure of latency, and > there are better tools for measuring latency (gstat). > >>> You didn't say what kind of disks, or how many, the configuration, etc - >>> so it's hard to answer much. The 55MB/s seems pretty decent for many >>> hard drives in a sequential use state (which is what dd tests really). >>> >> SAS disks. Seagate, i don't know what is the right model of disks. >> >> Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with >> squid cache and maybe should be a problem related with disks. But... >> i'm investigating and discharging problems. >> >> >>> Your errors before were probably caused because your queue depth is set >>> to 255 (or 256?) and the adapter can't do that many. You should use >>> camcontrol to reduce it, to maybe 32. See the camcontrol man page for >>> the right usage. It's something that needs setting on every boot, so a >>> startup file is a good place for it maybe. >>> >> Is there any way of get the right number to reduce?! >> > > If you're seeing erratic performance in production _AND_ you're seeing > lots of accompanying messages on the console about tag depth jumping > around, you can use camcontrol to force the depth to a lower number of > you're choosing. This kind of problem is pretty rare, though. Scott, you are far more of a SCSI guru than I, so please correct me if this is incorrect. Can't you get a good estimate, by knowing the queue depth of the target(s), and dividing it by the number of initiators? So in his case, he has one initiator, and (let's say) one target. If the queue depth of the target (being the Seagate SAS drive) is 128 (see Seagate's paper here: http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf ), then he should have to reduce it down from 25[56] to 128, correct? With QLogic cards connected to a fabric, I saw queue depth issues under heavy load. Eric From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 13:52:55 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 83D4516A4F6 for ; Thu, 30 Aug 2007 13:52:55 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 637B813C459 for ; Thu, 30 Aug 2007 13:52:55 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (209-163-168-124.static.twtelecom.net [209.163.168.124]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l7UDqYU4031104; Thu, 30 Aug 2007 08:52:34 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <46D6CBA2.8000703@freebsd.org> Date: Thu, 30 Aug 2007 08:52:34 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Scott Long References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <46D64020.3000503@samsco.org> <46D6C8C2.3000504@freebsd.org> <46D6CA91.3080707@samsco.org> In-Reply-To: <46D6CA91.3080707@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.7 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 13:52:55 -0000 Scott Long wrote: > Eric Anderson wrote: >> Scott Long wrote: >>> Eric Anderson wrote: >>>> Lutieri G. wrote: >>>>> I've make a test with dd command: >>>>> >>>>> # time dd if=/dev/zero of=./8gbfile bs=1024k count=8192 >>>>> 8192+0 records in >>>>> 8192+0 records out >>>>> 8589934592 bytes transferred in 155.653213 secs (55186362 bytes/sec) >>>>> 0.007u 25.129s 2:35.69 16.1% 55+6039k 117+68628io 0pf+0w >>>>> >>>>> in other terminal i ran iostat while dd were running and I get this: >>>>> >>>>> # iostat -I 1 >>>>> tty da0 pass0 cpu >>>>> tin tout KB/t xfrs MB KB/t xfrs MB us ni sy in id >>>>> 1 61 25.23 52958 1305.07 0.00 0 0.00 0 0 8 0 91 >>>>> 0 184 127.48 434 54.03 0.00 0 0.00 0 0 5 0 95 >>>>> 0 61 127.49 440 54.78 0.00 0 0.00 0 0 5 0 95 >>>>> 0 61 127.75 445 55.52 0.00 0 0.00 0 0 5 0 95 >>>>> 0 61 127.49 442 55.03 0.00 0 0.00 0 0 4 0 96 >>>>> 0 61 127.49 436 54.28 0.00 0 0.00 0 0 5 0 95 >>>>> 0 61 125.27 425 51.99 0.00 0 0.00 0 0 5 1 94 >>>>> 0 61 118.14 393 45.34 0.00 0 0.00 0 0 3 0 97 >>>>> >>>>> average 54MB/s with or without hw.mpt.enable_sata_wc seted in >>>>> loader.conf file. >>>>> >>>>> is it a normal speed for this adapter?! >>>>> >>>>> >>>> >>>> >>>> I'm confused - you said in your first post you were getting 3MB/s, >>>> where above you show something like 55MB/s. >>>> >>>> You didn't say what kind of disks, or how many, the configuration, >>>> etc - so it's hard to answer much. The 55MB/s seems pretty decent >>>> for many hard drives in a sequential use state (which is what dd >>>> tests really). >>>> >>>> Your errors before were probably caused because your queue depth is >>>> set to 255 (or 256?) and the adapter can't do that many. You should >>>> use camcontrol to reduce it, to maybe 32. See the camcontrol man >>>> page for the right usage. It's something that needs setting on >>>> every boot, so a startup file is a good place for it maybe. >>>> >>>> Eric >>>> >>>> >>> >>> Well, if he's using SATA (which I kinda assumed originally without >>> asking) then queue depth isn't going to matter; the MPT driver has no >>> interaction with how SATA NCQ operates, if he even has a rev of the >>> LSI chip that supports NCQ at all. If he's using SAS, then queue >>> depth will only be a minor factor, CAM is pretty good at autosizing >>> the depth with minimal impact. Now, if he's using SAS disks then >>> the boot tunable that I gave him will indeed have no impact at all. >>> >>> I believe that the Sun 4100 uses 2.5" disks, whether SATA or SAS. >>> 54MB/s is not all that bad for disks of this size. It's pretty close >>> to what I would expect, actually. >> >> If he's using a SAS Seagate 15k rpm 2.5" drive, he could see much >> better than 55MB/s. I have an LSI (PCI-X 133, model 1064 as he does) >> with some Seagate 15k RPM drives, and I can get 100MB/s. Tests were >> done on FreeBSD 7-CURRENT, with write caching enabled on the drive. >> Here's some of the numbers: >> > > Ah, I didn't know that Seagate made 15k drives in the 2.5" form factor, > I was expecting just 7200 or 10k. They are the only ones to my knowledge. These little drives are *FAST*. Eric From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 14:37:15 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BEB3B16A417 for ; Thu, 30 Aug 2007 14:37:15 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: from ug-out-1314.google.com (ug-out-1314.google.com [66.249.92.170]) by mx1.freebsd.org (Postfix) with ESMTP id 479B713C468 for ; Thu, 30 Aug 2007 14:37:14 +0000 (UTC) (envelope-from lutierigbtrabalho@gmail.com) Received: by ug-out-1314.google.com with SMTP id a2so201803ugf for ; Thu, 30 Aug 2007 07:37:13 -0700 (PDT) DKIM-Signature: a=rsa-sha1; c=relaxed/relaxed; d=gmail.com; s=beta; h=domainkey-signature:received:received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=Wi3n+gcMTacyhwYt9KxHw4EaJ8ek555J5zRbsA2X1z7lc0c/NiYhV8k3NekLp2ZoTr6efx+587s7yDiUON+Hl84XpviBzJtEwAJBg6sFGDL5lrk6F2eRQvRJ9lLc8Ym66Dm/3CtgKQBQLWDG5YJfCrRit0AQY4waYhXbLjeGmsw= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=beta; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=jst+T167+dG/oLEY7MKJclE7Yh6SyCSmuCcVs4wyj106A+yEv/PeizuJHPDLZQ0BKTyWSyqFdRQzWwG//o8/bddCmxgf7JADVxUKGn0xesdQONzL51x/zcNCrHTTZn3l9OcZqIWe4cSx+3w8fAVvp47O1PhPHRBVpqtMM3yC3js= Received: by 10.143.12.19 with SMTP id p19mr24356wfi.1188484632262; Thu, 30 Aug 2007 07:37:12 -0700 (PDT) Received: by 10.142.101.15 with HTTP; Thu, 30 Aug 2007 07:37:12 -0700 (PDT) Message-ID: <71d0ebb0708300737o4fc7966dj61cf0e68482da398@mail.gmail.com> Date: Thu, 30 Aug 2007 11:37:12 -0300 From: "Lutieri G." To: "Eric Anderson" In-Reply-To: <46D6CB71.4030707@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> <46D6CB71.4030707@freebsd.org> Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 14:37:15 -0000 This is my disks: Seagate Savvio(ST913401ss) 10K.1 SAS 3Gb/s 73-GB Hard Drive. In the manual file i found this information: Queue tagging (up to 64 queue tags supported) Is this the max # for setting using camcontrol?! syntax like this: camcontrol tags da0 -N 64 ?? 2007/8/30, Eric Anderson : > Scott Long wrote: > > Lutieri G. wrote: > >> 2007/8/30, Eric Anderson : > >>> I'm confused - you said in your first post you were getting 3MB/s, where > >>> above you show something like 55MB/s. > >> Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i > >> thinked that 3MB/s is the maximum speed. But i was wrong... > > > > %busy is a completely useless number for a anything but untagged, > > uncached disk subsystems. It's only an indirect measure of latency, and > > there are better tools for measuring latency (gstat). > > > >>> You didn't say what kind of disks, or how many, the configuration, etc - > >>> so it's hard to answer much. The 55MB/s seems pretty decent for many > >>> hard drives in a sequential use state (which is what dd tests really). > >>> > >> SAS disks. Seagate, i don't know what is the right model of disks. > >> > >> Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with > >> squid cache and maybe should be a problem related with disks. But... > >> i'm investigating and discharging problems. > >> > >> > >>> Your errors before were probably caused because your queue depth is set > >>> to 255 (or 256?) and the adapter can't do that many. You should use > >>> camcontrol to reduce it, to maybe 32. See the camcontrol man page for > >>> the right usage. It's something that needs setting on every boot, so a > >>> startup file is a good place for it maybe. > >>> > >> Is there any way of get the right number to reduce?! > >> > > > > If you're seeing erratic performance in production _AND_ you're seeing > > lots of accompanying messages on the console about tag depth jumping > > around, you can use camcontrol to force the depth to a lower number of > > you're choosing. This kind of problem is pretty rare, though. > > Scott, you are far more of a SCSI guru than I, so please correct me if > this is incorrect. Can't you get a good estimate, by knowing the queue > depth of the target(s), and dividing it by the number of initiators? So > in his case, he has one initiator, and (let's say) one target. If the > queue depth of the target (being the Seagate SAS drive) is 128 (see > Seagate's paper here: > http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf > ), then he should have to reduce it down from 25[56] to 128, correct? > > With QLogic cards connected to a fabric, I saw queue depth issues under > heavy load. > > Eric > > > > > -- Att. Lutieri G. B. From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 14:51:31 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id EACF216A417; Thu, 30 Aug 2007 14:51:31 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 96E4B13C46B; Thu, 30 Aug 2007 14:51:31 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7UEp9G3038749; Thu, 30 Aug 2007 08:51:10 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D6D952.40305@samsco.org> Date: Thu, 30 Aug 2007 08:50:58 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: Eric Anderson References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> <46D6CB71.4030707@freebsd.org> In-Reply-To: <46D6CB71.4030707@freebsd.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Thu, 30 Aug 2007 08:51:10 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 14:51:32 -0000 Eric Anderson wrote: > Scott Long wrote: >> Lutieri G. wrote: >>> 2007/8/30, Eric Anderson : >>>> I'm confused - you said in your first post you were getting 3MB/s, >>>> where >>>> above you show something like 55MB/s. >>> Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i >>> thinked that 3MB/s is the maximum speed. But i was wrong... >> >> %busy is a completely useless number for a anything but untagged, >> uncached disk subsystems. It's only an indirect measure of latency, and >> there are better tools for measuring latency (gstat). >> >>>> You didn't say what kind of disks, or how many, the configuration, >>>> etc - >>>> so it's hard to answer much. The 55MB/s seems pretty decent for many >>>> hard drives in a sequential use state (which is what dd tests really). >>>> >>> SAS disks. Seagate, i don't know what is the right model of disks. >>> >>> Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with >>> squid cache and maybe should be a problem related with disks. But... >>> i'm investigating and discharging problems. >>> >>> >>>> Your errors before were probably caused because your queue depth is set >>>> to 255 (or 256?) and the adapter can't do that many. You should use >>>> camcontrol to reduce it, to maybe 32. See the camcontrol man page for >>>> the right usage. It's something that needs setting on every boot, so a >>>> startup file is a good place for it maybe. >>>> >>> Is there any way of get the right number to reduce?! >>> >> >> If you're seeing erratic performance in production _AND_ you're seeing >> lots of accompanying messages on the console about tag depth jumping >> around, you can use camcontrol to force the depth to a lower number of >> you're choosing. This kind of problem is pretty rare, though. > > Scott, you are far more of a SCSI guru than I, so please correct me if > this is incorrect. Can't you get a good estimate, by knowing the queue > depth of the target(s), and dividing it by the number of initiators? So > in his case, he has one initiator, and (let's say) one target. If the > queue depth of the target (being the Seagate SAS drive) is 128 (see > Seagate's paper here: > http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf > ), then he should have to reduce it down from 25[56] to 128, correct? > > With QLogic cards connected to a fabric, I saw queue depth issues under > heavy load. > I understand what you're saying, but you're a bit confused on terminology =-) There are two factors in the calculation. One is how many transactions the controller (the initiator) can have in progress as once. This is really independent of what the disks are capable of or how many disks are on the bus. This is normally known to the driver in some chip-specific way. Second is how many tagged transactions a disk can handle. This actually isn't something that can be discovered in a generic way, so the SCSI layer in FreeBSD guesses, and then revises that guess over time based on feedback from the drive. Manually setting the queue depth is not something that he "should have to [do]". It perfectly normal to get console messages on occasion about the OS re-adjusting the depth. Where it becomes a problem is in high latency topologies (like FC fabrics) and buggy drive firmware where the algorithm winds up thrashing a bit. For direct attached SAS disks, I highly doubt that it is needed. Playing a guessing game with this will almost certainly result in lower performance. Scott From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 14:53:16 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id BB56E16A418; Thu, 30 Aug 2007 14:53:16 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 63E6C13C459; Thu, 30 Aug 2007 14:53:16 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.samsco.home (phobos.samsco.home [192.168.254.11]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id l7UEr2pA038770; Thu, 30 Aug 2007 08:53:02 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <46D6D9C3.6050202@samsco.org> Date: Thu, 30 Aug 2007 08:52:51 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.6) Gecko/20070802 SeaMonkey/1.1.4 MIME-Version: 1.0 To: "Lutieri G." References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> <46D6CB71.4030707@freebsd.org> <71d0ebb0708300737o4fc7966dj61cf0e68482da398@mail.gmail.com> In-Reply-To: <71d0ebb0708300737o4fc7966dj61cf0e68482da398@mail.gmail.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (pooker.samsco.org [168.103.85.57]); Thu, 30 Aug 2007 08:53:02 -0600 (MDT) X-Spam-Status: No, score=-1.4 required=5.5 tests=ALL_TRUSTED autolearn=failed version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 14:53:16 -0000 54MB/s is reasonable for 10k 2.5" disks. You might be able to squeeze some more performance by upgrading to FreeBSD 7.0. I _do_not_ recommend playing with the queue depth controls unless your console logs are getting quickly filled with messages about it. Scott Lutieri G. wrote: > This is my disks: > > Seagate Savvio(ST913401ss) 10K.1 SAS 3Gb/s 73-GB Hard Drive. In the > manual file i found this information: > > Queue tagging (up to 64 queue tags supported) > > Is this the max # for setting using camcontrol?! syntax like this: > camcontrol tags da0 -N 64 ?? > > 2007/8/30, Eric Anderson : >> Scott Long wrote: >>> Lutieri G. wrote: >>>> 2007/8/30, Eric Anderson : >>>>> I'm confused - you said in your first post you were getting 3MB/s, where >>>>> above you show something like 55MB/s. >>>> Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i >>>> thinked that 3MB/s is the maximum speed. But i was wrong... >>> %busy is a completely useless number for a anything but untagged, >>> uncached disk subsystems. It's only an indirect measure of latency, and >>> there are better tools for measuring latency (gstat). >>> >>>>> You didn't say what kind of disks, or how many, the configuration, etc - >>>>> so it's hard to answer much. The 55MB/s seems pretty decent for many >>>>> hard drives in a sequential use state (which is what dd tests really). >>>>> >>>> SAS disks. Seagate, i don't know what is the right model of disks. >>>> >>>> Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with >>>> squid cache and maybe should be a problem related with disks. But... >>>> i'm investigating and discharging problems. >>>> >>>> >>>>> Your errors before were probably caused because your queue depth is set >>>>> to 255 (or 256?) and the adapter can't do that many. You should use >>>>> camcontrol to reduce it, to maybe 32. See the camcontrol man page for >>>>> the right usage. It's something that needs setting on every boot, so a >>>>> startup file is a good place for it maybe. >>>>> >>>> Is there any way of get the right number to reduce?! >>>> >>> If you're seeing erratic performance in production _AND_ you're seeing >>> lots of accompanying messages on the console about tag depth jumping >>> around, you can use camcontrol to force the depth to a lower number of >>> you're choosing. This kind of problem is pretty rare, though. >> Scott, you are far more of a SCSI guru than I, so please correct me if >> this is incorrect. Can't you get a good estimate, by knowing the queue >> depth of the target(s), and dividing it by the number of initiators? So >> in his case, he has one initiator, and (let's say) one target. If the >> queue depth of the target (being the Seagate SAS drive) is 128 (see >> Seagate's paper here: >> http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf >> ), then he should have to reduce it down from 25[56] to 128, correct? >> >> With QLogic cards connected to a fabric, I saw queue depth issues under >> heavy load. >> >> Eric >> >> >> >> >> > > From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 16:21:01 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C240616A46E for ; Thu, 30 Aug 2007 16:21:01 +0000 (UTC) (envelope-from ken@nargothrond.kdm.org) Received: from nargothrond.kdm.org (nargothrond.kdm.org [70.56.43.81]) by mx1.freebsd.org (Postfix) with ESMTP id 3F75413C483 for ; Thu, 30 Aug 2007 16:21:01 +0000 (UTC) (envelope-from ken@nargothrond.kdm.org) Received: from nargothrond.kdm.org (localhost [127.0.0.1]) by nargothrond.kdm.org (8.13.6/8.13.6) with ESMTP id l7UG4gla050765; Thu, 30 Aug 2007 10:04:42 -0600 (MDT) (envelope-from ken@nargothrond.kdm.org) Received: (from ken@localhost) by nargothrond.kdm.org (8.13.6/8.13.6/Submit) id l7UG4fxr050764; Thu, 30 Aug 2007 10:04:41 -0600 (MDT) (envelope-from ken) Date: Thu, 30 Aug 2007 10:04:41 -0600 From: "Kenneth D. Merry" To: Scott Long Message-ID: <20070830160441.GA50706@nargothrond.kdm.org> References: <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> <46D6CB71.4030707@freebsd.org> <71d0ebb0708300737o4fc7966dj61cf0e68482da398@mail.gmail.com> <46D6D9C3.6050202@samsco.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <46D6D9C3.6050202@samsco.org> User-Agent: Mutt/1.4.2i X-Virus-Scanned: ClamAV 0.88.1/4108/Thu Aug 30 04:58:53 2007 on nargothrond.kdm.org X-Virus-Status: Clean Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 16:21:01 -0000 The "tagged openings now ..." message was hidden under bootverbose by Jordan 8 years ago. :) So it won't show up on the console, unless you boot with -v. Ken On Thu, Aug 30, 2007 at 08:52:51 -0600, Scott Long wrote: > 54MB/s is reasonable for 10k 2.5" disks. You might be able to squeeze > some more performance by upgrading to FreeBSD 7.0. I _do_not_ recommend > playing with the queue depth controls unless your console logs are > getting quickly filled with messages about it. > > Scott > > > Lutieri G. wrote: > >This is my disks: > > > >Seagate Savvio(ST913401ss) 10K.1 SAS 3Gb/s 73-GB Hard Drive. In the > >manual file i found this information: > > > >Queue tagging (up to 64 queue tags supported) > > > >Is this the max # for setting using camcontrol?! syntax like this: > >camcontrol tags da0 -N 64 ?? > > > >2007/8/30, Eric Anderson : > >>Scott Long wrote: > >>>Lutieri G. wrote: > >>>>2007/8/30, Eric Anderson : > >>>>>I'm confused - you said in your first post you were getting 3MB/s, > >>>>>where > >>>>> above you show something like 55MB/s. > >>>>Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i > >>>>thinked that 3MB/s is the maximum speed. But i was wrong... > >>>%busy is a completely useless number for a anything but untagged, > >>>uncached disk subsystems. It's only an indirect measure of latency, and > >>>there are better tools for measuring latency (gstat). > >>> > >>>>>You didn't say what kind of disks, or how many, the configuration, etc > >>>>>- > >>>>>so it's hard to answer much. The 55MB/s seems pretty decent for many > >>>>>hard drives in a sequential use state (which is what dd tests really). > >>>>> > >>>>SAS disks. Seagate, i don't know what is the right model of disks. > >>>> > >>>>Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with > >>>>squid cache and maybe should be a problem related with disks. But... > >>>>i'm investigating and discharging problems. > >>>> > >>>> > >>>>>Your errors before were probably caused because your queue depth is set > >>>>>to 255 (or 256?) and the adapter can't do that many. You should use > >>>>>camcontrol to reduce it, to maybe 32. See the camcontrol man page for > >>>>>the right usage. It's something that needs setting on every boot, so a > >>>>>startup file is a good place for it maybe. > >>>>> > >>>>Is there any way of get the right number to reduce?! > >>>> > >>>If you're seeing erratic performance in production _AND_ you're seeing > >>>lots of accompanying messages on the console about tag depth jumping > >>>around, you can use camcontrol to force the depth to a lower number of > >>>you're choosing. This kind of problem is pretty rare, though. > >>Scott, you are far more of a SCSI guru than I, so please correct me if > >>this is incorrect. Can't you get a good estimate, by knowing the queue > >>depth of the target(s), and dividing it by the number of initiators? So > >>in his case, he has one initiator, and (let's say) one target. If the > >>queue depth of the target (being the Seagate SAS drive) is 128 (see > >>Seagate's paper here: > >>http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf > >>), then he should have to reduce it down from 25[56] to 128, correct? > >> > >>With QLogic cards connected to a fabric, I saw queue depth issues under > >>heavy load. > >> > >>Eric > >> > >> > >> > >> > >> > > > > > > _______________________________________________ > freebsd-scsi@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-scsi > To unsubscribe, send any mail to "freebsd-scsi-unsubscribe@freebsd.org" -- Kenneth Merry ken@kdm.org From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 19:18:45 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 5BFA616A421 for ; Thu, 30 Aug 2007 19:18:45 +0000 (UTC) (envelope-from ml@netfence.it) Received: from parrot.aev.net (parrot.aev.net [212.31.247.179]) by mx1.freebsd.org (Postfix) with ESMTP id D8A7213C474 for ; Thu, 30 Aug 2007 19:18:44 +0000 (UTC) (envelope-from ml@netfence.it) Received: from soth.ventu ([151.77.224.150]) (authenticated bits=128) by parrot.aev.net (8.14.1/8.13.8) with ESMTP id l7UIu2tO014029 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=FAIL) for ; Thu, 30 Aug 2007 20:56:08 +0200 (CEST) (envelope-from ml@netfence.it) Received: from alamar.ventu (alamar.ventu [10.1.2.18]) by soth.ventu (8.14.1/8.13.8) with ESMTP id l7UIdOp5099635 for ; Thu, 30 Aug 2007 20:39:25 +0200 (CEST) (envelope-from ml@netfence.it) Message-ID: <46D70ECD.4040005@netfence.it> Date: Thu, 30 Aug 2007 20:39:09 +0200 From: Andrea Venturoli User-Agent: Thunderbird 2.0.0.6 (X11/20070806) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-15; format=flowed Content-Transfer-Encoding: 7bit X-Scanned-By: MIMEDefang 2.61 on 212.31.247.179 Subject: Info on SCSI error seriousness X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 19:18:45 -0000 Hello. Today I read this in the logs of a server I manage: > kernel: mly0: logical device 0 (da0) critical > kernel: mly0: physical device 0:0 sense data received > kernel: mly0: sense key 4 asc 01 ascq 00 > kernel: mly0: info 00c06d5f csi 00000000 Still, giving: # camcontrol devlist shows: > at scbus0 target 0 lun 0 (pass0,da0) I'm confused about the interpretation of the above codes: looks like an hardware error, which is explained as "NO INDEX/SECTOR SIGNAL", but that doesn't mean much to me. Any insight? Should I worry about this (unique for now) warning? bye & Thanks av. From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 20:03:02 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A3D7316A41A; Thu, 30 Aug 2007 20:03:02 +0000 (UTC) (envelope-from tom@samplonius.org) Received: from ly.sdf.com (ly.sdf.com [216.113.193.83]) by mx1.freebsd.org (Postfix) with ESMTP id 79EC413C4CB; Thu, 30 Aug 2007 20:03:02 +0000 (UTC) (envelope-from tom@samplonius.org) Received: from localhost (localhost [127.0.0.1]) by ly.sdf.com (Postfix) with ESMTP id 7542B2283A8; Thu, 30 Aug 2007 12:43:28 -0700 (PDT) X-Virus-Scanned: amavisd-new at X-Spam-Score: -3.978 X-Spam-Level: X-Spam-Status: No, score=-3.978 tagged_above=-10 required=6.6 tests=[ALL_TRUSTED=-1.8, AWL=0.421, BAYES_00=-2.599] Received: from ly.sdf.com ([127.0.0.1]) by localhost (ly.sdf.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id XGOkGcFI1VkA; Thu, 30 Aug 2007 12:43:25 -0700 (PDT) Received: from ly.sdf.com (ly.sdf.com [216.113.193.83]) by ly.sdf.com (Postfix) with ESMTP id 5CB2B228392; Thu, 30 Aug 2007 12:43:25 -0700 (PDT) Date: Thu, 30 Aug 2007 12:43:25 -0700 (PDT) From: Tom Samplonius To: "Lutieri G." Message-ID: <17377675.691188503005275.JavaMail.root@ly.sdf.com> In-Reply-To: <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-Originating-IP: [64.251.80.98] Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 20:03:02 -0000 ----- "Lutieri G." wrote: ... > Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with > squid cache and maybe should be a problem related with disks. But... > i'm investigating and discharging problems. ... Probably more related to the fact that Squid has some significant limitations. Use Varnish: http://varnish.projects.linpro.no/wiki/ArchitectNotes Tom From owner-freebsd-scsi@FreeBSD.ORG Thu Aug 30 20:10:52 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B1AAB16A418 for ; Thu, 30 Aug 2007 20:10:52 +0000 (UTC) (envelope-from ulf@alameda.net) Received: from mail.alameda.net (mail.alameda.net [194.55.105.10]) by mx1.freebsd.org (Postfix) with ESMTP id 8B24613C46A for ; Thu, 30 Aug 2007 20:10:52 +0000 (UTC) (envelope-from ulf@alameda.net) Received: by mail.alameda.net (Postfix, from userid 1000) id AAA7C33D33; Thu, 30 Aug 2007 13:10:37 -0700 (PDT) Date: Thu, 30 Aug 2007 13:10:37 -0700 From: Ulf Zimmermann To: Eric Anderson Message-ID: <20070830201037.GR824@evil.alameda.net> References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <46D64020.3000503@samsco.org> <46D6C8C2.3000504@freebsd.org> <46D6CA91.3080707@samsco.org> <46D6CBA2.8000703@freebsd.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <46D6CBA2.8000703@freebsd.org> User-Agent: Mutt/1.4.2.2i Organization: Alameda Networks, Inc. X-Operating-System: FreeBSD 5.3-STABLE X-ANI-MailScanner-Information: Please contact the ISP for more information X-ANI-MailScanner: Found to be clean X-ANI-MailScanner-From: ulf@alameda.net Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: ulf@Alameda.net List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 30 Aug 2007 20:10:52 -0000 On Thu, Aug 30, 2007 at 08:52:34AM -0500, Eric Anderson wrote: > >Ah, I didn't know that Seagate made 15k drives in the 2.5" form factor, > >I was expecting just 7200 or 10k. > > They are the only ones to my knowledge. These little drives are *FAST*. > > Eric HP brands the Seagate ones, there are 2.5" SAS 36GB and 72GB, both in 10K and 15K and then 146GB in 10K currently (at least what HP sells). I have been seeing on Raid 5 and Raid 6 ADG via HP P400 and P800 controllers as much as 180MB/sec during large writes on 4 and 6 drive arrays. I was impressed when I first got my initial servers with SAS. But under Linux I have seen lately some issues I need to check more. Wished I could run more FreeBSD though :-/ -- Regards, Ulf. --------------------------------------------------------------------- Ulf Zimmermann, 1525 Pacific Ave., Alameda, CA-94501, #: 510-865-0204 You can find my resume at: http://www.Alameda.net/~ulf/resume.html From owner-freebsd-scsi@FreeBSD.ORG Fri Aug 31 01:26:53 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 0407216A417 for ; Fri, 31 Aug 2007 01:26:53 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id CAFE413C45A for ; Fri, 31 Aug 2007 01:26:52 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (r74-193-81-203.pfvlcmta01.grtntx.tl.dh.suddenlink.net [74.193.81.203]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l7V1PKr9000977; Thu, 30 Aug 2007 20:25:20 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <46D76DFA.5010106@freebsd.org> Date: Thu, 30 Aug 2007 20:25:14 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Scott Long References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> <46D6CB71.4030707@freebsd.org> <46D6D952.40305@samsco.org> In-Reply-To: <46D6D952.40305@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2007 01:26:53 -0000 Scott Long wrote: > Eric Anderson wrote: >> Scott Long wrote: >>> Lutieri G. wrote: >>>> 2007/8/30, Eric Anderson : >>>>> I'm confused - you said in your first post you were getting 3MB/s, >>>>> where >>>>> above you show something like 55MB/s. >>>> Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i >>>> thinked that 3MB/s is the maximum speed. But i was wrong... >>> >>> %busy is a completely useless number for a anything but untagged, >>> uncached disk subsystems. It's only an indirect measure of latency, and >>> there are better tools for measuring latency (gstat). >>> >>>>> You didn't say what kind of disks, or how many, the configuration, >>>>> etc - >>>>> so it's hard to answer much. The 55MB/s seems pretty decent for many >>>>> hard drives in a sequential use state (which is what dd tests really). >>>>> >>>> SAS disks. Seagate, i don't know what is the right model of disks. >>>> >>>> Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with >>>> squid cache and maybe should be a problem related with disks. But... >>>> i'm investigating and discharging problems. >>>> >>>> >>>>> Your errors before were probably caused because your queue depth is >>>>> set >>>>> to 255 (or 256?) and the adapter can't do that many. You should use >>>>> camcontrol to reduce it, to maybe 32. See the camcontrol man page for >>>>> the right usage. It's something that needs setting on every boot, >>>>> so a >>>>> startup file is a good place for it maybe. >>>>> >>>> Is there any way of get the right number to reduce?! >>>> >>> >>> If you're seeing erratic performance in production _AND_ you're seeing >>> lots of accompanying messages on the console about tag depth jumping >>> around, you can use camcontrol to force the depth to a lower number of >>> you're choosing. This kind of problem is pretty rare, though. >> >> Scott, you are far more of a SCSI guru than I, so please correct me if >> this is incorrect. Can't you get a good estimate, by knowing the >> queue depth of the target(s), and dividing it by the number of >> initiators? So in his case, he has one initiator, and (let's say) one >> target. If the queue depth of the target (being the Seagate SAS >> drive) is 128 (see Seagate's paper here: >> http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf >> ), then he should have to reduce it down from 25[56] to 128, correct? >> >> With QLogic cards connected to a fabric, I saw queue depth issues >> under heavy load. >> > > I understand what you're saying, but you're a bit confused on > terminology =-) Figured as much :) > There are two factors in the calculation. One is how many transactions > the controller (the initiator) can have in progress as once. This is > really independent of what the disks are capable of or how many disks > are on the bus. This is normally known to the driver in some > chip-specific way. Second is how many tagged transactions a disk can > handle. This actually isn't something that can be discovered in a > generic way, so the SCSI layer in FreeBSD guesses, and then revises that > guess over time based on feedback from the drive. > > Manually setting the queue depth is not something that he "should have > to [do]". It perfectly normal to get console messages on occasion about > the OS re-adjusting the depth. Where it becomes a problem is in high > latency topologies (like FC fabrics) and buggy drive firmware where the > algorithm winds up thrashing a bit. For direct attached SAS disks, I > highly doubt that it is needed. Playing a guessing game with this will > almost certainly result in lower performance. Ok, that makes sense - my experience was in a heavily loaded fabric environment. Thanks for the great info! Eric From owner-freebsd-scsi@FreeBSD.ORG Fri Aug 31 01:48:17 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 7DB6B16A419 for ; Fri, 31 Aug 2007 01:48:17 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from ns.trinitel.com (186.161.36.72.static.reverse.ltdomains.com [72.36.161.186]) by mx1.freebsd.org (Postfix) with ESMTP id 3F14513C45B for ; Fri, 31 Aug 2007 01:48:17 +0000 (UTC) (envelope-from anderson@freebsd.org) Received: from proton.local (r74-193-81-203.pfvlcmta01.grtntx.tl.dh.suddenlink.net [74.193.81.203]) (authenticated bits=0) by ns.trinitel.com (8.14.1/8.14.1) with ESMTP id l7V1MaT7000764; Thu, 30 Aug 2007 20:22:37 -0500 (CDT) (envelope-from anderson@freebsd.org) Message-ID: <46D76D56.5070007@freebsd.org> Date: Thu, 30 Aug 2007 20:22:30 -0500 From: Eric Anderson User-Agent: Thunderbird 2.0.0.6 (Macintosh/20070728) MIME-Version: 1.0 To: Scott Long References: <71d0ebb0708291245g79d2141fx73cc8a6e76875944@mail.gmail.com> <46D5E17F.3070403@samsco.org> <71d0ebb0708291416v17351c65u7ccc1b7bbe0271d2@mail.gmail.com> <46D5E5B1.207@samsco.org> <71d0ebb0708291506i49649a60l8006deafb20891ac@mail.gmail.com> <46D63710.1020103@freebsd.org> <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <46D6BEC0.1050104@samsco.org> <46D6CB71.4030707@freebsd.org> <71d0ebb0708300737o4fc7966dj61cf0e68482da398@mail.gmail.com> <46D6D9C3.6050202@samsco.org> In-Reply-To: <46D6D9C3.6050202@samsco.org> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.8 required=5.0 tests=AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on ns.trinitel.com Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2007 01:48:17 -0000 Scott Long wrote: > 54MB/s is reasonable for 10k 2.5" disks. You might be able to squeeze > some more performance by upgrading to FreeBSD 7.0. I _do_not_ recommend > playing with the queue depth controls unless your console logs are > getting quickly filled with messages about it. Yea, 55-65MB/s is about right for that drive.. Also, when I played with the tagged queue depth previously, I never had any issue, and it solved several SCSI (fabric/fiber channel thought) issues I was having. The performance didn't change measurably when changing it down to 64, but below that it did see a performance hit. Eric > Lutieri G. wrote: >> This is my disks: >> >> Seagate Savvio(ST913401ss) 10K.1 SAS 3Gb/s 73-GB Hard Drive. In the >> manual file i found this information: >> >> Queue tagging (up to 64 queue tags supported) >> >> Is this the max # for setting using camcontrol?! syntax like this: >> camcontrol tags da0 -N 64 ?? >> >> 2007/8/30, Eric Anderson : >>> Scott Long wrote: >>>> Lutieri G. wrote: >>>>> 2007/8/30, Eric Anderson : >>>>>> I'm confused - you said in your first post you were getting 3MB/s, >>>>>> where >>>>>> above you show something like 55MB/s. >>>>> Sorry! using blogbench i got 3MB/s and 100% busy. Once is 100% busy i >>>>> thinked that 3MB/s is the maximum speed. But i was wrong... >>>> %busy is a completely useless number for a anything but untagged, >>>> uncached disk subsystems. It's only an indirect measure of latency, >>>> and >>>> there are better tools for measuring latency (gstat). >>>> >>>>>> You didn't say what kind of disks, or how many, the configuration, >>>>>> etc - >>>>>> so it's hard to answer much. The 55MB/s seems pretty decent for many >>>>>> hard drives in a sequential use state (which is what dd tests >>>>>> really). >>>>>> >>>>> SAS disks. Seagate, i don't know what is the right model of disks. >>>>> >>>>> Ok. If 55Mb/s is a decent speed i'm happy. I'm getting problems with >>>>> squid cache and maybe should be a problem related with disks. But... >>>>> i'm investigating and discharging problems. >>>>> >>>>> >>>>>> Your errors before were probably caused because your queue depth >>>>>> is set >>>>>> to 255 (or 256?) and the adapter can't do that many. You should use >>>>>> camcontrol to reduce it, to maybe 32. See the camcontrol man page >>>>>> for >>>>>> the right usage. It's something that needs setting on every boot, >>>>>> so a >>>>>> startup file is a good place for it maybe. >>>>>> >>>>> Is there any way of get the right number to reduce?! >>>>> >>>> If you're seeing erratic performance in production _AND_ you're seeing >>>> lots of accompanying messages on the console about tag depth jumping >>>> around, you can use camcontrol to force the depth to a lower number of >>>> you're choosing. This kind of problem is pretty rare, though. >>> Scott, you are far more of a SCSI guru than I, so please correct me if >>> this is incorrect. Can't you get a good estimate, by knowing the queue >>> depth of the target(s), and dividing it by the number of initiators? So >>> in his case, he has one initiator, and (let's say) one target. If the >>> queue depth of the target (being the Seagate SAS drive) is 128 (see >>> Seagate's paper here: >>> http://www.seagate.com/staticfiles/support/disc/manuals/enterprise/savvio/Savvio%2015K.1/SAS/100407739b.pdf >>> >>> ), then he should have to reduce it down from 25[56] to 128, correct? >>> >>> With QLogic cards connected to a fabric, I saw queue depth issues under >>> heavy load. >>> >>> Eric >>> >>> >>> >>> >>> >> >> > From owner-freebsd-scsi@FreeBSD.ORG Fri Aug 31 14:48:50 2007 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 9695C16A418 for ; Fri, 31 Aug 2007 14:48:50 +0000 (UTC) (envelope-from dwmalone@maths.tcd.ie) Received: from salmon.maths.tcd.ie (salmon.maths.tcd.ie [IPv6:2001:770:10:300::86e2:510b]) by mx1.freebsd.org (Postfix) with SMTP id 854AA13C428 for ; Fri, 31 Aug 2007 14:48:49 +0000 (UTC) (envelope-from dwmalone@maths.tcd.ie) Received: from walton.maths.tcd.ie ([134.226.81.10] helo=walton.maths.tcd.ie) by salmon.maths.tcd.ie with SMTP id ; 31 Aug 2007 15:48:47 +0100 (BST) Date: Fri, 31 Aug 2007 15:48:46 +0100 From: David Malone To: Tom Samplonius Message-ID: <20070831144846.GA11028@walton.maths.tcd.ie> References: <71d0ebb0708300502x632fe83bo617f84ca2008dc7d@mail.gmail.com> <17377675.691188503005275.JavaMail.root@ly.sdf.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <17377675.691188503005275.JavaMail.root@ly.sdf.com> User-Agent: Mutt/1.5.6i Sender: dwmalone@maths.tcd.ie Cc: FREEBSD - SCSI - LIST Subject: Re: performance with LSI SAS 1064 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 31 Aug 2007 14:48:50 -0000 On Thu, Aug 30, 2007 at 12:43:25PM -0700, Tom Samplonius wrote: > Probably more related to the fact that Squid has some significant limitations. > Use Varnish: http://varnish.projects.linpro.no/wiki/ArchitectNotes I thought that Varnish was only suitable as a reverse proxy? http://varnish.projects.linpro.no/wiki/FAQ#DoesthatmeanIcantuseVarnishasaforwardproxy (Though it isn't clear to me if Squid is being used as a forward or reverse proxy in this case). David.