From owner-freebsd-scsi@FreeBSD.ORG Mon May 19 11:06:59 2008 Return-Path: Delivered-To: freebsd-scsi@FreeBSD.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D48511065674 for ; Mon, 19 May 2008 11:06:59 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:4f8:fff6::28]) by mx1.freebsd.org (Postfix) with ESMTP id BB8BD8FC27 for ; Mon, 19 May 2008 11:06:59 +0000 (UTC) (envelope-from owner-bugmaster@FreeBSD.org) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.2/8.14.2) with ESMTP id m4JB6x42011715 for ; Mon, 19 May 2008 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.2/8.14.1/Submit) id m4JB6x9i011711 for freebsd-scsi@FreeBSD.org; Mon, 19 May 2008 11:06:59 GMT (envelope-from owner-bugmaster@FreeBSD.org) Date: Mon, 19 May 2008 11:06:59 GMT Message-Id: <200805191106.m4JB6x9i011711@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: gnats set sender to owner-bugmaster@FreeBSD.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to freebsd-scsi@FreeBSD.org X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 19 May 2008 11:06:59 -0000 Current FreeBSD problem reports Critical problems Serious problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/39388 scsi ncr/sym drivers fail with 53c810 and more than 256MB m o kern/40895 scsi wierd kernel / device driver bug o kern/52638 scsi [panic] SCSI U320 on SMP server won't run faster than s kern/57398 scsi [mly] Current fails to install on mly(4) based RAID di o kern/60598 scsi wire down of scsi devices conflicts with config o kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C810 under load s kern/61165 scsi [panic] kernel page fault after calling cam_send_ccb o kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5.3 o kern/90282 scsi [sym] SCSI bus resets cause loss of ch device o kern/92798 scsi [ahc] SCSI problem with timeouts o kern/94838 scsi Kernel panic while mounting SD card with lock switch o o kern/99954 scsi [ahc] reading from DVD failes on 6.x [regression] o kern/110847 scsi [ahd] Tyan U320 onboard problem with more than 3 disks o kern/120247 scsi [mpt] FreeBSD 6.3 and LSI Logic 1030 = only 3.300MB/s 14 problems total. Non-critical problems S Tracker Resp. Description -------------------------------------------------------------------------------- o kern/35234 scsi World access to /dev/pass? (for scanner) requires acce o kern/38828 scsi [dpt] [request] DPT PM2012B/90 doesn't work o kern/44587 scsi dev/dpt/dpt.h is missing defines required for DPT_HAND o kern/76178 scsi [ahd] Problem with ahd and large SCSI Raid system o kern/114597 scsi [sym] System hangs at SCSI bus reset with dual HBAs o kern/119668 scsi [cam] [patch] certain errors are too verbose comparing o kern/120487 scsi [sg] scsi_sg incompatible with scanners o sparc/121676 scsi [iscsi] iscontrol do not connect iscsi-target on sparc o kern/123666 scsi [aac] aac(4) will not work with Adaptec SAS RAID 3805 o kern/123674 scsi [ahc] ahc driver dumping 10 problems total. From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 12:42:46 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id DC3151065689 for ; Tue, 20 May 2008 12:42:45 +0000 (UTC) (envelope-from warren.guy@calorieking.com) Received: from soyuz.guy.net.au (soyuz.guy.net.au [67.19.57.58]) by mx1.freebsd.org (Postfix) with ESMTP id A76C18FC16 for ; Tue, 20 May 2008 12:42:45 +0000 (UTC) (envelope-from warren.guy@calorieking.com) Received: from [10.1.1.2] (124-169-7-227.dyn.iinet.net.au [124.169.7.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by soyuz.guy.net.au (Postfix) with ESMTPSA id 9C64B5C01F for ; Tue, 20 May 2008 20:26:40 +0800 (WST) Message-ID: <4832C397.3090004@calorieking.com> Date: Tue, 20 May 2008 20:27:03 +0800 From: Warren Guy User-Agent: Mozilla-Thunderbird 2.0.0.12 (X11/20080420) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org X-Enigmail-Version: 0.95.0 OpenPGP: id=E7B1C075 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enigB7F1168B8B1B8686DE54F248" Subject: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 12:42:46 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enigB7F1168B8B1B8686DE54F248 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Hi, We have a few recently acquired Dell Poweredge SC440 machines in our offi= ce running FreeBSD 6.3 that are experiencing very poor disk performance. The= RAID controller they are supplied with appears to be an "LSI Logic SAS 3000 Se= ries". There are two 250GB SATA disks in RAID 1 configuration. 6-7MB/s seems to = be the best I can get out of them. A Linux 2.6 machine with identical hardware a= nd disk/controller configuration exhibits no such performance impediments. Any information or pointers are greatly appreciated. bonnie++ output: Version 1.93c ------Sequential Output------ --Sequential Input- --R= andom- Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --S= eeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /s= ec %CP eva.internal 300M 306 99 6722 1 5591 1 533 99 +++++ +++ 23= 59 37 Latency 26715us 157ms 2193ms 25385us 808us 38= 52us Version 1.93c ------Sequential Create------ --------Random Create--= ------ eva.internal -Create-- --Read--- -Delete-- -Create-- --Read--- -De= lete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /s= ec %CP 16 4572 11 +++++ +++ +++++ +++ 3019 7 +++++ +++ 283= 67 40 Latency 1277ms 23us 33us 1396ms 92us 3= 17ms 1.93c,1.93c,eva.internal,1,1211281869,300M,,306,99,6722,1,5591,1,533,99,+= ++++,+++,2359,37,16,,,,,4572,11,+++++,+++,+++++,+++,3019,7,+++++,+++,2836= 7,40,26715us,157ms,2193ms,25385us,808us,3852us,1277ms,23us,33us,1396ms,92= us,317ms and bonnie++ from the Linux 2.6 machine: Version 1.03 ------Sequential Output------ --Sequential Input- --R= andom- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --S= eeks-- Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /s= ec %CP webvms 300M 41255 78 114551 26 565402 88 52867 94 +++++ +++ += ++++ +++ ------Sequential Create------ --------Random Create--= ------ -Create-- --Read--- -Delete-- -Create-- --Read--- -De= lete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /s= ec %CP 16 3571 95 +++++ +++ +++++ +++ 3543 92 +++++ +++ 107= 87 82 webvms,300M,41255,78,114551,26,565402,88,52867,94,+++++,+++,+++++,+++,16,= 3571,95,+++++,+++,+++++,+++,3543,92,+++++,+++,10787,82 uname: FreeBSD eva.internal 6.3-RELEASE-p1 FreeBSD 6.3-RELEASE-p1 #0: Wed Feb 13= 02:56:56 UTC 2008 root@i386-builder.daemonology.net:/usr/obj/usr/src/= sys/SMP i386 from dmesg: CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.01-MHz 686-cla= ss CPU) real memory =3D 2145894400 (2046 MB) avail memory =3D 2090598400 (1993 MB) =2E.. mpt0: port 0xdc00-0xdcff mem 0xefdec000-0xefdeffff,0xefdf0000-0xefdfffff irq 16 at device 8.0 on pci2 mpt0: [GIANT-LOCKED] mpt0: MPI Version=3D1.5.13.0 mpt0: mpt_cam_event: 0x16 mpt0: Unhandled Event Notify Frame. Event 0x16 (ACK not required). mpt0: mpt_cam_event: 0x12 mpt0: Unhandled Event Notify Frame. Event 0x12 (ACK not required). mpt0: mpt_cam_event: 0x12 mpt0: Unhandled Event Notify Frame. Event 0x12 (ACK not required). mpt0: mpt_cam_event: 0x16 mpt0: Unhandled Event Notify Frame. Event 0x16 (ACK not required). =2E.. da0 at mpt0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-5 device da0: 300.000MB/s transfers, Tagged Queueing Enabled da0: 237464MB (486326272 512 byte sectors: 255H 63S/T 30272C) pciconf: mpt0@pci2:8:0: class=3D0x010000 card=3D0x1f091028 chip=3D0x00541000 rev=3D= 0x01 hdr=3D0x00 vendor =3D 'LSI Logic (Was: Symbios Logic, NCR)' device =3D 'SAS 3000 series, 8-port with 1068 -StorPort' class =3D mass storage subclass =3D SCSI --------------enigB7F1168B8B1B8686DE54F248 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkgyw5cACgkQvGwbGeexwHXH/QCgj9ndcb4cPYA6czu9i3+sJ8cG ilEAoI2gMsWEDSeANE6omjRa/8MCCU2e =YeTT -----END PGP SIGNATURE----- --------------enigB7F1168B8B1B8686DE54F248-- From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 14:32:20 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 3DF4B1065677 for ; Tue, 20 May 2008 14:32:20 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id DD8C68FC1A for ; Tue, 20 May 2008 14:32:19 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local ([192.168.254.200]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id m4KEWEoJ024298; Tue, 20 May 2008 08:32:14 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <4832E0EE.3030402@samsco.org> Date: Tue, 20 May 2008 08:32:14 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Warren Guy References: <4832C397.3090004@calorieking.com> In-Reply-To: <4832C397.3090004@calorieking.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.0 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 14:32:20 -0000 Add the following to /boot/loader.conf: hw.mpt.enable_sata_wc=1 Scott Warren Guy wrote: > Hi, > > We have a few recently acquired Dell Poweredge SC440 machines in our office > running FreeBSD 6.3 that are experiencing very poor disk performance. The RAID > controller they are supplied with appears to be an "LSI Logic SAS 3000 Series". > There are two 250GB SATA disks in RAID 1 configuration. 6-7MB/s seems to be the > best I can get out of them. A Linux 2.6 machine with identical hardware and > disk/controller configuration exhibits no such performance impediments. > > Any information or pointers are greatly appreciated. > > bonnie++ output: > > Version 1.93c ------Sequential Output------ --Sequential Input- --Random- > Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > eva.internal 300M 306 99 6722 1 5591 1 533 99 +++++ +++ 2359 37 > Latency 26715us 157ms 2193ms 25385us 808us 3852us > Version 1.93c ------Sequential Create------ --------Random Create-------- > eva.internal -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 4572 11 +++++ +++ +++++ +++ 3019 7 +++++ +++ 28367 40 > Latency 1277ms 23us 33us 1396ms 92us 317ms > 1.93c,1.93c,eva.internal,1,1211281869,300M,,306,99,6722,1,5591,1,533,99,+++++,+++,2359,37,16,,,,,4572,11,+++++,+++,+++++,+++,3019,7,+++++,+++,28367,40,26715us,157ms,2193ms,25385us,808us,3852us,1277ms,23us,33us,1396ms,92us,317ms > > and bonnie++ from the Linux 2.6 machine: > > Version 1.03 ------Sequential Output------ --Sequential Input- --Random- > -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- > Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP > webvms 300M 41255 78 114551 26 565402 88 52867 94 +++++ +++ +++++ +++ > ------Sequential Create------ --------Random Create-------- > -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- > files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP > 16 3571 95 +++++ +++ +++++ +++ 3543 92 +++++ +++ 10787 82 > webvms,300M,41255,78,114551,26,565402,88,52867,94,+++++,+++,+++++,+++,16,3571,95,+++++,+++,+++++,+++,3543,92,+++++,+++,10787,82 > > > > uname: > FreeBSD eva.internal 6.3-RELEASE-p1 FreeBSD 6.3-RELEASE-p1 #0: Wed Feb 13 > 02:56:56 UTC 2008 root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/SMP > i386 > > from dmesg: > CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.01-MHz 686-class CPU) > real memory = 2145894400 (2046 MB) > avail memory = 2090598400 (1993 MB) > ... > mpt0: port 0xdc00-0xdcff mem > 0xefdec000-0xefdeffff,0xefdf0000-0xefdfffff irq 16 at device 8.0 on pci2 > mpt0: [GIANT-LOCKED] > mpt0: MPI Version=1.5.13.0 > mpt0: mpt_cam_event: 0x16 > mpt0: Unhandled Event Notify Frame. Event 0x16 (ACK not required). > mpt0: mpt_cam_event: 0x12 > mpt0: Unhandled Event Notify Frame. Event 0x12 (ACK not required). > mpt0: mpt_cam_event: 0x12 > mpt0: Unhandled Event Notify Frame. Event 0x12 (ACK not required). > mpt0: mpt_cam_event: 0x16 > mpt0: Unhandled Event Notify Frame. Event 0x16 (ACK not required). > ... > da0 at mpt0 bus 0 target 0 lun 0 > da0: Fixed Direct Access SCSI-5 device > da0: 300.000MB/s transfers, Tagged Queueing Enabled > da0: 237464MB (486326272 512 byte sectors: 255H 63S/T 30272C) > > pciconf: > mpt0@pci2:8:0: class=0x010000 card=0x1f091028 chip=0x00541000 rev=0x01 hdr=0x00 > vendor = 'LSI Logic (Was: Symbios Logic, NCR)' > device = 'SAS 3000 series, 8-port with 1068 -StorPort' > class = mass storage > subclass = SCSI > From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 14:56:49 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4D5C61065674 for ; Tue, 20 May 2008 14:56:49 +0000 (UTC) (envelope-from warren.guy@calorieking.com) Received: from soyuz.guy.net.au (soyuz.guy.net.au [67.19.57.58]) by mx1.freebsd.org (Postfix) with ESMTP id 2AF378FC19 for ; Tue, 20 May 2008 14:56:48 +0000 (UTC) (envelope-from warren.guy@calorieking.com) Received: from [10.1.1.2] (124-169-7-227.dyn.iinet.net.au [124.169.7.227]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by soyuz.guy.net.au (Postfix) with ESMTPSA id 2DF8A5C024; Tue, 20 May 2008 22:56:46 +0800 (WST) Message-ID: <4832E6C2.7040205@calorieking.com> Date: Tue, 20 May 2008 22:57:06 +0800 From: Warren Guy User-Agent: Mozilla-Thunderbird 2.0.0.12 (X11/20080420) MIME-Version: 1.0 To: Scott Long References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> In-Reply-To: <4832E0EE.3030402@samsco.org> X-Enigmail-Version: 0.95.0 OpenPGP: id=E7B1C075 Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="------------enig25B79A63C07638FB2E43D374" Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 14:56:49 -0000 This is an OpenPGP/MIME signed message (RFC 2440 and 3156) --------------enig25B79A63C07638FB2E43D374 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Scott, Thanks a lot for that. This seems to have alleviated the problem, I'm see= ing decent performance now in my limited benchmark. It seems quite odd to me = that the write cache is not enabled by default, but oh well. Thanks again for your help! Warren Scott Long wrote: > Add the following to /boot/loader.conf: >=20 > hw.mpt.enable_sata_wc=3D1 >=20 >=20 > Scott >=20 >=20 > Warren Guy wrote: >> Hi, >> >> We have a few recently acquired Dell Poweredge SC440 machines in our >> office >> running FreeBSD 6.3 that are experiencing very poor disk performance. >> The RAID >> controller they are supplied with appears to be an "LSI Logic SAS 3000= >> Series". >> There are two 250GB SATA disks in RAID 1 configuration. 6-7MB/s seems >> to be the >> best I can get out of them. A Linux 2.6 machine with identical >> hardware and >> disk/controller configuration exhibits no such performance impediments= =2E >> >> Any information or pointers are greatly appreciated. >> >> bonnie++ output: >> >> Version 1.93c ------Sequential Output------ --Sequential Input- >> --Random- >> Concurrency 1 -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- >> --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = >> /sec %CP >> eva.internal 300M 306 99 6722 1 5591 1 533 99 +++++ +++ = >> 2359 37 >> Latency 26715us 157ms 2193ms 25385us 808us = >> 3852us >> Version 1.93c ------Sequential Create------ --------Random >> Create-------- >> eva.internal -Create-- --Read--- -Delete-- -Create-- --Read--- >> -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = >> /sec %CP >> 16 4572 11 +++++ +++ +++++ +++ 3019 7 +++++ +++ >> 28367 40 >> Latency 1277ms 23us 33us 1396ms =20 >> 92us 317ms >> 1.93c,1.93c,eva.internal,1,1211281869,300M,,306,99,6722,1,5591,1,533,9= 9,+++++,+++,2359,37,16,,,,,4572,11,+++++,+++,+++++,+++,3019,7,+++++,+++,2= 8367,40,26715us,157ms,2193ms,25385us,808us,3852us,1277ms,23us,33us,1396ms= ,92us,317ms >> >> >> and bonnie++ from the Linux 2.6 machine: >> >> Version 1.03 ------Sequential Output------ --Sequential Input- >> --Random- >> -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- >> --Seeks-- >> Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP = >> /sec %CP >> webvms 300M 41255 78 114551 26 565402 88 52867 94 +++++ >> +++ +++++ +++ >> ------Sequential Create------ --------Random >> Create-------- >> -Create-- --Read--- -Delete-- -Create-- --Read--- >> -Delete-- >> files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP = >> /sec %CP >> 16 3571 95 +++++ +++ +++++ +++ 3543 92 +++++ +++ >> 10787 82 >> webvms,300M,41255,78,114551,26,565402,88,52867,94,+++++,+++,+++++,+++,= 16,3571,95,+++++,+++,+++++,+++,3543,92,+++++,+++,10787,82 >> >> >> >> >> uname: >> FreeBSD eva.internal 6.3-RELEASE-p1 FreeBSD 6.3-RELEASE-p1 #0: Wed Feb= 13 >> 02:56:56 UTC 2008 =20 >> root@i386-builder.daemonology.net:/usr/obj/usr/src/sys/SMP >> i386 >> >> from dmesg: >> CPU: Intel(R) Xeon(R) CPU 3050 @ 2.13GHz (2128.01-MHz >> 686-class CPU) >> real memory =3D 2145894400 (2046 MB) >> avail memory =3D 2090598400 (1993 MB) >> ... >> mpt0: port 0xdc00-0xdcff mem >> 0xefdec000-0xefdeffff,0xefdf0000-0xefdfffff irq 16 at device 8.0 on pc= i2 >> mpt0: [GIANT-LOCKED] >> mpt0: MPI Version=3D1.5.13.0 >> mpt0: mpt_cam_event: 0x16 >> mpt0: Unhandled Event Notify Frame. Event 0x16 (ACK not required). >> mpt0: mpt_cam_event: 0x12 >> mpt0: Unhandled Event Notify Frame. Event 0x12 (ACK not required). >> mpt0: mpt_cam_event: 0x12 >> mpt0: Unhandled Event Notify Frame. Event 0x12 (ACK not required). >> mpt0: mpt_cam_event: 0x16 >> mpt0: Unhandled Event Notify Frame. Event 0x16 (ACK not required). >> ... >> da0 at mpt0 bus 0 target 0 lun 0 >> da0: Fixed Direct Access SCSI-5 device >> da0: 300.000MB/s transfers, Tagged Queueing Enabled >> da0: 237464MB (486326272 512 byte sectors: 255H 63S/T 30272C) >> >> pciconf: >> mpt0@pci2:8:0: class=3D0x010000 card=3D0x1f091028 chip=3D0x00541000= >> rev=3D0x01 hdr=3D0x00 >> vendor =3D 'LSI Logic (Was: Symbios Logic, NCR)' >> device =3D 'SAS 3000 series, 8-port with 1068 -StorPort' >> class =3D mass storage >> subclass =3D SCSI >> >=20 --------------enig25B79A63C07638FB2E43D374 Content-Type: application/pgp-signature; name="signature.asc" Content-Description: OpenPGP digital signature Content-Disposition: attachment; filename="signature.asc" -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (GNU/Linux) iEYEARECAAYFAkgy5sUACgkQvGwbGeexwHWutgCgqRL6m98KU0OA1EmZ1aJo3/Z9 FQYAoJw+OxNiEvFHvqVOwsWvG1rdhlSP =oiHN -----END PGP SIGNATURE----- --------------enig25B79A63C07638FB2E43D374-- From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 22:09:49 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 49D391065672 for ; Tue, 20 May 2008 22:09:49 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (host-122-100-2-232.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 114A68FC19 for ; Tue, 20 May 2008 22:09:48 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id C846F1733D; Wed, 21 May 2008 07:53:09 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.163] (142.19.96.58.exetel.com.au [58.96.19.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 4BE3E17236; Wed, 21 May 2008 07:53:05 +1000 (EST) Message-ID: <4833483B.4030208@modulus.org> Date: Wed, 21 May 2008 07:52:59 +1000 From: Andrew Snow User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> In-Reply-To: <4832E6C2.7040205@calorieking.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 22:09:49 -0000 Warren Guy wrote: > Thanks a lot for that. This seems to have alleviated the problem, I'm seeing > decent performance now in my limited benchmark. It seems quite odd to me that > the write cache is not enabled by default, but oh well. Technically with UFS, it can lead to filesystem or database corruption - when the power goes off suddenly, the OS+controller thinks the data has been written, but the drive has it in cache that is not battery-backed. From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 23:06:15 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 975D1106566C for ; Tue, 20 May 2008 23:06:15 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 43E7E8FC0C for ; Tue, 20 May 2008 23:06:15 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [202.108.54.204]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 4ABCC2848B for ; Wed, 21 May 2008 07:06:14 +0800 (CST) Received: from localhost (tarsier.geekcn.org [202.108.54.204]) by tarsier.geekcn.org (Postfix) with ESMTP id 60AF1EB66F8; Wed, 21 May 2008 07:06:13 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([202.108.54.204]) by localhost (mail.geekcn.org [202.108.54.204]) (amavisd-new, port 10024) with ESMTP id SLS83CFlDBUg; Wed, 21 May 2008 07:06:08 +0800 (CST) Received: from charlie.delphij.net (71.5.7.139.ptr.us.xo.net [71.5.7.139]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 54239EB611A; Wed, 21 May 2008 07:06:07 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=G3JlBby67nh26ku0OlTZ6f+knYvnrpK0lUcq3SifN6lNN9vBvCaOlbrd4EI3OmbAS 5XK0ghFwG4p69CgJM1OBQ== Message-ID: <4833595D.1070409@delphij.net> Date: Tue, 20 May 2008 16:06:05 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.14 (X11/20080505) MIME-Version: 1.0 To: Andrew Snow References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> <4833483B.4030208@modulus.org> In-Reply-To: <4833483B.4030208@modulus.org> X-Enigmail-Version: 0.95.6 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 23:06:15 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andrew Snow wrote: | Warren Guy wrote: |> Thanks a lot for that. This seems to have alleviated the problem, I'm |> seeing |> decent performance now in my limited benchmark. It seems quite odd to |> me that |> the write cache is not enabled by default, but oh well. | | Technically with UFS, it can lead to filesystem or database corruption - | when the power goes off suddenly, the OS+controller thinks the data has | been written, but the drive has it in cache that is not battery-backed. I believe that data corruption can happen on every place if writing cache is lost, it's not an UFS-specific feature :) Cheers, - -- ** Help China's quake relief at http://www.redcross.org.cn/ |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkgzWV0ACgkQi+vbBBjt66C31ACfQHkDuW7ahYNIjL652ALOm/S4 pZYAoJLWOkyhs1PBbrKpg36MqGFN8iE9 =csG8 -----END PGP SIGNATURE----- From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 23:33:53 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 4AA4C1065678 for ; Tue, 20 May 2008 23:33:53 +0000 (UTC) (envelope-from andrew@modulus.org) Received: from email.octopus.com.au (host-122-100-2-232.octopus.com.au [122.100.2.232]) by mx1.freebsd.org (Postfix) with ESMTP id 124E48FC0C for ; Tue, 20 May 2008 23:33:52 +0000 (UTC) (envelope-from andrew@modulus.org) Received: by email.octopus.com.au (Postfix, from userid 1002) id 0C4B2173D3; Wed, 21 May 2008 09:33:51 +1000 (EST) X-Spam-Checker-Version: SpamAssassin 3.2.3 (2007-08-08) on email.octopus.com.au X-Spam-Level: X-Spam-Status: No, score=-1.4 required=10.0 tests=ALL_TRUSTED autolearn=failed version=3.2.3 Received: from [10.1.50.163] (142.19.96.58.exetel.com.au [58.96.19.142]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) (Authenticated sender: admin@email.octopus.com.au) by email.octopus.com.au (Postfix) with ESMTP id 749D31720E; Wed, 21 May 2008 09:33:46 +1000 (EST) Message-ID: <48335FD5.3000800@modulus.org> Date: Wed, 21 May 2008 09:33:41 +1000 From: Andrew Snow User-Agent: Thunderbird 1.5.0.9 (Windows/20061207) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> <4833483B.4030208@modulus.org> <4833595D.1070409@delphij.net> In-Reply-To: <4833595D.1070409@delphij.net> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: d@delphij.net Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 23:33:53 -0000 Xin LI wrote: > I believe that data corruption can happen on every place if writing > cache is lost, it's not an UFS-specific feature :) ZFS has the promise of not requirring safe hardware write-caches due to its parity data and intent log. From my understanding, it can promise that a data block has been written completely and safely, and that filesystem metadata is not corrupted, but out-of-order writes could still happen. - Andrew From owner-freebsd-scsi@FreeBSD.ORG Tue May 20 23:38:38 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id B33BD1065676 for ; Tue, 20 May 2008 23:38:38 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.delphij.net (delphij-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:2c9::2]) by mx1.freebsd.org (Postfix) with ESMTP id 5EDB98FC1B for ; Tue, 20 May 2008 23:38:38 +0000 (UTC) (envelope-from delphij@delphij.net) Received: from tarsier.geekcn.org (tarsier.geekcn.org [202.108.54.204]) (using TLSv1 with cipher ADH-CAMELLIA256-SHA (256/256 bits)) (No client certificate requested) by tarsier.delphij.net (Postfix) with ESMTPS id 7FB402846D for ; Wed, 21 May 2008 07:38:37 +0800 (CST) Received: from localhost (tarsier.geekcn.org [202.108.54.204]) by tarsier.geekcn.org (Postfix) with ESMTP id 50507EB6718; Wed, 21 May 2008 07:38:37 +0800 (CST) X-Virus-Scanned: amavisd-new at geekcn.org Received: from tarsier.geekcn.org ([202.108.54.204]) by localhost (mail.geekcn.org [202.108.54.204]) (amavisd-new, port 10024) with ESMTP id YsjODObiA9ej; Wed, 21 May 2008 07:38:32 +0800 (CST) Received: from charlie.delphij.net (71.5.7.139.ptr.us.xo.net [71.5.7.139]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by tarsier.geekcn.org (Postfix) with ESMTPSA id 80C1EEB1B22; Wed, 21 May 2008 07:38:31 +0800 (CST) DomainKey-Signature: a=rsa-sha1; s=default; d=delphij.net; c=nofws; q=dns; h=message-id:date:from:reply-to:organization:user-agent: mime-version:to:cc:subject:references:in-reply-to: x-enigmail-version:openpgp:content-type:content-transfer-encoding; b=lXO1AMf5iPPlOrrr9VrFycnMI0jjWngE/nk1gk0SZ2G6KeyOsbscrWYA7uKRyIqF4 GnjnvVYS3/HnHLbyni03Q== Message-ID: <483360F5.6090404@delphij.net> Date: Tue, 20 May 2008 16:38:29 -0700 From: Xin LI Organization: The FreeBSD Project User-Agent: Thunderbird 2.0.0.14 (X11/20080505) MIME-Version: 1.0 To: Andrew Snow References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> <4833483B.4030208@modulus.org> <4833595D.1070409@delphij.net> <48335FD5.3000800@modulus.org> In-Reply-To: <48335FD5.3000800@modulus.org> X-Enigmail-Version: 0.95.6 OpenPGP: id=18EDEBA0; url=http://www.delphij.net/delphij.asc Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org, d@delphij.net Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: d@delphij.net List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 20 May 2008 23:38:38 -0000 -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Andrew Snow wrote: | Xin LI wrote: |> I believe that data corruption can happen on every place if writing |> cache is lost, it's not an UFS-specific feature :) | | ZFS has the promise of not requirring safe hardware write-caches due to | its parity data and intent log. | | From my understanding, it can promise that a data block has been | written completely and safely, and that filesystem metadata is not | corrupted, but out-of-order writes could still happen. ZFS's promise is based on the fact that it does not overwrite data. Oops, I think I should not use the term 'corruption', I meant 'loss'. Cheers, - -- ** Help China's quake relief at http://www.redcross.org.cn/ |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> Xin LI http://www.delphij.net/ FreeBSD - The Power to Serve! -----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.9 (FreeBSD) iEYEARECAAYFAkgzYPUACgkQi+vbBBjt66CSsgCgsWfChktCt/XfocU/olHV1OSZ jq0An0NZaQOp8fJcxhS3YB2z3Wp7va3E =5a34 -----END PGP SIGNATURE----- From owner-freebsd-scsi@FreeBSD.ORG Wed May 21 00:36:57 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id A6419106564A for ; Wed, 21 May 2008 00:36:57 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 644B78FC1F for ; Wed, 21 May 2008 00:36:57 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local ([192.168.254.200]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id m4L0amHO036386; Tue, 20 May 2008 18:36:49 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <48336EA0.3050109@samsco.org> Date: Tue, 20 May 2008 18:36:48 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Warren Guy References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> In-Reply-To: <4832E6C2.7040205@calorieking.com> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.2 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 00:36:57 -0000 Warren Guy wrote: > Scott, > > Thanks a lot for that. This seems to have alleviated the problem, I'm seeing > decent performance now in my limited benchmark. It seems quite odd to me that > the write cache is not enabled by default, but oh well. > > Thanks again for your help! > > Warren For data reliability, you really don't want it enabled by default. The problem is that SATA/ATA performs so poorly without it that everyone turns it on and lives with the consequences. The tweak that I recommended puts it in line with what the FreeBSD ATA driver has been doing for years. According to your original benchmark, Linux performs better on the sequential tests, but those simply aren't representative of most people's workloads. Linux indeed has some tricks to make sequential benchmarks perform well, but they aren't tricks that I'm all that interested in implementing in FreeBSD (though increasing the maxio size for 64-bit platforms would help and has few detrimental effects). The same benchmark shows that FreeBSD performs just as well, if not better, than Linux in random tests, even without the write cache enabled. Those tests are more representative of typical workloads. So, it's up to you to analyze what kind of workload you expect, and make the appropriate tradeoffs. Scott From owner-freebsd-scsi@FreeBSD.ORG Wed May 21 17:29:17 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 6A1DA106568C for ; Wed, 21 May 2008 17:29:17 +0000 (UTC) (envelope-from sbruno@miralink.com) Received: from plato.miralink.com (mail.miralink.com [70.103.185.20]) by mx1.freebsd.org (Postfix) with ESMTP id 45C8E8FC1E for ; Wed, 21 May 2008 17:29:17 +0000 (UTC) (envelope-from sbruno@miralink.com) Received: from localhost (localhost.localdomain [127.0.0.1]) by plato.miralink.com (Postfix) with ESMTP id B7ACE1A9224 for ; Wed, 21 May 2008 10:29:15 -0700 (PDT) X-Virus-Scanned: amavisd-new at X-Spam-Flag: NO X-Spam-Score: -4.399 X-Spam-Level: X-Spam-Status: No, score=-4.399 tagged_above=-10 required=6.6 tests=[ALL_TRUSTED=-1.8, BAYES_00=-2.599] Received: from plato.miralink.com ([127.0.0.1]) by localhost (plato.miralink.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bcm90UcHhPBZ for ; Wed, 21 May 2008 10:29:15 -0700 (PDT) Received: from iago.office.miralink.com (iago.office.miralink.com [10.0.0.40]) by plato.miralink.com (Postfix) with ESMTP id 7E85C1A9217 for ; Wed, 21 May 2008 10:29:15 -0700 (PDT) Message-ID: <48345BEC.8020700@miralink.com> Date: Wed, 21 May 2008 10:29:16 -0700 From: Sean Bruno User-Agent: Thunderbird 2.0.0.14 (X11/20080501) MIME-Version: 1.0 To: freebsd-scsi@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: ISCSI Patch X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 17:29:17 -0000 This small change allows the iSCSI initiator to compile and load under RELENG_6. isc_cam.c 398a399 > #if __FreeBSD_version >= 700000 399a401,403 > #else > if(xpt_bus_register(sim, 0/*bus_number*/) != CAM_SUCCESS) > #endif -- Sean Bruno MiraLink Corporation 6015 NE 80th Ave, Ste 100 Portland, OR 97218 Phone 503-621-5143 Fax 503-621-5199 From owner-freebsd-scsi@FreeBSD.ORG Wed May 21 19:46:26 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 32EC2106566B for ; Wed, 21 May 2008 19:46:26 +0000 (UTC) (envelope-from cdillon@wolves.k12.mo.us) Received: from mail.wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by mx1.freebsd.org (Postfix) with ESMTP id 0530D8FC18 for ; Wed, 21 May 2008 19:46:25 +0000 (UTC) (envelope-from cdillon@wolves.k12.mo.us) Received: from localhost (localhost [127.0.0.1]) by mail.wolves.k12.mo.us (Postfix) with ESMTP id 8BF83B82F; Wed, 21 May 2008 14:30:52 -0500 (CDT) X-Virus-Scanned: amavisd-new at wolves.k12.mo.us Received: from mail.wolves.k12.mo.us ([127.0.0.1]) by localhost (mail.wolves.k12.mo.us [127.0.0.1]) (amavisd-new, port 10024) with LMTP id AT0Fs3zyfZcf; Wed, 21 May 2008 14:30:51 -0500 (CDT) Received: from wolves.k12.mo.us (mail.wolves.k12.mo.us [207.160.214.1]) by mail.wolves.k12.mo.us (Postfix) with ESMTP id 52FB4B820; Wed, 21 May 2008 14:30:51 -0500 (CDT) Received: from rstech21.int.wolves.k12.mo.us (rstech21.int.wolves.k12.mo.us [10.1.3.201]) by www.wolves.k12.mo.us (Horde Framework) with HTTP; Wed, 21 May 2008 14:30:51 -0500 Message-ID: <20080521143051.17771kseoxrlhy7f@www.wolves.k12.mo.us> Date: Wed, 21 May 2008 14:30:51 -0500 From: Chris Dillon To: Scott Long References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> <48336EA0.3050109@samsco.org> In-Reply-To: <48336EA0.3050109@samsco.org> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes"; format="flowed" Content-Disposition: inline Content-Transfer-Encoding: quoted-printable User-Agent: Internet Messaging Program (IMP) H3 (4.2-RC2) / FreeBSD-6.3 Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 19:46:26 -0000 Quoting Scott Long : > For data reliability, you really don't want it enabled by default. The > problem is that SATA/ATA performs so poorly without it that everyone > turns it on and lives with the consequences. The tweak that I > recommended puts it in line with what the FreeBSD ATA driver has been > doing for years. Doesn't SATA NCQ solve this particular performance vs. reliability =20 problem since it safely allows multiple outstanding write requests? =20 Of course that means the SATA RAID controller would have to use NCQ on =20 the drives and would probably also need its own non-volatile cache. =20 I've always assumed this is how SCSI/SAS drives (with TCQ) perform as =20 well as they do without sacrificing data integrity. We recently bought a new HP DL380G5 server with a P800 SAS RAID =20 controller, MSA60 external drive shelf with 12 750GB SATA drives, =20 11-drive RAID5 array w/ hot-spare (a few too many drives in a single =20 RAID5 array, I know, but I'm experimenting). The system is running =20 Windows Server 2K3 R2. Without telling the P800 to enable the SATA WC =20 (it has an option to do so, off by default), when doing a drag and =20 drop file copy of several very large files from the internal SAS array =20 to the external SATA array it writes 300MB/sec. I briefly enabled the =20 "Physical Drive Write Cache" on the controller just a few minutes ago =20 and ran another test and didn't notice any difference in write speed. =20 I can only assume from this that the P800 is using NCQ on the SATA =20 drives. --=20 Chris Dillon - NetEng/SysAdm Reeds Spring R-IV School District Technology Department 175 Elementary Rd. Reeds Spring, MO 65737 Voice: 417-272-8266 Fax: 417-272-0015 From owner-freebsd-scsi@FreeBSD.ORG Wed May 21 20:08:27 2008 Return-Path: Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 14798106567C for ; Wed, 21 May 2008 20:08:25 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.freebsd.org (Postfix) with ESMTP id 9F0FD8FC19 for ; Wed, 21 May 2008 20:08:24 +0000 (UTC) (envelope-from scottl@samsco.org) Received: from phobos.local ([192.168.254.200]) (authenticated bits=0) by pooker.samsco.org (8.13.8/8.13.8) with ESMTP id m4LK8HtU045239; Wed, 21 May 2008 14:08:17 -0600 (MDT) (envelope-from scottl@samsco.org) Message-ID: <48348131.3040602@samsco.org> Date: Wed, 21 May 2008 14:08:17 -0600 From: Scott Long User-Agent: Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US; rv:1.8.1.13) Gecko/20080313 SeaMonkey/1.1.9 MIME-Version: 1.0 To: Chris Dillon References: <4832C397.3090004@calorieking.com> <4832E0EE.3030402@samsco.org> <4832E6C2.7040205@calorieking.com> <48336EA0.3050109@samsco.org> <20080521143051.17771kseoxrlhy7f@www.wolves.k12.mo.us> In-Reply-To: <20080521143051.17771kseoxrlhy7f@www.wolves.k12.mo.us> Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-4.3 required=3.8 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.8 X-Spam-Checker-Version: SpamAssassin 3.1.8 (2007-02-13) on pooker.samsco.org Cc: freebsd-scsi@freebsd.org Subject: Re: Very poor performance from Dell/LSI Logic SAS 3000 series SATA/SAS RAID controller FreeBSD 6.3 X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 21 May 2008 20:08:27 -0000 Chris Dillon wrote: > Quoting Scott Long : > >> For data reliability, you really don't want it enabled by default. The >> problem is that SATA/ATA performs so poorly without it that everyone >> turns it on and lives with the consequences. The tweak that I >> recommended puts it in line with what the FreeBSD ATA driver has been >> doing for years. > > Doesn't SATA NCQ solve this particular performance vs. reliability > problem since it safely allows multiple outstanding write requests? Of > course that means the SATA RAID controller would have to use NCQ on the > drives and would probably also need its own non-volatile cache. I've > always assumed this is how SCSI/SAS drives (with TCQ) perform as well as > they do without sacrificing data integrity. Yes and no. NCQ gets you 90% the way there, but the lack of an ordered tag operation in the NCQ protocol means that i/o streams can be starved, forcing you to do unpleasant i/o scheduling hacks. But yes, it helps quite a bit, and I have a prototype driver already working that supports NCQ and performs very well with write cache turned off. > > We recently bought a new HP DL380G5 server with a P800 SAS RAID > controller, MSA60 external drive shelf with 12 750GB SATA drives, > 11-drive RAID5 array w/ hot-spare (a few too many drives in a single > RAID5 array, I know, but I'm experimenting). The system is running > Windows Server 2K3 R2. Without telling the P800 to enable the SATA WC > (it has an option to do so, off by default), when doing a drag and drop > file copy of several very large files from the internal SAS array to the > external SATA array it writes 300MB/sec. I briefly enabled the > "Physical Drive Write Cache" on the controller just a few minutes ago > and ran another test and didn't notice any difference in write speed. I > can only assume from this that the P800 is using NCQ on the SATA drives. > The cache and queueing mechanism on most IOP raid cards will smooth over the performance problems with ATA/SATA, so your results aren't too surprising. Scott