From owner-freebsd-scsi@FreeBSD.ORG Mon Feb 13 10:20:37 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C4E2616A420 for ; Mon, 13 Feb 2006 10:20:37 +0000 (GMT) (envelope-from peceka@gmail.com) Received: from zproxy.gmail.com (zproxy.gmail.com [64.233.162.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5CF9043D45 for ; Mon, 13 Feb 2006 10:20:37 +0000 (GMT) (envelope-from peceka@gmail.com) Received: by zproxy.gmail.com with SMTP id o1so159880nzf for ; Mon, 13 Feb 2006 02:20:36 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=pUI2ekt6BDhMNtebRdRzaY1n4WIuLZq+/NhaduyPyekpcDe47+e2xSs5xkdAhe3ITgOwdzPPRjeG9Gbq2hI3p8yXM6OPs3R7eXw0BIeWfUnnwh3AjTCviU8lo4zxld25iYvj0nL37U+7wika8AosVo6ghGnFXRpecVvxWODeenE= Received: by 10.65.230.17 with SMTP id h17mr905787qbr; Mon, 13 Feb 2006 02:20:31 -0800 (PST) Received: by 10.65.253.15 with HTTP; Mon, 13 Feb 2006 02:20:31 -0800 (PST) Message-ID: Date: Mon, 13 Feb 2006 11:20:31 +0100 From: peceka To: freebsd-scsi@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: Re: problem with low efficiency of HP Smart Array 6i under FBSD X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Feb 2006 10:20:37 -0000 Hi, > > Time of executing this scripts: > > FreeBSD: 149m 28s > > Linux: 97m 13s > >Well, so Linux is about 33% faster. That's not an order >of magnitude. It could be explained by differences in >the file system parameters. By the way, what kind of >file system did you use on FreeBSD and Linux? What >parameters did you use with newfs? On Linux there is ext3 On FreeBSD is UFS2, made by installer program, so there are standard parameters in newfs. >In fact, the difference could also be caused by the >harddisks not being the same. And even if they are >the same models, the location of your test files on the >harddisk can be different. Most harddisks are much >faster when files are stored on the lower cylinders. Disks are this same, with this same firmware. Except that on Linux disks are 10k rpm and of FBSD are 15k rpm. >I don't think the difference is caused by the SCSI code. We've made exactly the same tests on other machine (devel1) with FBSD: RAM: 191 MB CPU: AMD Sempron(tm) Processor 2600+ (1599.83-MHz 686-class CPU) HDD: ad0: 38166MB at ata0-master UDMA100 Filesystem: UFS2 (with standard newfs parameters) > freebsd: 149m 28s > Linux: 97m 13s devel1: 182s > freebsd: 2m 41s > linux: 1m 27s devel1: 2m 06s > freebsd: 299,6s > linux: 214,1s devel1: 268.3s > freebsd: 277,2s > linux: 199s devel1: 246.2s In every test it's faster than HP DL380 with SCSI with FBSD disks and slower that machine with Linux. So, a think it's a problem with HP SmartArray 6i drivers under FBSD. Best regards, p. From owner-freebsd-scsi@FreeBSD.ORG Mon Feb 13 10:24:01 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B4FD416A420 for ; Mon, 13 Feb 2006 10:24:01 +0000 (GMT) (envelope-from ps@freebsd.org) Received: from elvis.mu.org (elvis.mu.org [192.203.228.196]) by mx1.FreeBSD.org (Postfix) with ESMTP id 87AFF43D46 for ; Mon, 13 Feb 2006 10:24:01 +0000 (GMT) (envelope-from ps@freebsd.org) Received: from [192.168.1.88] (64-142-76-135.dsl.static.sonic.net [64.142.76.135]) by elvis.mu.org (Postfix) with ESMTP id 529791A3C1D; Mon, 13 Feb 2006 02:24:01 -0800 (PST) Message-ID: <43F05E43.2000806@freebsd.org> Date: Mon, 13 Feb 2006 02:24:03 -0800 From: Paul Saab User-Agent: Thunderbird 1.5 (Macintosh/20051201) MIME-Version: 1.0 To: peceka References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Cc: freebsd-scsi@freebsd.org Subject: Re: problem with low efficiency of HP Smart Array 6i under FBSD X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Feb 2006 10:24:01 -0000 peceka wrote: > In every test it's faster than HP DL380 with SCSI with FBSD disks and > slower that machine with Linux. > > So, a think it's a problem with HP SmartArray 6i drivers under FBSD. > > > CISS is still under Giant. Are you sure you have a battery cache in each machine? From owner-freebsd-scsi@FreeBSD.ORG Mon Feb 13 11:02:45 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ABABC16A420 for ; Mon, 13 Feb 2006 11:02:45 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D2CB43D4C for ; Mon, 13 Feb 2006 11:02:45 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k1DB2jKT067427 for ; Mon, 13 Feb 2006 11:02:45 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k1DB2hAo067421 for freebsd-scsi@freebsd.org; Mon, 13 Feb 2006 11:02:43 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 13 Feb 2006 11:02:43 GMT Message-Id: <200602131102.k1DB2hAo067421@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-scsi@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 13 Feb 2006 11:02:45 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/05/03] kern/27059 scsi [sym] SCSI subsystem hangs under heavy lo o [2001/06/29] kern/28508 scsi problems with backup to Tandberg SLR40 st o [2002/06/17] kern/39388 scsi ncr/sym drivers fail with 53c810 and more o [2002/07/22] kern/40895 scsi wierd kernel / device driver bug o [2003/05/24] kern/52638 scsi [panic] SCSI U320 on SMP server won't run s [2003/09/30] kern/57398 scsi [mly] Current fails to install on mly(4) o [2003/12/26] kern/60598 scsi wire down of scsi devices conflicts with o [2003/12/27] kern/60641 scsi [sym] Sporadic SCSI bus resets with 53C81 s [2004/01/10] kern/61165 scsi [panic] kernel page fault after calling c o [2004/12/02] kern/74627 scsi [ahc] [hang] Adaptec 2940U2W Can't boot 5 o [2005/06/04] kern/81887 scsi [aac] Adaptec SCSI 2130S aac0: GetDeviceP o [2005/12/12] kern/90282 scsi [sym] SCSI bus resets cause loss of ch de o [2006/02/04] kern/92798 scsi [ahc] SCSI problem with timeouts o [2006/02/10] kern/93128 scsi [sym] FreeBSD 6.1 BETA 1 has problems wit 14 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/12/06] kern/23314 scsi aic driver fails to detect Adaptec 1520B o [2002/02/23] kern/35234 scsi World access to /dev/pass? (for scanner) o [2002/06/02] kern/38828 scsi [feature request] DPT PM2012B/90 doesn't o [2002/10/29] kern/44587 scsi dev/dpt/dpt.h is missing defines required o [2005/01/12] kern/76178 scsi [ahd] Problem with ahd and large SCSI Rai 5 problems total. From owner-freebsd-scsi@FreeBSD.ORG Tue Feb 14 15:08:46 2006 Return-Path: X-Original-To: scsi@FreeBSD.org Delivered-To: freebsd-scsi@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AA24F16A423; Tue, 14 Feb 2006 15:08:46 +0000 (GMT) (envelope-from gpalmer@freebsd.org) Received: from noop.colo.erols.net (noop.colo.erols.net [207.96.1.150]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1ACC743D5D; Tue, 14 Feb 2006 15:08:45 +0000 (GMT) (envelope-from gpalmer@freebsd.org) Received: from gjp by noop.colo.erols.net with local (Exim 4.52 (FreeBSD)) id 1F91nJ-0003Fm-4U; Tue, 14 Feb 2006 10:08:45 -0500 Date: Tue, 14 Feb 2006 10:08:45 -0500 From: Gary Palmer To: Tom Samplonius Message-ID: <20060214150845.GB29569@in-addr.com> References: <2CEE6163475607F32A420FA1@jordgubbe.pingpong.net> <20060207121257.D53605@mgmt.uniserve.ca> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20060207121257.D53605@mgmt.uniserve.ca> Cc: Palle Girgensohn , scsi@FreeBSD.org Subject: Re: NAS w/ multipath X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2006 15:08:47 -0000 On Tue, Feb 07, 2006 at 12:22:33PM -0800, Tom Samplonius wrote: > Now, in FreeBSD you could also multipath in the GEOM layer. GEOM knows > about devices going away, and knows how to handle that (ex. gmirror). > There is some support in GEOM for round-robin IO to two devices. However, > phk has reported that the isp driver can hang forever on some timeouts, so > it might not be useful. And I don't even know if GEOM round-robin is even > finished. Be very careful with that kind of work. There are SAN units out there that do not share caches across multiple controllers. To work around that limitation, they have one controller than can "own" the LUN at any one time, to ensure data consistency. So if you write to a LUN through the controller which does not "own" the LUN, it actually transfers "ownership" of the LUN to the controller you sent the request through. In at least one implimentation, the LUN vanishes from both controllers for a period measured in seconds while management of the LUN is handed off from one controller to the other. The vendor worked around this with special drivers that lived on the OS, which wasn't an option for us as they didn't have FreeBSD support. I suspect higher end devices (e.g. HDS and EMC Symmetrix units) this isn't a problem, but in mid range and lower end stuff I'd expect problems if the paths landed on separate controllers on the array. Gary From owner-freebsd-scsi@FreeBSD.ORG Tue Feb 14 18:40:20 2006 Return-Path: X-Original-To: scsi@FreeBSD.org Delivered-To: freebsd-scsi@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 89DDB16A420; Tue, 14 Feb 2006 18:40:20 +0000 (GMT) (envelope-from tom@uniserve.com) Received: from mx5.uniserve.ca (mx5.uniserve.ca [216.113.192.46]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7E07143D53; Tue, 14 Feb 2006 18:40:18 +0000 (GMT) (envelope-from tom@uniserve.com) Received: from mgmt.uniserve.ca ([216.113.192.30]) by mx5.uniserve.ca with esmtp (Exim 4.50) id 1F9562-000Agl-6W; Tue, 14 Feb 2006 10:40:18 -0800 Date: Tue, 14 Feb 2006 10:40:18 -0800 (PST) From: Tom Samplonius X-X-Sender: tom@mgmt.uniserve.ca To: Gary Palmer In-Reply-To: <20060214150845.GB29569@in-addr.com> Message-ID: <20060214103146.K99735@mgmt.uniserve.ca> References: <2CEE6163475607F32A420FA1@jordgubbe.pingpong.net> <20060207121257.D53605@mgmt.uniserve.ca> <20060214150845.GB29569@in-addr.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Scanner: OK. Scanned. Cc: Palle Girgensohn , scsi@FreeBSD.org Subject: Re: NAS w/ multipath X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 14 Feb 2006 18:40:20 -0000 On Tue, 14 Feb 2006, Gary Palmer wrote: > On Tue, Feb 07, 2006 at 12:22:33PM -0800, Tom Samplonius wrote: >> Now, in FreeBSD you could also multipath in the GEOM layer. GEOM knows >> about devices going away, and knows how to handle that (ex. gmirror). >> There is some support in GEOM for round-robin IO to two devices. However, >> phk has reported that the isp driver can hang forever on some timeouts, so >> it might not be useful. And I don't even know if GEOM round-robin is even >> finished. > > Be very careful with that kind of work. There are SAN units out there > that do not share caches across multiple controllers. To work around that > limitation, they have one controller than can "own" the LUN at any one > time, to ensure data consistency. So if you write to a LUN through the > controller which does not "own" the LUN, it actually transfers "ownership" > of the LUN to the controller you sent the request through. In at least Yes, trespass support. Some controllers support auto-trespassing, but if the controllers do not have cache consistancy, you should probably make sure auto-trespass is disabled. In manual trespassing, a specific SCSI command needs to be sent to activate the dormant LUN, which is generally vendor specific. > one implimentation, the LUN vanishes from both controllers for a period > measured in seconds while management of the LUN is handed off from one > controller to the other. The vendor worked around this with special > drivers that lived on the OS, which wasn't an option for us as they > didn't have FreeBSD support. Yes, there are a lot of patches on the net to hack in various kinds of trespass support into Linux for various types of boxes. > I suspect higher end devices (e.g. HDS and EMC Symmetrix units) this > isn't a problem, but in mid range and lower end stuff I'd expect problems > if the paths landed on separate controllers on the array. I don't think this is a problem with current mid-range stuff. A mirrored write cache is considered a basic feature. Not only does a mirrored write cache protect against controller cache consistancy, it also protects losing the contents of the write cache if a controller fails, which is generally a much bigger problem. > Gary > Tom From owner-freebsd-scsi@FreeBSD.ORG Wed Feb 15 07:26:04 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 84F7716A420 for ; Wed, 15 Feb 2006 07:26:04 +0000 (GMT) (envelope-from peceka@gmail.com) Received: from wproxy.gmail.com (wproxy.gmail.com [64.233.184.206]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2231843D45 for ; Wed, 15 Feb 2006 07:26:04 +0000 (GMT) (envelope-from peceka@gmail.com) Received: by wproxy.gmail.com with SMTP id i12so55068wra for ; Tue, 14 Feb 2006 23:26:03 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition; b=TGj2MpQwxdlrleeUGcW1LMwBni7Ag/ts7mDQStDseSo8Pz/0jYOD7t/TtpPdssX9pQYleyBx7Q63qo8ZDTDd0gTMU1L7LZdwHbpsTNy1GZD6aZlv2VRlv0K0yBjhzry8z39WYXVDzczP43sTvMGl99mGbFF8617vbmDXRrGRuLs= Received: by 10.64.195.7 with SMTP id s7mr2327441qbf; Tue, 14 Feb 2006 23:26:03 -0800 (PST) Received: by 10.65.253.15 with HTTP; Tue, 14 Feb 2006 23:26:03 -0800 (PST) Message-ID: Date: Wed, 15 Feb 2006 08:26:03 +0100 From: peceka To: freebsd-scsi@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Subject: Re: problem with low efficiency of HP Smart Array 6i under FBSD X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Feb 2006 07:26:04 -0000 > > In every test it's faster than HP DL380 with SCSI with FBSD disks and > > slower that machine with Linux. > > > > So, a think it's a problem with HP SmartArray 6i drivers under FBSD. > > > > > > > CISS is still under Giant. Are you sure you have a battery cache in > each machine? Yes, i'm sure - when machine starts it shows: Slot 0 HP Smart Array 6i Controller (192 MB, v2.58) 1 Logical Drive and when we put it out it shows: Slot 0 HP Smart Array 6i Controller (64 MB, v2.58) 1 Logical Drive We did tests with and without battery: dl380 (CPU: 3.6GHz) without cache: 2h 34m =3D 154m with cache: 2h 25m without cache: 1m 24s with cache: 2m 49s wo. cache: 4m 58s w. cache: 4m 57s wo. cache: 4m 40s w. cache: 4m 38s Best Regards, p. From owner-freebsd-scsi@FreeBSD.ORG Wed Feb 15 16:01:21 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 671AC16A420 for ; Wed, 15 Feb 2006 16:01:21 +0000 (GMT) (envelope-from os@rsu.ru) Received: from mail.r61.net (mail.r61.net [195.208.245.235]) by mx1.FreeBSD.org (Postfix) with ESMTP id 98B4E43D49 for ; Wed, 15 Feb 2006 16:01:20 +0000 (GMT) (envelope-from os@rsu.ru) Received: from brain.cc.rsu.ru (brain.cc.rsu.ru [195.208.252.154]) (authenticated bits=0) by mail.r61.net (8.13.4/8.13.4) with ESMTP id k1FG1Hug082401 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Feb 2006 19:01:17 +0300 (MSK) (envelope-from os@rsu.ru) Date: Wed, 15 Feb 2006 19:01:17 +0300 (MSK) From: Oleg Sharoiko To: freebsd-scsi@freebsd.org Message-ID: <20060215102749.D58480@brain.cc.rsu.ru> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: ClamAV version 0.86.2, clamav-milter version 0.86 on asterix.r61.net X-Virus-Status: Clean Cc: Andrey Beresovsky Subject: Boot hangs on ips0: resetting adapter, this may take up to 5 minutes X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Feb 2006 16:01:21 -0000 Hello! I'm trying to install 6.0 on IBM eServer xSeries 226 (2 CPUs, IBM ServeRAID 6i). During normal boot GENERIC kernel hangs with the last message being ips0: resetting adapter, this may take up to 5 minutes Ctrl-Alt-Del doesn't work. Pressing power button doesn't work either, have to push it and wait for several seconds to switch the system off. With hint.apic.0.disabled=1 GENERIC seems to boot fine. It's also possible to boot FreeBSD/i386 when hyper-threading is enabled (4 logical CPUs) and kernel has SMP option (just tried default SMP kernel). With only 2 CPUs (HTT disabled) SMP kernel also hangs at the same point. FreeBSD/amd64 only boots with apic disabled. This is not specific for 6.0 as RELENG_6 and CURRENT also have this problem. Could somebody please help me debugging this problem? Logs of verbose boots are available at http://rsu.ru/~os/ips/ I'm ready to provide any possible help needed to resolve this issue. I'd greatly appreciate any help. -- Oleg Sharoiko. Software and Network Engineer Computer Center of Rostov State University. From owner-freebsd-scsi@FreeBSD.ORG Wed Feb 15 20:26:43 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F025A16A420; Wed, 15 Feb 2006 20:26:43 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from pooker.samsco.org (pooker.samsco.org [168.103.85.57]) by mx1.FreeBSD.org (Postfix) with ESMTP id 24A3743D6A; Wed, 15 Feb 2006 20:26:37 +0000 (GMT) (envelope-from scottl@samsco.org) Received: from [10.10.3.185] ([69.15.205.254]) (authenticated bits=0) by pooker.samsco.org (8.13.4/8.13.4) with ESMTP id k1FKQYE8046342; Wed, 15 Feb 2006 13:26:35 -0700 (MST) (envelope-from scottl@samsco.org) Message-ID: <43F38E74.6020705@samsco.org> Date: Wed, 15 Feb 2006 13:26:28 -0700 From: Scott Long User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.7.12) Gecko/20060206 X-Accept-Language: en-us, en MIME-Version: 1.0 To: Oleg Sharoiko References: <20060215102749.D58480@brain.cc.rsu.ru> In-Reply-To: <20060215102749.D58480@brain.cc.rsu.ru> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=0.0 required=3.8 tests=none autolearn=failed version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on pooker.samsco.org Cc: freebsd-scsi@freebsd.org, Andrey Beresovsky , John Baldwin Subject: Re: Boot hangs on ips0: resetting adapter, this may take up to 5 minutes X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Feb 2006 20:26:44 -0000 Oleg Sharoiko wrote: > Hello! > > I'm trying to install 6.0 on IBM eServer xSeries 226 (2 CPUs, IBM > ServeRAID 6i). During normal boot GENERIC kernel hangs with the last > message being > > ips0: resetting adapter, this may take up to 5 minutes > > Ctrl-Alt-Del doesn't work. Pressing power button doesn't work either, > have to push it and wait for several seconds to switch the system off. > > With hint.apic.0.disabled=1 GENERIC seems to boot fine. It's also possible > to boot FreeBSD/i386 when hyper-threading is enabled (4 logical CPUs) and > kernel has SMP option (just tried default SMP kernel). With only 2 CPUs > (HTT disabled) SMP kernel also hangs at the same point. FreeBSD/amd64 only > boots with apic disabled. This is not specific for 6.0 as RELENG_6 and > CURRENT also have this problem. > > Could somebody please help me debugging this problem? Logs of verbose > boots are available at http://rsu.ru/~os/ips/ I'm ready to provide any > possible help needed to resolve this issue. I'd greatly appreciate any > help. > This sounds like an interrupt routing problem. The symptoms are similar to others where APIC-routed interrupts don't seem to make it to the active CPUs, depending on whether SMP or HTT is enabled or disabled. Maybe John has some insight here. Scott From owner-freebsd-scsi@FreeBSD.ORG Thu Feb 16 22:37:45 2006 Return-Path: X-Original-To: scsi@freebsd.org Delivered-To: freebsd-scsi@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6E1BF16A420 for ; Thu, 16 Feb 2006 22:37:45 +0000 (GMT) (envelope-from joao.barros@gmail.com) Received: from xproxy.gmail.com (xproxy.gmail.com [66.249.82.201]) by mx1.FreeBSD.org (Postfix) with ESMTP id EACB743D53 for ; Thu, 16 Feb 2006 22:37:43 +0000 (GMT) (envelope-from joao.barros@gmail.com) Received: by xproxy.gmail.com with SMTP id i26so211559wxd for ; Thu, 16 Feb 2006 14:37:43 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references; b=KcQh+L9agFOF45GKQJZyk3LumCQqE5cMCe47/HA9OASy7fhIr/ybryDpvSW1TH92z7YZtAVyGKs4M77LsKzFoUTEqSvfW0yI/cjALm7kOvGOd976LTp07ynFLdi+NDnn+azFQwYbfVLQ4UxXozwl+y8tSH/yqgwwU3pkN9oVAmc= Received: by 10.70.48.16 with SMTP id v16mr240119wxv; Thu, 16 Feb 2006 14:37:43 -0800 (PST) Received: by 10.70.9.9 with HTTP; Thu, 16 Feb 2006 14:37:43 -0800 (PST) Message-ID: <70e8236f0602161437o1593c147na97239cf9054610e@mail.gmail.com> Date: Thu, 16 Feb 2006 22:37:43 +0000 From: Joao Barros To: Tom Samplonius In-Reply-To: <20060214103146.K99735@mgmt.uniserve.ca> MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline References: <2CEE6163475607F32A420FA1@jordgubbe.pingpong.net> <20060207121257.D53605@mgmt.uniserve.ca> <20060214150845.GB29569@in-addr.com> <20060214103146.K99735@mgmt.uniserve.ca> Cc: Palle Girgensohn , scsi@freebsd.org Subject: Re: NAS w/ multipath X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 16 Feb 2006 22:37:45 -0000 On 2/14/06, Tom Samplonius wrote: > > On Tue, 14 Feb 2006, Gary Palmer wrote: > > > I suspect higher end devices (e.g. HDS and EMC Symmetrix units) this > > isn't a problem, but in mid range and lower end stuff I'd expect proble= ms > > if the paths landed on separate controllers on the array. > > I don't think this is a problem with current mid-range stuff. A mirro= red > write cache is considered a basic feature. Not only does a mirrored writ= e cache > protect against controller cache consistancy, it also protects losing the > contents of the write cache if a controller fails, which is generally a m= uch > bigger problem. The EMC Clarion Series, at least the CX600 model I work with has mirrored write cache. In the event of controller failure it is disabled until redundancy is resto= red. I had training on an entry level model, a CX300 and the funcionality was the same. The Symetrix I bet it has but EMC doesn't let us touch those ;-) -- Joao Barros From owner-freebsd-scsi@FreeBSD.ORG Sat Feb 18 01:29:02 2006 Return-Path: X-Original-To: freebsd-scsi@freebsd.org Delivered-To: freebsd-scsi@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C93716A420 for ; Sat, 18 Feb 2006 01:29:02 +0000 (GMT) (envelope-from bogo.readlist@gmail.com) Received: from nproxy.gmail.com (nproxy.gmail.com [64.233.182.202]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4E4A643D45 for ; Sat, 18 Feb 2006 01:29:01 +0000 (GMT) (envelope-from bogo.readlist@gmail.com) Received: by nproxy.gmail.com with SMTP id y38so353791nfb for ; Fri, 17 Feb 2006 17:29:00 -0800 (PST) DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws; s=beta; d=gmail.com; h=received:message-id:date:from:to:subject:mime-version:content-type; b=RNOADgqcEQ/WLD1X+Zc4M9jtLSG7YD+Z8/M9XktjAXwJcxT+b4rqpcSpVj7WAPGuBw3nrFzQjE9AiIKGTyE8nvXpA90QYTzPS9aoRd4B0GcxxjKrZIqFgiB8J7AakCM2drHJw1hV8dH774WMDjVtrLsyqt9+nw/qKbbGSxCnOsA= Received: by 10.48.144.20 with SMTP id r20mr602961nfd; Fri, 17 Feb 2006 17:28:59 -0800 (PST) Received: by 10.48.224.8 with HTTP; Fri, 17 Feb 2006 17:28:59 -0800 (PST) Message-ID: Date: Fri, 17 Feb 2006 17:28:59 -0800 From: "bogo logo" To: freebsd-scsi@freebsd.org MIME-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: quoted-printable Content-Disposition: inline X-Content-Filtered-By: Mailman/MimeDel 2.1.5 Subject: freebsd5.4 stable amd64 with 3ware 9500 crashing problem X-BeenThere: freebsd-scsi@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: SCSI subsystem List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 18 Feb 2006 01:29:02 -0000 hi list, i posted this on -QUESTION, but did not see a response so i am posting here .. hoping that someone might have ran into it earlier (i read Alfred's post= s regarding the 3ware 9500 problems). --- we have an amd64 machine on freebsd 5.4-stable (sync'ed + compiled today) with a 3ware 9500-8S card. The 3ware card is handling a 1.1TB RAID5 volume (4x 400gb SATA). Recently, we rebooted the box to do some upgrades; now there are several problems: 1) everytime we try to mount the 1.1TB partition, the machine reboots without any messages. 2) when we run fsck on it, this is what we get: box# fsck /dev/da0s1a ** /dev/da0s1a CANNOT READ BLK: 2343322528 CONTINUE? [yn] y THE FOLLOWING DISK SECTORS COULD NOT BE READ: 2343322528, 2343322529, 2343322530, 2343322531, LOOK FOR ALTERNATE SUPERBLOCKS? [yn] y 32 is not a file system superblock CANNOT READ BLK: 458302416 CONTINUE? [yn] y THE FOLLOWING DISK SECTORS COULD NOT BE READ: 458302416, 458302417, 458302418, 458302419, 458302420, 458302421, 458302422, 458302423, 458302424= =3D , 458302425, 458302426, 458302427, 458302428, 458302429, 458302430, 458302431= =3D , CANNOT READ BLK: 916604800 CONTINUE? [yn] y THE FOLLOWING DISK SECTORS COULD NOT BE READ: 916604800, 916604801, 916604802, 916604803, 916604804, 916604805, 916604806, 916604807, 916604808= =3D , 916604809, 916604810, 916604811, 916604812, 916604813, 916604814, 916604815= =3D , CANNOT READ BLK: 1374907184 CONTINUE? [yn] y THE FOLLOWING DISK SECTORS COULD NOT BE READ: 1374907184, 1374907185, 1374907186, 1374907187, 1374907188, 1374907189, 1374907190, 1374907191, 1374907192, 1374907193, 1374907194, 1374907195, 1374907196, 1374907197, 1374907198, 1374907199, CANNOT READ BLK: 1833209568 CONTINUE? [yn] y THE FOLLOWING DISK SECTORS COULD NOT BE READ: 1833209568, 1833209569, 1833209570, 1833209571, 1833209572, 1833209573, 1833209574, 1833209575, 1833209576, 1833209577, 1833209578, 1833209579, 1833209580, 1833209581, 1833209582, 1833209583, CANNOT SEEK BLK: -2003455344 CONTINUE? [yn] y CANNOT READ BLK: -2003455344 CONTINUE? [yn] y CANNOT SEEK BLK: -2003455344 CONTINUE? [yn] y THE FOLLOWING DISK SECTORS COULD NOT BE READ: -2003455344, -2003455343, -2003455342, -2003455341, -2003455340, -2003455339, -2003455338, -2003455337, -2003455336, -2003455335, -2003455334, -2003455333, -2003455332, -2003455331, -2003455330, -2003455329, SEARCH FOR ALTERNATE SUPER-BLOCK FAILED. YOU MUST USE THE -b OPTION TO FSCK TO SPECIFY THE LOCATION OF AN ALTERNATE SUPER-BLOCK TO SUPPLY NEEDED INFORMATION; SEE fsck(8). ------------- How do we fix this? Why are there NEGATIVE disk sectors? The box freezes every time when we mount with with `mount -f /dev/da0s1a /mnt` thank you. -- this is the dmesg output.. -- Copyright (c) 1992-2005 The FreeBSD Project. Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994 The Regents of the University of California. All rights reserved. FreeBSD 5.5-PRERELEASE #0: Sun Feb 12 21:50:34 UTC 2006 root@box:/usr/obj/usr/src/sys/CRAP ACPI APIC Table: Timecounter "i8254" frequency 1193182 Hz quality 0 CPU: AMD Athlon(tm) 64 Processor 3200+ ( 2210.77-MHz K8-class CPU) Origin =3D3D "AuthenticAMD" Id =3D3D 0xfc0 Stepping =3D3D 0 Features=3D3D0x78bfbff AMD Features=3D3D0xe0500800 real memory =3D3D 268369920 (255 MB) avail memory =3D3D 247742464 (236 MB) ioapic0 irqs 0-23 on motherboard acpi0: on motherboard acpi0: Power Button (fixed) Timecounter "ACPI-fast" frequency 3579545 Hz quality 1000 acpi_timer0: <24-bit timer at 3.579545MHz> port 0x1008-0x100b on acpi0 cpu0: on acpi0 acpi_button0: on acpi0 pcib0: port 0xcf0-0xcf3,0xcf8-0xcff on acpi0 pci0: on pcib0 isab0: at device 1.0 on pci0 isa0: on isab0 pci0: at device 1.1 (no driver attached) ohci0: mem 0xfc003000-0xfc003fff irq 22 at device 2.0 on pci0 usb0: OHCI version 1.0, legacy support usb0: on ohci0 usb0: USB revision 1.0 uhub0: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub0: 4 ports with 4 removable, self powered ohci1: mem 0xfc004000-0xfc004fff irq 21 at device 2.1 on pci0 usb1: OHCI version 1.0, legacy support usb1: on ohci1 usb1: USB revision 1.0 uhub1: nVidia OHCI root hub, class 9/0, rev 1.00/1.00, addr 1 uhub1: 4 ports with 4 removable, self powered pci0: at device 2.2 (no driver attached) pci0: at device 5.0 (no driver attached) pci0: at device 6.0 (no driver attached) atapci0: port 0xf000-0xf00f,0x376,0x170-0x177,0x3f6,0x1f0-0x1f7 at device 8.0 on pci0 ata0: channel #0 on atapci0 ata1: channel #1 on atapci0 atapci1: port 0xdc00-0xdc0f,0xb70-0xb73,0x970-0x977,0xbf0-0xbf3,0x9f0-0x9f7 irq 20 at device 10.0 on pci0 ata2: channel #0 on atapci1 ata3: channel #1 on atapci1 pcib1: at device 11.0 on pci0 pci1: on pcib1 pci1: at device 0.0 (no driver attached) pcib2: at device 14.0 on pci0 pci2: on pcib2 3ware device driver for 9000 series storage controllers, version: 3.50.02.012 twa0: <3ware 9000 series Storage Controller> port 0x7000-0x70ff mem 0xfb000000-0xfb7fffff,0xfb80e000-0xfb80e0ff irq 17 at device 9.0 on pci2 twa0: INFO: (0x15: 0x1300): Controller details:: Model 9500S-8, 8 ports, Firmware FE9X 2.06.00.009, BIOS BE9X 2.03.01.051 skc0: port 0x7400-0x74ff mem 0xfb800000-0xfb803fff irq 18 at device 10.0 on pci2 skc0: (null) rev. (0x1) sk0: on skc0 sk0: Ethernet address: 00:0f:3d:f2:45:0c miibus0: on sk0 e1000phy0: on miibus0 e1000phy0: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto skc1: port 0x7800-0x78ff mem 0xfb804000-0xfb807fff irq 19 at device 11.0 on pci2 skc1: Marvell Yukon Lite Gigabit Ethernet rev. A3(0x7) sk1: on skc1 sk1: Ethernet address: 00:0d:61:7e:b2:5e miibus1: on sk1 e1000phy1: on miibus1 e1000phy1: 10baseT, 10baseT-FDX, 100baseTX, 100baseTX-FDX, 1000baseTX-FDX, auto atapci2: port 0x8c00-0x8c0f,0x8800-0x8803,0x8410-0x8417,0x8000-0x8003,0x7c10-0x7c17 irq 1= =3D 6 at device 12.0 on pci2 ata4: channel #0 on atapci2 ata5: channel #1 on atapci2 atapci3: port 0xa000-0xa00f,0x9c00-0x9c03,0x9800-0x9807,0x9400-0x9403,0x9000-0x9007 mem 0xfb80c000-0xfb80c1ff irq 17 at device 13.0 on pci2 ata6: channel #0 on atapci3 ata7: channel #1 on atapci3 fwohci0: mem 0xfb808000-0xfb80bfff,0xfb80d000-0xfb80d7ff irq 18 at device 14.0 on pci2 fwohci0: OHCI version 1.10 (ROM=3D3D1) fwohci0: No. of Isochronous channels is 4. fwohci0: EUI64 00:0d:61:56:00:7d:36:9a fwohci0: invalid speed 7 (fixed to 3). fwohci0: Phy 1394a available S800, 3 ports. fwohci0: Link S800, max_rec 4096 bytes. firewire0: on fwohci0 fwe0: on firewire0 if_fwe0: Fake Ethernet address: 02:0d:61:7d:36:9a fwe0: Ethernet address: 02:0d:61:7d:36:9a fwe0: if_start running deferred for Giant sbp0: on firewire0 fwohci0: Initiate bus reset fwohci0: node_id=3D3D0xc800ffc0, gen=3D3D1, CYCLEMASTER mode firewire0: 1 nodes, maxhop <=3D3D 0, cable IRM =3D3D 0 (me) firewire0: bus manager 0 (me) fwohci0: phy int fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 atkbdc0: port 0x64,0x60 irq 1 on acpi0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 orm0: at iomem 0xd0000-0xd17ff,0xc0000-0xcc7ff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=3D3D0x300> vga0: at port 0x3c0-0x3df iomem 0xa0000-0xbffff on isa0 Timecounter "TSC" frequency 2210769641 Hz quality 800 Timecounters tick every 1.000 msec ipfw2 initialized, divert enabled, rule-based forwarding disabled, default to accept, logging unlimited acd0: DVDROM at ata0-master PIO4 ad2: 8063MB [16383/16/63] at ata1-master UDMA33 da0 at twa0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-3 device da0: 100.000MB/s transfers da0: 1144377MB (2343684096 512 byte sectors: 255H 63S/T 145887C) da1 at twa0 bus 0 target 1 lun 0 da1: Fixed Direct Access SCSI-3 device da1: 100.000MB/s transfers da1: 190724MB (390602752 512 byte sectors: 255H 63S/T 24313C) Mounting root from ufs:/dev/ad2s1a WARNING: / was not properly dismounted WARNING: /tmp was not properly dismounted WARNING: /usr was not properly dismounted WARNING: /var was not properly dismounted