From owner-freebsd-stable@FreeBSD.ORG Wed Oct 15 17:54:57 2008 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 534A1106568F; Wed, 15 Oct 2008 17:54:57 +0000 (UTC) (envelope-from p.christias@noc.ntua.gr) Received: from diomedes.noc.ntua.gr (diomedes.noc.ntua.gr [IPv6:2001:648:2000:de::220]) by mx1.freebsd.org (Postfix) with ESMTP id 5A1018FC15; Wed, 15 Oct 2008 17:54:56 +0000 (UTC) (envelope-from p.christias@noc.ntua.gr) Received: from ajax.noc.ntua.gr (ajax6.noc.ntua.gr [IPv6:2001:648:2000:dc::1]) by diomedes.noc.ntua.gr (8.14.0/8.14.0) with ESMTP id m9FHsrkv010592 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 20:54:54 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) Received: from ajax.noc.ntua.gr (localhost.noc.ntua.gr [127.0.0.1]) by ajax.noc.ntua.gr (8.13.8/8.13.8) with ESMTP id m9FHsrGP005474 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NOT); Wed, 15 Oct 2008 20:54:53 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) Received: (from christia@localhost) by ajax.noc.ntua.gr (8.13.8/8.13.8/Submit) id m9FHsrHh005469; Wed, 15 Oct 2008 20:54:53 +0300 (EEST) (envelope-from p.christias@noc.ntua.gr) X-Authentication-Warning: ajax.noc.ntua.gr: christia set sender to p.christias@noc.ntua.gr using -f Date: Wed, 15 Oct 2008 20:54:53 +0300 From: Panagiotis Christias To: Oleg Sharoiko Message-ID: <20081015175453.GA3260@noc.ntua.gr> References: <20081014222343.GA8706@noc.ntua.gr> <1224049455.1277.44.camel@brain.cc.rsu.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <1224049455.1277.44.camel@brain.cc.rsu.ru> User-Agent: Mutt/1.5.16 (2007-06-09) X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-3.0 (diomedes.noc.ntua.gr [IPv6:2001:648:2000:de::220]); Wed, 15 Oct 2008 20:54:54 +0300 (EEST) X-Virus-Scanned: ClamAV version 0.93.3, clamav-milter version 0.93.3 on diomedes.noc.ntua.gr X-Virus-Status: Clean Cc: freebsd-scsi@freebsd.org, freebsd-stable@freebsd.org Subject: Re: FreeBSD 7-STABLE, isp(4), QLE2462: panic & deadlocks X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 15 Oct 2008 17:54:57 -0000 On Wed, Oct 15, 2008 at 09:44:15AM +0400, Oleg Sharoiko wrote: > Hi! > > On Wed, 2008-10-15 at 01:23 +0300, Panagiotis Christias wrote: > > > However, when we connect them to the CX3-40, create and mount a new > > partition and then do something as simple as "tar -C /san -xf ports.tgz" > > the system panics and deadlocks. We have tried several FreeBSD versions > > (6.3 i386/adm64, 7.0 i386/adm64, 7.1 i386/adm64 and lastly 7-STABLE i386 > > - we also tried the latest 8-CURRENT snapshot but it panicked too soon). > > The result is always the same; panic and deadlock. > > Try reducing the number of "tagged openings" with 'camcontrol tags' down > to 46. If it doesn't work try reducing it further to 2. Also be advised > that I've seen panics with geom_multipath in FreeBSD-7, unfortunately I > had no time to test it in -current. Hm.. that would probably explain the fact that I was unable to panic the system when I had set the hint.isp.0.debug="0x1F" in /boot/device.hints. Currently I am stress testing the server with the tagged openings set to 44 (first value tested). Until now there is no panic or deadlock. I am trying concurrent tar extractions and rsync copies. The filesystem looks ok till now according to fsck. I will let it write/copy/delete overnight and tomorrow I will try different tagged opening values. Thank you for the hint! I am wondering what is the performance penalty with decreased tagged openings. Also, is there anything else I could try in order to get more useful debug output? I have at least three servers that I could use for any kind of tests and I am willing to spend as much time I can get to help solving the problem. Finally, the only output in the logs is: Expensive timeout(9) function: 0xc06f4210(0xc67e1200) 0.059422635 s Expensive timeout(9) function: 0xc08d4fd0(0) 0.060676147 s I suppose that is related to the CAMDEBUG kernel config options. Regards, Panagiotis -- Panagiotis J. Christias Network Management Center P.Christias@noc.ntua.gr National Technical Univ. of Athens, GREECE