From owner-freebsd-fs@FreeBSD.ORG Wed Oct 26 10:29:14 2011 Return-Path: Delivered-To: fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 73613106566B for ; Wed, 26 Oct 2011 10:29:14 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.emeryville.ca.mail.comcast.net (qmta09.emeryville.ca.mail.comcast.net [76.96.30.96]) by mx1.freebsd.org (Postfix) with ESMTP id 594BD8FC15 for ; Wed, 26 Oct 2011 10:29:14 +0000 (UTC) Received: from omta13.emeryville.ca.mail.comcast.net ([76.96.30.52]) by qmta09.emeryville.ca.mail.comcast.net with comcast id pNFq1h00317UAYkA9NFxfV; Wed, 26 Oct 2011 10:15:57 +0000 Received: from koitsu.dyndns.org ([67.180.84.87]) by omta13.emeryville.ca.mail.comcast.net with comcast id pNEx1h0051t3BNj8ZNExRq; Wed, 26 Oct 2011 10:14:57 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 94615102C1A; Wed, 26 Oct 2011 03:16:02 -0700 (PDT) Date: Wed, 26 Oct 2011 03:16:02 -0700 From: Jeremy Chadwick To: Karli Sj?berg Message-ID: <20111026101602.GA9768@icarus.home.lan> References: <82B38DBF-DD3A-46CD-93F6-02CDB6506E05@slu.se> <20111025193302.GA30409@nargothrond.kdm.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Cc: "freebsd-scsi@freebsd.org" , "Kenneth D. Merry" , "fs@freebsd.org" Subject: Re: AOC-USAS2-L8i zfs panics and SCSI errors in messages X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Oct 2011 10:29:14 -0000 On Wed, Oct 26, 2011 at 11:36:44AM +0200, Karli Sj?berg wrote: > Hi all, > > I tracked down what causes the panics! > > I got a tip from aragon and phoenix at the forum about > /etc/periodic/security/100.chksetuid > > And to put: > daily_status_security_chksetuid_enable="NO" > into /etc/periodic.conf This is not truly the cause of the panic, it simply exacerbates it. Many of the periodic scripts will do things like iterate over all files on the filesystem looking for specific attributes, etc.. This tends to stress filesystems heavily. This isn't the only one. :-) > I can now run periodic daily without any panics. I?m still wondering > about the cause of this, the explanation from the forum was that that > phase is too demanding for multi TB systems. But I have several multi > TB servers with FreeBSD and ZFS, and none of them has ever behaved > this way. Besides, the panic is instantaneous, not degenerative. I > imagine that a run like that would start out OK and then just get > worse and worse, getting gradually slower and slower until it just > wouldn?t cope any more and hang. This feels more like hitting a wall. > As if it found something that is couldn?t deal with and has no choice > but to panic immediately. It may be possible that you have some underlying filesystem corruption that triggers this situation. Have you actually tried doing a "zpool scrub" of your pools and seeing if any errors happen or if the panic occurs there? I'm inclined to think what you're experiencing is probably a bug or "quirk" in the storage controller driver you're using. There are other drivers that have had fixes applied to them "to make them work decently with ZFS", meaning the kind of stressful I/O ZFS puts on them results in the controller driver behaving oddly or freaking out, case in point. It could also be a controller firmware bug/quirk/design issue. Seriously. I believe the AOC-USAS2-L8i controller has been discussed on freebsd-stable, re: mps(4) driver problems or equivalent, but I'm not going to CC that list given that there would be 3 cross-posted lists involved and that is liable to upset some folks. You should search the mailing lists for discussion of Supermicro controllers that work reliably with FreeBSD. It would be worthwhile to discuss this condition on -stable, mainly with something like "Anyone else using the AOC-USAS2-L8i reliably with ZFS?" You get the idea. -- | Jeremy Chadwick jdc at parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, US | | Making life hard for others since 1977. PGP 4BD6C0CB |