From owner-freebsd-fs@FreeBSD.ORG  Thu Jul  4 13:44:03 2013
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 by hub.freebsd.org (Postfix) with ESMTP id 1E9FC684
 for <freebsd-fs@FreeBSD.org>; Thu,  4 Jul 2013 13:44:03 +0000 (UTC)
 (envelope-from bofh@terranova.net)
Received: from tog.net (tog.net [IPv6:2605:5a00::5])
 by mx1.freebsd.org (Postfix) with ESMTP id E99BB1722
 for <freebsd-fs@FreeBSD.org>; Thu,  4 Jul 2013 13:44:02 +0000 (UTC)
Received: from [IPv6:2605:5a00:ffff::face] (unknown
 [IPv6:2605:5a00:ffff::face])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by tog.net (Postfix) with ESMTPSA id 3bmL4x1VRxz6X8
 for <freebsd-fs@FreeBSD.org>; Thu,  4 Jul 2013 09:44:01 -0400 (EDT)
Message-ID: <51D57C19.1080906@terranova.net>
Date: Thu, 04 Jul 2013 09:43:53 -0400
From: Travis Mikalson <bofh@terranova.net>
Organization: TerraNovaNet Internet Services
User-Agent: Thunderbird 2.0.0.24 (Windows/20100228)
MIME-Version: 1.0
To: freebsd-fs@FreeBSD.org
Subject: Re: Report: ZFS deadlock in 9-STABLE
References: <51D45401.5050801@terranova.net> <51D5776F.5060101@FreeBSD.org>
In-Reply-To: <51D5776F.5060101@FreeBSD.org>
X-Enigmail-Version: 0.96.0
OpenPGP: url=http://www.terranova.net/pgp/bofh
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 04 Jul 2013 13:44:03 -0000

Andriy Gapon wrote:
> on 03/07/2013 19:40 Travis Mikalson said the following:
>> Hello,
>>
>> To cut to the chase, I have a procstat -kk -a captured during a livelock
>> for you here:
>> http://tog.net/freebsd/zfsdeadlock-storage1-20130703
> 
> BTW, https://wiki.freebsd.org/AvgZfsDeadlockDebug
> 
> "If neither of the above is true. That is, you do see zio_wait and you don't see
> either of zio_done or zio_interrupt, then the problem is most likely with the
> storage layer..."

Yes, that helpful article is where I got the run-down on how best to
report what was going on here. I still believe this is an actual
deadlock bug and not a storage layer issue.

I have not seen any indications of any problems with my storage layer.
You'd think there would be some scary-looking complaint on the console
during one of these deadlocks if it had suddenly lost the capability to
communicate with most or all the disks, but I've deadlocked at least 10
times now in 2013 and never anything of the sort. Thanks to IPMI, I have
actually viewed the console each time it has happened.