From owner-freebsd-fs@FreeBSD.ORG  Thu Aug 29 10:00:42 2013
Return-Path: <owner-freebsd-fs@FreeBSD.ORG>
Delivered-To: freebsd-fs@FreeBSD.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTP id 1C1EC511
 for <freebsd-fs@FreeBSD.org>; Thu, 29 Aug 2013 10:00:42 +0000 (UTC)
 (envelope-from avg@FreeBSD.org)
Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140])
 by mx1.freebsd.org (Postfix) with ESMTP id 679AE255F
 for <freebsd-fs@FreeBSD.org>; Thu, 29 Aug 2013 10:00:41 +0000 (UTC)
Received: from porto.starpoint.kiev.ua (porto-e.starpoint.kiev.ua
 [212.40.38.100])
 by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id NAA24663;
 Thu, 29 Aug 2013 13:00:38 +0300 (EEST)
 (envelope-from avg@FreeBSD.org)
Received: from localhost ([127.0.0.1])
 by porto.starpoint.kiev.ua with esmtp (Exim 4.34 (FreeBSD))
 id 1VEz1q-0002nE-Jw; Thu, 29 Aug 2013 13:00:38 +0300
Message-ID: <521F1B74.7020402@FreeBSD.org>
Date: Thu, 29 Aug 2013 12:59:16 +0300
From: Andriy Gapon <avg@FreeBSD.org>
User-Agent: Mozilla/5.0 (X11; FreeBSD amd64;
 rv:17.0) Gecko/20130810 Thunderbird/17.0.8
MIME-Version: 1.0
To: Kuang-che Wu <kuangche@gmail.com>
Subject: Re: zfs dead lock
References: <CAOZYDX6FK0WcVqJZOLoWNb7P+wNEpt2oiLD2PMhDP27q5aLQsA@mail.gmail.com>
In-Reply-To: <CAOZYDX6FK0WcVqJZOLoWNb7P+wNEpt2oiLD2PMhDP27q5aLQsA@mail.gmail.com>
X-Enigmail-Version: 1.5.1
Content-Type: text/plain; charset=ISO-8859-1
Content-Transfer-Encoding: 7bit
Cc: freebsd-fs@FreeBSD.org
X-BeenThere: freebsd-fs@freebsd.org
X-Mailman-Version: 2.1.14
Precedence: list
List-Id: Filesystems <freebsd-fs.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-fs>
List-Post: <mailto:freebsd-fs@freebsd.org>
List-Help: <mailto:freebsd-fs-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-fs>,
 <mailto:freebsd-fs-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Thu, 29 Aug 2013 10:00:42 -0000

on 15/08/2013 21:36 Kuang-che Wu said the following:
> I suspect I encountered zfs deadlock yesterday.(According to
> https://wiki.freebsd.org/AvgZfsDeadlockDebug)
> Following is the report.
> 
> I have two zpool -- one is named zroot and the other is zdata. They are
> both raidz2 on same set of 6 disks.
> While zroot works well, any processes hang if they touch zdata.
> And those processes hang forever (at least more than half hours).
> 
> Before zdata hang, there are about 10 processes busy read/write to zdata.
> Their read/write patterns are linear read/write lots of large files.
> (download hundred MB files and calculate checksum)
> Average zpool write speed about 10MB/s.
> 
> In output of procstat, I found 8 processes have zio_wait + zio_done in
> stacks and this is very
> likely a ZFS deadlock according to the wiki page.
> 
> 
> System: FreeBSD kcwu.csie.org 10.0-CURRENT FreeBSD 10.0-CURRENT #5 b37d7ce:
> Tue Jul 30 21:25:46 CST 2013 root@kcwu.csie.org:/usr/obj/usr/src/sys/DESKTOP
> amd64
> 
> 
> 
> procstat log and boot dmesg are attached.
> 
> Please let me know if you need more information.

Thank you very much for the report!  And sorry for the delay.
Do you still have this locked up system?
Or are you able to reproduce the lock up?
I would like to examine some things with kgdb.

-- 
Andriy Gapon