From owner-freebsd-fs@FreeBSD.ORG Tue Oct 12 15:18:54 2010 Return-Path: Delivered-To: freebsd-fs@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id D261F1065670 for ; Tue, 12 Oct 2010 15:18:54 +0000 (UTC) (envelope-from jdc@koitsu.dyndns.org) Received: from qmta09.westchester.pa.mail.comcast.net (qmta09.westchester.pa.mail.comcast.net [76.96.62.96]) by mx1.freebsd.org (Postfix) with ESMTP id 795A88FC12 for ; Tue, 12 Oct 2010 15:18:53 +0000 (UTC) Received: from omta13.westchester.pa.mail.comcast.net ([76.96.62.52]) by qmta09.westchester.pa.mail.comcast.net with comcast id Hmnm1f00417dt5G59rJuTW; Tue, 12 Oct 2010 15:18:54 +0000 Received: from koitsu.dyndns.org ([98.248.41.155]) by omta13.westchester.pa.mail.comcast.net with comcast id HrJt1f00B3LrwQ23ZrJthF; Tue, 12 Oct 2010 15:18:54 +0000 Received: by icarus.home.lan (Postfix, from userid 1000) id 0EAB79B425; Tue, 12 Oct 2010 08:18:52 -0700 (PDT) Date: Tue, 12 Oct 2010 08:18:52 -0700 From: Jeremy Chadwick To: Andriy Gapon Message-ID: <20101012151852.GA35014@icarus.home.lan> References: <20101011151508.GA10917@icarus.home.lan> <4CB32C75.2060000@icyb.net.ua> <20101011183707.GA13925@icarus.home.lan> <4CB3870F.7070107@icyb.net.ua> <20101012100709.GA29861@icarus.home.lan> <4CB4429C.9040109@icyb.net.ua> <20101012130245.GA32584@icarus.home.lan> <4CB46CE9.20905@icyb.net.ua> <20101012143559.GA34396@icarus.home.lan> <4CB47355.1050109@icyb.net.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <4CB47355.1050109@icyb.net.ua> User-Agent: Mutt/1.5.21 (2010-09-15) Cc: freebsd-fs@freebsd.org Subject: Re: Locked up processes after upgrade to ZFS v15 X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 12 Oct 2010 15:18:55 -0000 On Tue, Oct 12, 2010 at 05:40:21PM +0300, Andriy Gapon wrote: > on 12/10/2010 17:35 Jeremy Chadwick said the following: > > Nope, but since it's a single-CPU system I can probably enable it > > safely (I thought I remember something about CTF not working right on > > SMP systems...). Give me a little while to figure out how to get it > > Never heard about such issue and never hit them on my two-core systems. What I'm remembering (two issues) are below, but apply to RELENG_7. Sounds like issue may have gotten addressed in RELENG_8. http://lists.freebsd.org/pipermail/freebsd-stable/2008-September/045093.html http://lists.freebsd.org/pipermail/freebsd-stable/2008-September/045180.html > > working, assuming you're willing to step me through what I need to do to > > simulate the lock condition. :-) > > Enabling DTrace is really easy: > http://wiki.freebsd.org/DTrace > > I would appreciate if you could enable it on your test system. Got it -- just finished and is currently running/working. I also installed ports/sysutils/DTraceToolkit and shells/ksh93 "just in case". testbox# dtrace -l | head ID PROVIDER MODULE FUNCTION NAME 1 dtrace BEGIN 2 dtrace END 3 dtrace ERROR 4 dtmalloc fbt malloc 5 dtmalloc fbt free 6 dtmalloc cyclic malloc 7 dtmalloc cyclic free 8 dtmalloc zones_data malloc 9 dtmalloc zones_data free I can provide you root-level access to the box as well as serial console if you'd prefer to do the debugging yourself, otherwise step me through what's needed and I'll be happy to act as remote hands. This should prove educational in another way, as I use Solaris regularly at work but have never sat down to tinker with DTrace (just basic mdb and adb). Fun fun fun. -- | Jeremy Chadwick jdc@parodius.com | | Parodius Networking http://www.parodius.com/ | | UNIX Systems Administrator Mountain View, CA, USA | | Making life hard for others since 1977. PGP: 4BD6C0CB |