From owner-freebsd-hackers@FreeBSD.ORG  Fri Nov 21 06:25:29 2014
Return-Path: <owner-freebsd-hackers@FreeBSD.ORG>
Delivered-To: freebsd-hackers@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org
 [IPv6:2001:1900:2254:206a::19:1])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id E0778C8F;
 Fri, 21 Nov 2014 06:25:28 +0000 (UTC)
Received: from mail-ob0-x22d.google.com (mail-ob0-x22d.google.com
 [IPv6:2607:f8b0:4003:c01::22d])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id A29D6C97;
 Fri, 21 Nov 2014 06:25:28 +0000 (UTC)
Received: by mail-ob0-f173.google.com with SMTP id uy5so3558000obc.32
 for <multiple recipients>; Thu, 20 Nov 2014 22:25:28 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=LuY5HVqr7s82gpK37dkkFaEebxzR7jcGV8yE8o1rxUo=;
 b=zsgS+wIuxLS380s0W4sz3DMRRdkJH0Jh2mUni2FBBhDxkUkYI4oiZL/KNaYrkqzEX1
 p7dW/j0bhhnLSIB0mcf8MSBxnSWanL61eaCUA/PMOQtai4YGV4EC7aTiRGPkUephFw0b
 yBycPnAFkCsXaZqpxbwHmLzCCLRFcX55mWwqxXUV/w+rvmIrDeehDWUlN3OZmJLBy564
 0u66/y83qCXn8uZrdxbHlsRPUofhOYYy2m5Kn+cXRCL1tuaw26vRjDyVuHUC9+/yL8iJ
 eX6DAKG4g19gayMPF15H0TXmBu2ORnSXSPWPbXXnDyCpwRHNMNDF0CsfGJHiUSkkhpoA
 1r5Q==
MIME-Version: 1.0
X-Received: by 10.182.65.105 with SMTP id w9mr707416obs.60.1416551128044; Thu,
 20 Nov 2014 22:25:28 -0800 (PST)
Received: by 10.76.0.138 with HTTP; Thu, 20 Nov 2014 22:25:27 -0800 (PST)
In-Reply-To: <CAHM0Q_O1ZN6cfuXUboits0iwpCAX7dZpFhaoKiOM=6BjEn+2+A@mail.gmail.com>
References: <CACpH0Mcm57vRq=RLjwhbc6sMwAv=zVQvftt-t2MOUV4tQjB5BQ@mail.gmail.com>
 <CAHM0Q_O1ZN6cfuXUboits0iwpCAX7dZpFhaoKiOM=6BjEn+2+A@mail.gmail.com>
Date: Fri, 21 Nov 2014 01:25:27 -0500
Message-ID: <CACpH0Mdk4ZC8-ZiQDQYwZzSdtJZCxY2RS6-DeyigD02G4nJYkA@mail.gmail.com>
Subject: Re: Debugging the ZDB debugger.
From: Zaphod Beeblebrox <zbeeble@gmail.com>
To: "K. Macy" <kmacy@freebsd.org>
Content-Type: text/plain; charset=UTF-8
X-Content-Filtered-By: Mailman/MimeDel 2.1.18-1
Cc: FreeBSD Hackers <freebsd-hackers@freebsd.org>
X-BeenThere: freebsd-hackers@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: Technical Discussions relating to FreeBSD
 <freebsd-hackers.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-hackers>, 
 <mailto:freebsd-hackers-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-hackers/>
List-Post: <mailto:freebsd-hackers@freebsd.org>
List-Help: <mailto:freebsd-hackers-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-hackers>,
 <mailto:freebsd-hackers-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 21 Nov 2014 06:25:29 -0000

Ok... that advice got me somewhere... now my stack is:

(gdb) bt
#0  0x00000000004098a9 in dump_dir (os=0x80d302000)
    at
/usr/src/cddl/usr.sbin/zdb/../../../cddl/contrib/opensolaris/cmd/zdb/zdb.c:1464
#1  0x0000000000406222 in main (argc=0, argv=<value optimized out>)
    at
/usr/src/cddl/usr.sbin/zdb/../../../cddl/contrib/opensolaris/cmd/zdb/zdb.c:3604

and we got here on a segmentation fault.  Now ... I'm a little confused:

(gdb) frame 0
#0  0x00000000004098a9 in dump_dir (os=0x80d302000)
    at
/usr/src/cddl/usr.sbin/zdb/../../../cddl/contrib/opensolaris/cmd/zdb/zdb.c:1464
1464            zdb_nicenum(dl->dl_phys->dl_used, bytes);
(gdb) p dl
No symbol "dl" in current context.
(gdb) p *dl
No symbol "dl" in current context.

I thought for a second that I was using gdb wrong (it's been awhile), but:

(gdb) frame 1
#1  0x0000000000406222 in main (argc=0, argv=<value optimized out>)
    at
/usr/src/cddl/usr.sbin/zdb/../../../cddl/contrib/opensolaris/cmd/zdb/zdb.c:3604
3604                            dump_dir(os);
(gdb) p os
$3 = (objset_t *) 0x80d302000

... my first thought was "is the stack trashed"? ... but shouldn't gdb know
what 'dl' is regardless of the process state?

Then I realized that line 1464 isn't in dump_dir() ... it's in dump_dead()

help?