From owner-freebsd-current@FreeBSD.ORG  Fri Jun  6 14:51:51 2014
Return-Path: <owner-freebsd-current@FreeBSD.ORG>
Delivered-To: freebsd-current@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 9C91948F;
 Fri,  6 Jun 2014 14:51:51 +0000 (UTC)
Received: from mail-ob0-x22b.google.com (mail-ob0-x22b.google.com
 [IPv6:2607:f8b0:4003:c01::22b])
 (using TLSv1 with cipher ECDHE-RSA-RC4-SHA (128/128 bits))
 (Client CN "smtp.gmail.com",
 Issuer "Google Internet Authority G2" (verified OK))
 by mx1.freebsd.org (Postfix) with ESMTPS id 5D6CF29B6;
 Fri,  6 Jun 2014 14:51:51 +0000 (UTC)
Received: by mail-ob0-f171.google.com with SMTP id wn1so2909740obc.16
 for <multiple recipients>; Fri, 06 Jun 2014 07:51:50 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=mime-version:in-reply-to:references:date:message-id:subject:from:to
 :cc:content-type;
 bh=E1yHhkIney906tFBr2dlLcDwzedbCq10YE31ayrHUY8=;
 b=ykNqy4cmXebof+hh2xs69iRXZoDpIcS+XYD0scW2Gkzs3xB4piWzJQEgawd4CzYa8p
 05eQ00LpZmn3LpRRvUjbgYYdm4cpFMGDgpXtTBq94JEK1aDENSxbr/kADX9gUD9iYRhD
 UiHP1nEBWjKgu5tct5QKTFGGb3/+BsCTvC6CUv0CSbHaGznQqYwn9Aeq2L95vZVPg6hZ
 XGS64v6Iujoy6pm+kN7Pi4aWhbU8PvooKasvby/drhEnHtOGdzDR57rC4jMCaEJTXrXb
 ufCIZjoEcC8aEqcgz+5YlxLWZKEkfpSag+YtRIZtS6id+Mp04G3MyJSxATJntA3E3Qei
 pAcw==
MIME-Version: 1.0
X-Received: by 10.182.20.169 with SMTP id o9mr5658719obe.59.1402066309941;
 Fri, 06 Jun 2014 07:51:49 -0700 (PDT)
Received: by 10.76.23.130 with HTTP; Fri, 6 Jun 2014 07:51:49 -0700 (PDT)
In-Reply-To: <20140606141215.GE33882@hub.FreeBSD.org>
References: <20140606141215.GE33882@hub.FreeBSD.org>
Date: Fri, 6 Jun 2014 10:51:49 -0400
Message-ID: <CAFMmRNy7sxJA-=O+pt=9Kr6Y03nxYZNYeRkmaSPYS1Pr5xypAQ@mail.gmail.com>
Subject: Re: panic in deadlkres() on r267110
From: Ryan Stone <rysto32@gmail.com>
To: Glen Barber <gjb@freebsd.org>
Content-Type: text/plain; charset=UTF-8
Cc: FreeBSD Current <freebsd-current@freebsd.org>
X-BeenThere: freebsd-current@freebsd.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: Discussions about the use of FreeBSD-current
 <freebsd-current.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-current>, 
 <mailto:freebsd-current-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-current/>
List-Post: <mailto:freebsd-current@freebsd.org>
List-Help: <mailto:freebsd-current-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-current>,
 <mailto:freebsd-current-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Jun 2014 14:51:51 -0000

On Fri, Jun 6, 2014 at 10:12 AM, Glen Barber <gjb@freebsd.org> wrote:
> Two machines in the cluster panic last night with the same backtrace.
> It is unclear yet exactly what was happening on the systems, but both
> are port building machines using ports-mgmt/tinderbox.
>
> Any ideas or information on how to further debug this would be
> appreciated.

The first thing that I would do is try to figure out who holds the
allproc_lock.  If the lock is held exclusively then
the pointer to the thread struct will be in allproc_lock.sx_lock.

If the lock is held shared then it gets trickier.  I'm not sure if
there is a better possible way than doing a thread apply all bt in
kgdb and finding which stack(s) could hold allproc_lock.