From owner-freebsd-current@FreeBSD.ORG Sat Jan 26 13:09:37 2008 Return-Path: Delivered-To: freebsd-current@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 2C84716A418 for ; Sat, 26 Jan 2008 13:09:37 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from fg-out-1718.google.com (fg-out-1718.google.com [72.14.220.153]) by mx1.freebsd.org (Postfix) with ESMTP id A3E5113C448 for ; Sat, 26 Jan 2008 13:09:36 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by fg-out-1718.google.com with SMTP id 16so1108000fgg.35 for ; Sat, 26 Jan 2008 05:09:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:received:received:message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; bh=GQyYcX+sdAP9aHltplo+/jRb4Fgfg2CVz1r6zYhjJLQ=; b=R9ZJBLVlH/g5UQJdMM9rXL6m/6I/7rn49V5mHzxFEiMw70gZNtpYXyUsFaPDJoPFAwQfLiGWPCdVy8C0Jljnu76iQXFSJ5K23ExSrT+pvGyyDywrfd4v/W1rboMjjUUq5AU6RJL5hnwvPcTqGrIpjhEdJ+qLPtGoJDoc32FUP5s= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=message-id:date:from:sender:to:subject:cc:in-reply-to:mime-version:content-type:content-transfer-encoding:content-disposition:references:x-google-sender-auth; b=CtudfZkzqfdAuDkPogYXjqGBXDSW0+p6Um8pe1DrtYBZqM6aFZ7glH4GRwq4K7wpWJNCQuGA664CLh2JmaqFis3ECf6EXUdPDlloEstke07TATLFyt2Hj5lztUa/xo4XDrPtmTnFqvl7DEvtNfHwpx8xOVWx178yZHSSj5mS0/g= Received: by 10.86.72.15 with SMTP id u15mr2994375fga.21.1201352975231; Sat, 26 Jan 2008 05:09:35 -0800 (PST) Received: by 10.86.53.7 with HTTP; Sat, 26 Jan 2008 05:09:35 -0800 (PST) Message-ID: <3bbf2fe10801260509o12f7d5dof399a751ef48e5c7@mail.gmail.com> Date: Sat, 26 Jan 2008 14:09:35 +0100 From: "Attilio Rao" Sender: asmrookie@gmail.com To: "Scot Hetzel" In-Reply-To: <790a9fff0801252015l34f524d1k704fa0accffe2545@mail.gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Content-Disposition: inline References: <790a9fff0801150552l542a4238ofc12efe5fdb45fc2@mail.gmail.com> <20080115143924.GB57756@deviant.kiev.zoral.com.ua> <20080124122808.GA15600@freefall.freebsd.org> <3bbf2fe10801240518i6e18b2f5w84de652d4170c95b@mail.gmail.com> <790a9fff0801252015l34f524d1k704fa0accffe2545@mail.gmail.com> X-Google-Sender-Auth: ba6f17f5fd573026 Cc: freebsd-current@freebsd.org Subject: Re: panic: System call lstat returning with 1 locks held X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jan 2008 13:09:37 -0000 2008/1/26, Scot Hetzel : > On 1/24/08, Attilio Rao wrote: > > 2008/1/24, Yar Tikhiy : > > > On Tue, Jan 15, 2008 at 04:39:24PM +0200, Kostik Belousov wrote: > > > > > > > > I think this could be related to the recent vn_lock()/VOP_LOCK() KPI changes. > > > > Please, add DEBUG_VFS_LOCKS to the kernel config, and do the > > > > show lockedvnods > > > > from the ddb prompt when the panic occurs. The witness does not track > > > > the lockmgr locks. > > > > > > I think I'm seeing the same panic on UFS. It's rather nasty: I > > > cannot rebuild CURRENT natively due to it so I have to build it > > > under 6-STABLE. My favourite way to trigger the panic reliably is > > > running `make install' in a simple port directory, e.g., portmaster, > > > but my system also panics during daily scripts run and, as already > > > said, if trying to build world. > > > > Yar, > > as it seems reproducible for you, can you please add this patch to the tree: > > http://www.freebsd.org/~attilio/debug_tdlocks.diff > > > > compile your kernel with: > > options KTR > > options KTR_COMPILE=(KTR_SPARE2) > > options KTR_MASK=(KTR_SPARE2) > > options KTR_ENTRIES=32768 > > > > and once kernel panics, at ddb prompts do: > > > show ktr > > > > I added the above options to my kernel, and performed a scripted textdump. > > /sbin/ddb script lockinfo="show locks; show alllocks; show lockedvnods" > /sbin/ddb script kdb.enter.panic="textdump set; capture on; show ktr ; > run lockinfo ; show pcpu; bt; ps; alltrace; capture off; call doadump; > reset" > > After the kernel paniced, the kdb.enter.panic script ran and created a > textdump. When I extracted the 2.7M ddb.txt file, it didn't show any > calls to lockmgr_disown in the ktr trace. > > Let me know if there is anything else that I can do. > > To get this dump, DB_CAPTURE_MAXBUFSIZE (sys/ddb/db_capture.c) needed > to be increased from its default of 512K to 5M, and then setting the > debug.ddb.capure.bufsize to 5M after rebooting with the new kernel. Scot, thanks a lot for your effort. I think I will produce more patch which can help for diagnosis very soon so that we can gather more informations and see what is the real culprit of this. > See PR 119993 (http://www.freebsd.org/cgi/query-pr.cgi?pr=119993) > which adds two new kernel options to allow the capture buffer size to > be changed at compile time. Oh, there is a PR bugathon too, this week... :) Attilio -- Peace can only be achieved by understanding - A. Einstein