From owner-freebsd-stable@FreeBSD.ORG Thu Aug 18 00:35:38 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id C73D9106566B; Thu, 18 Aug 2011 00:35:38 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-yx0-f182.google.com (mail-yx0-f182.google.com [209.85.213.182]) by mx1.freebsd.org (Postfix) with ESMTP id 5D32F8FC0C; Thu, 18 Aug 2011 00:35:38 +0000 (UTC) Received: by yxn22 with SMTP id 22so254950yxn.13 for ; Wed, 17 Aug 2011 17:35:37 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; bh=nwJCiqzwXhEcptHUzCuSNX6Tqtwr9zjUxyoSseNFsi8=; b=qQrWvcZ9eyLkxMPJGE9od7N9AFoXHEpM+5hs+lMTqK9Cv5sighZh+dw5fYXAD3QIsl WDV1H/80GNMSiWqjkLLrzsgPnV6VlCsYE92Ahuxnx4jUa2M0DGs5lG3Qp08WeBYghNcY d983McfPYNhLo8AeJ5jQUU+JdU2lwEaVrzl/A= MIME-Version: 1.0 Received: by 10.236.170.9 with SMTP id o9mr11588yhl.43.1313627737497; Wed, 17 Aug 2011 17:35:37 -0700 (PDT) Sender: asmrookie@gmail.com Received: by 10.236.108.33 with HTTP; Wed, 17 Aug 2011 17:35:37 -0700 (PDT) In-Reply-To: <20110818.091600.831954331552558249.hrs@allbsd.org> References: <20110818.023832.373949045518579359.hrs@allbsd.org> <20110818.043332.27079545013461535.hrs@allbsd.org> <20110818.091600.831954331552558249.hrs@allbsd.org> Date: Thu, 18 Aug 2011 02:35:37 +0200 X-Google-Sender-Auth: MCw4hh_Hde0OfacevQCtfvzP3CU Message-ID: From: Attilio Rao To: Hiroki Sato Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Cc: kostikbel@gmail.com, freebsd-stable@freebsd.org, avg@freebsd.org Subject: Re: panic: spin lock held too long (RELENG_8 from today) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 18 Aug 2011 00:35:39 -0000 2011/8/18 Hiroki Sato : > Hiroki Sato wrote > =C2=A0in <20110818.043332.27079545013461535.hrs@allbsd.org>: > > hr> Attilio Rao wrote > hr> =C2=A0 in : > hr> > hr> at> 2011/8/17 Hiroki Sato : > hr> at> > Hi, > hr> at> > > hr> at> > Mike Tancsa wrote > hr> at> > =C2=A0in <4E15A08C.6090407@sentex.net>: > hr> at> > > hr> at> > mi> On 7/7/2011 7:32 AM, Mike Tancsa wrote: > hr> at> > mi> > On 7/7/2011 4:20 AM, Kostik Belousov wrote: > hr> at> > mi> >> > hr> at> > mi> >> BTW, we had a similar panic, "spinlock held too long", t= he spinlock > hr> at> > mi> >> is the sched lock N, on busy 8-core box recently upgrade= d to the > hr> at> > mi> >> stable/8. Unfortunately, machine hung dumping core, so t= he stack trace > hr> at> > mi> >> for the owner thread was not available. > hr> at> > mi> >> > hr> at> > mi> >> I was unable to make any conclusion from the data that w= as present. > hr> at> > mi> >> If the situation is reproducable, you coulld try to reve= rt r221937. This > hr> at> > mi> >> is pure speculation, though. > hr> at> > mi> > > hr> at> > mi> > Another crash just now after 5hrs uptime. I will try and = revert r221937 > hr> at> > mi> > unless there is any extra debugging you want me to add to= the kernel > hr> at> > mi> > instead =C2=A0? > hr> at> > > hr> at> > =C2=A0I am also suffering from a reproducible panic on an 8-STA= BLE box, an > hr> at> > =C2=A0NFS server with heavy I/O load. =C2=A0I could not get a k= ernel dump > hr> at> > =C2=A0because this panic locked up the machine just after it oc= curred, but > hr> at> > =C2=A0according to the stack trace it was the same as posted on= e. > hr> at> > =C2=A0Switching to an 8.2R kernel can prevent this panic. > hr> at> > > hr> at> > =C2=A0Any progress on the investigation? > hr> at> > hr> at> Hiroki, > hr> at> how easilly can you reproduce it? > hr> > hr> =C2=A0It takes 5-10 hours. =C2=A0I installed another kernel for debug= ging just > hr> =C2=A0now, so I think I will be able to collect more detail informati= on in > hr> =C2=A0a couple of days. > hr> > hr> at> It would be important to have a DDB textdump with these informati= ons: > hr> at> - bt > hr> at> - ps > hr> at> - show allpcpu > hr> at> - alltrace > hr> at> > hr> at> Alternatively, a coredump which has the stop cpu patch which Andr= yi can provide. > hr> > hr> =C2=A0Okay, I will post them once I can get another panic. =C2=A0Than= ks! > > =C2=A0I got the panic with a crash dump this time. =C2=A0The result of bt= , ps, > =C2=A0allpcpu, and traces can be found at the following URL: > > =C2=A0http://people.allbsd.org/~hrs/FreeBSD/pool-panic_20110818-1.txt I'm not sure I understand it, is also a corefile available? If yes, where I could get it? (with the relevant sources and kernel.debug). Thanks, Attilio --=20 Peace can only be achieved by understanding - A. Einstein