From owner-freebsd-stable@FreeBSD.ORG Tue Jul 26 23:12:24 2011 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id CB6341065672; Tue, 26 Jul 2011 23:12:24 +0000 (UTC) (envelope-from rmacklem@uoguelph.ca) Received: from esa-jnhn.mail.uoguelph.ca (esa-jnhn.mail.uoguelph.ca [131.104.91.44]) by mx1.freebsd.org (Postfix) with ESMTP id 6CCB98FC19; Tue, 26 Jul 2011 23:12:24 +0000 (UTC) X-IronPort-Anti-Spam-Filtered: true X-IronPort-Anti-Spam-Result: Ap4EAHZJL06DaFvO/2dsb2JhbAA1AQEFKVwdGAICDSUCFlEHJIRJo3KJALA/kS6BK4QGgQ8EknKQfA X-IronPort-AV: E=Sophos;i="4.67,272,1309752000"; d="scan'208";a="132358196" Received: from erie.cs.uoguelph.ca (HELO zcs3.mail.uoguelph.ca) ([131.104.91.206]) by esa-jnhn-pri.mail.uoguelph.ca with ESMTP; 26 Jul 2011 19:12:23 -0400 Received: from zcs3.mail.uoguelph.ca (localhost.localdomain [127.0.0.1]) by zcs3.mail.uoguelph.ca (Postfix) with ESMTP id 612F3B3F3A; Tue, 26 Jul 2011 19:12:23 -0400 (EDT) Date: Tue, 26 Jul 2011 19:12:23 -0400 (EDT) From: Rick Macklem To: Kostik Belousov Message-ID: <132828699.1045046.1311721943354.JavaMail.root@erie.cs.uoguelph.ca> In-Reply-To: <20110726093258.GF17489@deviant.kiev.zoral.com.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Originating-IP: [172.17.91.202] X-Mailer: Zimbra 6.0.10_GA_2692 (ZimbraWebClient - FF3.0 (Win)/6.0.10_GA_2692) Cc: rmacklem@freebsd.org, Herve Boulouis , freebsd-stable@freebsd.org Subject: Re: Sleeping thread owns a nonsleepable lock panic (& lor) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 26 Jul 2011 23:12:24 -0000 Kostik Belousov wrote: > On Tue, Jul 26, 2011 at 01:17:52PM +0200, Herve Boulouis wrote: > > Le 26/07/2011 12:06, Kostik Belousov a =D0=98crit: > > > On Tue, Jul 26, 2011 at 11:49:13AM +0200, Herve Boulouis wrote: > > > > Le 25/07/2011 11:59, Kostik Belousov a ?crit: > > > > > > > > Ok the patched server crashed this morning strangely : all httpd > > > > processes were stuck in nfs or vmopar > > > > and were unkillable. Below is the full ps. > > > > > > Please see the > > > http://www.freebsd.org/doc/en_US.ISO8859-1/books/developers-handbook/= kerneldebug-deadlocks.html > > > for information required to debug the deadlocks. > > > > the box was not stricly deadlocked since I was able to interact with > > it but I suppose you want me to > > break into debugger when the symptoms appears again and report all > > the commands listed in the handbook > > deadlock section ? >=20 > Exactly. >=20 > I think everything was hung that accessed an nfs mount point. > From the usermode, procstat -kk could catch some interesting > information, > but it is redundant if ddb output is captured. Would it be worth considering reverting r223054? (Note that I don't understand the VM side, so this may be completely wrong:-) The sleeps on vmopar could be happening because a dirty page is busy and r223054 changes the VM_PAGER_xx value set a couple of ways. 1 - When it returns VM_PAGER_ERROR instead of VM_PAGER_AGAIN, the return value of "runlen" from vm_pageout_flush() changes. 2 - I'm not sure, but I think the pre-r223054 code marked a partially written page as VM_PAGER_OK instead of VM_PAGER_AGAIN? (I'm wondering about this one, since the problem seems to happen when the file's size has been truncated.) Herve Boulouis, if you want to see what r223054 changes, just go to http://svn.freebsd.org/viewvc/stable/8/sys/nfsclient and then click on nfs_bio.c. (The changes are small and could easily be reverted with a manual edit.) Since r223054 went into stable/8 on Jun 13, it seems a possible explanation? rick