From owner-freebsd-stable@FreeBSD.ORG Mon Jun 29 15:13:11 2009 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id E0A08106576F for ; Mon, 29 Jun 2009 15:13:11 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: from mail-bw0-f211.google.com (mail-bw0-f211.google.com [209.85.218.211]) by mx1.freebsd.org (Postfix) with ESMTP id 59CAE8FC26 for ; Mon, 29 Jun 2009 15:13:11 +0000 (UTC) (envelope-from asmrookie@gmail.com) Received: by bwz7 with SMTP id 7so1674484bwz.19 for ; Mon, 29 Jun 2009 08:13:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=gamma; h=domainkey-signature:mime-version:sender:received:in-reply-to :references:date:x-google-sender-auth:message-id:subject:from:to:cc :content-type:content-transfer-encoding; bh=aNyq+9+j0+u3UEpvlFFlj41AEB0nZPMBVSUku6MIOBM=; b=ZPAN6ULwVIU1m2ZFdFmK+zTE9wOt7Wto+Ge1z4d/F0rS8qZz74ceZ7r2841otEZP8I +m5h2O0doZgKCsRaedMo7Gqp1eYBIt5RmgjXPjLai77v1n93AhvLRyGj2+dmiHRHWYP8 kQzGhauFFKbifOTdJyvSRgKTbD3PzmPVsqVzE= DomainKey-Signature: a=rsa-sha1; c=nofws; d=gmail.com; s=gamma; h=mime-version:sender:in-reply-to:references:date :x-google-sender-auth:message-id:subject:from:to:cc:content-type :content-transfer-encoding; b=E8mGFndRrEeaTb5Bu6wxJ/4gsv1ibt5ADHdUxAg1eMrj4RypAkY5X6iiV1RDY9BKYH Egv/QMqL8QzzYhz/3DF72IvY+pdnqUJbvQOjh4YoXpbqOTG0XXnlP85Nj+c3N5hNcQBA yBeTFE88Thew8cNHNvx8DcHrBOMIIbXHjPfEI= MIME-Version: 1.0 Sender: asmrookie@gmail.com Received: by 10.223.126.66 with SMTP id b2mr4515782fas.3.1246288389866; Mon, 29 Jun 2009 08:13:09 -0700 (PDT) In-Reply-To: <3bbf2fe10906290801l3ed2e04ey44dd5f40f541a714@mail.gmail.com> References: <3bbf2fe10906290458v3d57441ar44c4ed8f36c957f@mail.gmail.com> <3bbf2fe10906290611j683a0ddawbd524e406e832d54@mail.gmail.com> <3bbf2fe10906290620r7005f47dtfef8becc7f439515@mail.gmail.com> <3bbf2fe10906290706q349211c1pe00cc57cc18a1315@mail.gmail.com> <3bbf2fe10906290801l3ed2e04ey44dd5f40f541a714@mail.gmail.com> Date: Mon, 29 Jun 2009 17:13:09 +0200 X-Google-Sender-Auth: 7f2159aaf6342e38 Message-ID: <3bbf2fe10906290813h731a94bcrb1142e6e094a9877@mail.gmail.com> From: Attilio Rao To: pluknet Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Cc: freebsd-stable Subject: Re: [nfs] process locks in "bo_wwait" on 6.4 X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 29 Jun 2009 15:13:12 -0000 2009/6/29 Attilio Rao : > 2009/6/29 pluknet : >> 2009/6/29 Attilio Rao : >>> 2009/6/29 pluknet : >>>> 2009/6/29 Attilio Rao : >>>>> 2009/6/29 pluknet : >>>>>> 2009/6/29 Attilio Rao : >>>>>>> 2009/6/29 pluknet : >>>>>>>> 2009/6/29 Attilio Rao : >>>>>>>>> 2009/6/29 pluknet : >>>>>>>>>> 2009/6/26 pluknet : >>>>>>>>>>> 2009/6/26 pluknet : >>>>>>>>>>>> Hello. >>>>>>>>>>>> >>>>>>>>>>>> While building a module on nfs mounted /usr/src >>>>>>>>>>>> I got an unkillable process waiting forever in bo_wwait. >>>>>>>>>>> >>>>>>>>>>> Small note: iface on NFS server has mtu changed from 1500 to 1450. >>>>>>>>>>> Can this be a source of the problem? >>>>>>>>>> >>>>>>>>>> This is 100% reproducible. Lock in the same place. Any hints? >>>>>>>>> >>>>>>>>> Can you also show the value of ps? >>>>>>>>> A precise map of what processes are doing would give an help. >>>>>>>>> Also would be useful to printout traces for other threads and not only >>>>>>>>> the stucked one. >>>>>>>>> >>>>>>>> >>>>>>>> >From another run: >>>>>>> >>>>>>> I'm unable to see who would be locking the buffer object in question. >>>>>>> Do you have INVARIANT_SUPPORT/INVARIANTS on? >>>>>> >>>>>> Yes, I do both. >>>>>> >>>>>>> What revision of /usr/src/sys/kern/vfs_bio.c are you running with? >>>>>>> >>>>>> >>>>>> As of 6.4-R: CVS rev 1.491.2.12.4.1 / SVN rev 183531. >>>>> >>>>> Please try this patch and report. >>>>> >>>>> Thanks, >>>>> Attilio >>>>> >>>>> --- src/sys/nfsclient/nfs_vnops.c 2008/02/13 20:44:18 1.281 >>>>> +++ src/sys/nfsclient/nfs_vnops.c 2008/03/22 09:15:15 1.282 >>>>> @@ -33,7 +33,7 @@ >>>>> */ >>>>> >>>>> #include >>>>> -__FBSDID("$FreeBSD: >>>>> /usr/local/www/cvsroot/FreeBSD/src/sys/nfsclient/nfs_vnops.c,v 1.281 >>>>> 2008/02/13 20:44:18 attilio Exp $"); >>>>> +__FBSDID("$FreeBSD: >>>>> /usr/local/www/cvsroot/FreeBSD/src/sys/nfsclient/nfs_vnops.c,v 1.282 >>>>> 2008/03/22 09:15:15 jeff Exp $"); >>>>> >>>> >>>> Do you refer to the whole svn r177493, or is its nfs part will be enough? >>>> This only vfs_vnops.c diff seems not applicable without underneath >>>> kernel part changes. >>>> >>>> I'll try. Thanks. >>> >>> The NFS part should be enough, though I don't understand why it >>> doesn't trigger a panic on STABLE_6 as long as, at least in my >>> revision, there is an assert for the buffer object lock to be held in >>> bufobj_wwait(). What's your sys/kern/vfs_bio.c rev? >>> >> >> As of 6.4-R. >> $FreeBSD: src/sys/kern/vfs_bio.c,v 1.491.2.12.4.1 2008/10/02 02:57:24 >> kensmith Exp $ > > That's it, the revision doesn't have the assert. > If it does fix the problem for you, I will let you test a more > comprehensive patch as there is also at least another fix I want to > bring in along with this one (and the relative asserts). Uhm, wait, after better looking at the code I don't think this patch can fix your problem. I will let you know with a bit of more time to study the deadlock. Thanks, Attilio -- Peace can only be achieved by understanding - A. Einstein