Date: Mon, 29 Jun 2009 17:13:09 +0200 From: Attilio Rao <attilio@freebsd.org> To: pluknet <pluknet@gmail.com> Cc: freebsd-stable <freebsd-stable@freebsd.org> Subject: Re: [nfs] process locks in "bo_wwait" on 6.4 Message-ID: <3bbf2fe10906290813h731a94bcrb1142e6e094a9877@mail.gmail.com> In-Reply-To: <3bbf2fe10906290801l3ed2e04ey44dd5f40f541a714@mail.gmail.com> References: <a31046fc0906260944h68eed26bs256dcad1f29871ff@mail.gmail.com> <3bbf2fe10906290458v3d57441ar44c4ed8f36c957f@mail.gmail.com> <a31046fc0906290555ib9abf4fy90502eb01e2ef8d3@mail.gmail.com> <3bbf2fe10906290611j683a0ddawbd524e406e832d54@mail.gmail.com> <a31046fc0906290618r31e5bfb0h9e267fadef53408a@mail.gmail.com> <3bbf2fe10906290620r7005f47dtfef8becc7f439515@mail.gmail.com> <a31046fc0906290659t44d11818x21dfb560ee509b44@mail.gmail.com> <3bbf2fe10906290706q349211c1pe00cc57cc18a1315@mail.gmail.com> <a31046fc0906290756s4f83c2f6jee199dad54244253@mail.gmail.com> <3bbf2fe10906290801l3ed2e04ey44dd5f40f541a714@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
2009/6/29 Attilio Rao <attilio@freebsd.org>:
> 2009/6/29 pluknet <pluknet@gmail.com>:
>> 2009/6/29 Attilio Rao <attilio@freebsd.org>:
>>> 2009/6/29 pluknet <pluknet@gmail.com>:
>>>> 2009/6/29 Attilio Rao <attilio@freebsd.org>:
>>>>> 2009/6/29 pluknet <pluknet@gmail.com>:
>>>>>> 2009/6/29 Attilio Rao <attilio@freebsd.org>:
>>>>>>> 2009/6/29 pluknet <pluknet@gmail.com>:
>>>>>>>> 2009/6/29 Attilio Rao <attilio@freebsd.org>:
>>>>>>>>> 2009/6/29 pluknet <pluknet@gmail.com>:
>>>>>>>>>> 2009/6/26 pluknet <pluknet@gmail.com>:
>>>>>>>>>>> 2009/6/26 pluknet <pluknet@gmail.com>:
>>>>>>>>>>>> Hello.
>>>>>>>>>>>>
>>>>>>>>>>>> While building a module on nfs mounted /usr/src
>>>>>>>>>>>> I got an unkillable process waiting forever in bo_wwait.
>>>>>>>>>>>
>>>>>>>>>>> Small note: iface on NFS server has mtu changed from 1500 to 1450.
>>>>>>>>>>> Can this be a source of the problem?
>>>>>>>>>>
>>>>>>>>>> This is 100% reproducible. Lock in the same place. Any hints?
>>>>>>>>>
>>>>>>>>> Can you also show the value of ps?
>>>>>>>>> A precise map of what processes are doing would give an help.
>>>>>>>>> Also would be useful to printout traces for other threads and not only
>>>>>>>>> the stucked one.
>>>>>>>>>
>>>>>>>>
>>>>>>>> >From another run:
>>>>>>>
>>>>>>> I'm unable to see who would be locking the buffer object in question.
>>>>>>> Do you have INVARIANT_SUPPORT/INVARIANTS on?
>>>>>>
>>>>>> Yes, I do both.
>>>>>>
>>>>>>> What revision of /usr/src/sys/kern/vfs_bio.c are you running with?
>>>>>>>
>>>>>>
>>>>>> As of 6.4-R: CVS rev 1.491.2.12.4.1 / SVN rev 183531.
>>>>>
>>>>> Please try this patch and report.
>>>>>
>>>>> Thanks,
>>>>> Attilio
>>>>>
>>>>> --- src/sys/nfsclient/nfs_vnops.c 2008/02/13 20:44:18 1.281
>>>>> +++ src/sys/nfsclient/nfs_vnops.c 2008/03/22 09:15:15 1.282
>>>>> @@ -33,7 +33,7 @@
>>>>> */
>>>>>
>>>>> #include <sys/cdefs.h>
>>>>> -__FBSDID("$FreeBSD:
>>>>> /usr/local/www/cvsroot/FreeBSD/src/sys/nfsclient/nfs_vnops.c,v 1.281
>>>>> 2008/02/13 20:44:18 attilio Exp $");
>>>>> +__FBSDID("$FreeBSD:
>>>>> /usr/local/www/cvsroot/FreeBSD/src/sys/nfsclient/nfs_vnops.c,v 1.282
>>>>> 2008/03/22 09:15:15 jeff Exp $");
>>>>>
>>>>
>>>> Do you refer to the whole svn r177493, or is its nfs part will be enough?
>>>> This only vfs_vnops.c diff seems not applicable without underneath
>>>> kernel part changes.
>>>>
>>>> I'll try. Thanks.
>>>
>>> The NFS part should be enough, though I don't understand why it
>>> doesn't trigger a panic on STABLE_6 as long as, at least in my
>>> revision, there is an assert for the buffer object lock to be held in
>>> bufobj_wwait(). What's your sys/kern/vfs_bio.c rev?
>>>
>>
>> As of 6.4-R.
>> $FreeBSD: src/sys/kern/vfs_bio.c,v 1.491.2.12.4.1 2008/10/02 02:57:24
>> kensmith Exp $
>
> That's it, the revision doesn't have the assert.
> If it does fix the problem for you, I will let you test a more
> comprehensive patch as there is also at least another fix I want to
> bring in along with this one (and the relative asserts).
Uhm, wait, after better looking at the code I don't think this patch
can fix your problem.
I will let you know with a bit of more time to study the deadlock.
Thanks,
Attilio
--
Peace can only be achieved by understanding - A. Einstein
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3bbf2fe10906290813h731a94bcrb1142e6e094a9877>