Date: Sat, 24 Jun 2006 16:45:49 -0300 (ADT) From: "Marc G. Fournier" <scrappy@hub.org> To: Kostik Belousov <kostikbel@gmail.com> Cc: freebsd-stable@freebsd.org, Dmitry Morozovsky <marck@rinet.ru> Subject: Re: vmstat 'b' (disk busy?) field keeps climbing ... Message-ID: <20060624164343.V1114@ganymede.hub.org> In-Reply-To: <20060624190912.GD79678@deviant.kiev.zoral.com.ua> References: <20060623172557.H1114@ganymede.hub.org> <261AD16B-C3FE-4671-996E-563053508CE8@mac.com> <20060624022227.X1114@ganymede.hub.org> <20060624115505.E14669@woozle.rinet.ru> <20060624090656.GB79678@deviant.kiev.zoral.com.ua> <20060624145432.A1114@ganymede.hub.org> <20060624185203.GC79678@deviant.kiev.zoral.com.ua> <20060624190912.GD79678@deviant.kiev.zoral.com.ua>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 24 Jun 2006, Kostik Belousov wrote:
> On Sat, Jun 24, 2006 at 09:52:03PM +0300, Kostik Belousov wrote:
>> On Sat, Jun 24, 2006 at 02:57:27PM -0300, Marc G. Fournier wrote:
>>> On Sat, 24 Jun 2006, Kostik Belousov wrote:
>>>
>>>> On Sat, Jun 24, 2006 at 11:55:26AM +0400, Dmitry Morozovsky wrote:
>>>>> On Sat, 24 Jun 2006, Marc G. Fournier wrote:
>>>>>
>>>>> MGF> > 'b' stands for "blocked", not "busy". Judging by your page fault
>>>>> rate
>>>>> MGF> > and the high number of frees and pages being scanned, you're
>>>>> probably
>>>>> MGF> > swapping tasks in and out and are waiting on disk. Take a look at
>>>>> MGF> > "vmstat -s", and consider adding more RAM if this is correct...
>>>>> MGF>
>>>>> MGF> is there a way of finding out what processes are blocked?
>>>>>
>>>>> Aren't they in 'D' status by ps?
>>>> Use ps axlww. In this way, at least actual blocking points are shown.
>>>
>>> 'k, stupid question then ... what am I searching for?
>>>
>>> # ps axlww | awk '{print $9}' | sort | uniq -c | sort -nr
>>> 654 select
>>> 230 lockf
>>> 166 wait
>>> 85 -
>>> 80 piperd
>>> 71 nanslp
>>> 33 kserel
>>> 22 user
>>> 10 pause
>>> 9 ttyin
>>> 5 sbwait
>>> 3 psleep
>>> 3 accept
>>> 2 kqread
>>> 2 Giant
>>> 1 vlruwt
>>> 1 syncer
>>> 1 sdflus
>>> 1 ppwait
>>> 1 ktrace
>>> 1 MWCHAN
>>>
>>> According to vmstat, I'm holding at '4 blocked' for the most part ...
>>> sbwwait is socket related, not disk ... and none of the others look right
>>> ...
>> I would say, using big magic cristall ball, that you problems are
>> not kernel-related. I see only too suspicious points:
>>
>> 1. high number of pipe readers and waiters for file locks. It may be
>> normal for your load.
>>
>> 2. 2 Giant holders/lockers. Is it constant ? Are the processes holding/waiting
>> for Giant are the same ?
>>
>> Anyway, being in your shoes, I would start looking at applications.
>>
>> Ah, and does dmesg show anything ?
>
> And another question: what are the processes in the state "user" ?
> I never see that state. More, search thru the sources does not show
> what this could be.
Odd, I'm not finding any, but, I did get a Giant on a grep of the ps listing::
pluto# ps axlww | grep " user "
0 93055 46540 0 96 0 348 212 Giant L+ p4 0:00.00 grep user
Not sure where those 'user' came from though ... just ran the above again:
# ps axlww | awk '{print $9}' | sort | uniq -c | sort -nr
603 select
231 lockf
71 nanslp
33 -
30 kserel
23 wait
9 ttyin
9 sbwait
7 pause
6 accept
4 piperd
3 psleep
3 kqread
3 Giant
1 syncer
1 sdflus
1 ppwait
1 pgzero
1 ktrace
1 MWCHAN
And nothing ...
Got a Giant lock on sshd too?
pluto# ps axlww | grep Giant
0 693 556 1 96 0 6096 2080 Giant Ls ?? 0:02.18 sshd: root@ttyp0 (sshd)
0 94334 46540 0 96 0 348 208 - R+ p4 0:00.00 grep Giant
----
Marc G. Fournier Hub.Org Networking Services (http://www.hub.org)
Email . scrappy@hub.org MSN . scrappy@hub.org
Yahoo . yscrappy Skype: hub.org ICQ . 7615664
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20060624164343.V1114>