Date: Wed, 3 Nov 2004 19:25:45 +0300 (MSK) From: Igor Sysoev <is@rambler-co.ru> To: Uwe Doering <gemini@geminix.org> Cc: stable@freebsd.org Subject: Re: vnode_pager_putpages errors and DOS? Message-ID: <20041103191641.K63546@is.park.rambler.ru> In-Reply-To: <4168578F.7060706@geminix.org> References: <Pine.NEB.3.96L.1041009150440.93055O-100000@fledge.watson.org> <4168578F.7060706@geminix.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sat, 9 Oct 2004, Uwe Doering wrote:
> Robert Watson wrote:
> > On Fri, 8 Oct 2004, Steve Shorter wrote:
> >
> >>> I have some machines that run customers cgi stuff.
> >>>These machines have started to hang and become unresponsive.
> >>>At first I thought it was a hardware issue, but I discovered in
> >>>a cyclades log the following stuff that got logged to the
> >>>console which explains the cause of the system hangs/failures.
> >>>
> >>>vnode_pager_putpages: residual I/O 65536 at 347
> >>>vnode_pager_putpages: I/O error 28]
> >>>vnode_pager_putpages: residual I/O 65536 at 285]
> >>
> >> Aha! also at the same time I get in syslog
> >>
> >> /kernel: pid 6 (syncer), uid 0 on /chroot/tmp: file system full
> >>
> >> Whats happening? Can a full filesystem bring the thing down?
> >>Ideas? Fixes?
> >
> > Ideally not, but many UNIX programs respond poorly to being out of memory
> > and disk space ("No space, wot?"). Are you using a swap file, and if so,
> > how did you create the swapfile? Are you using sparse files much?
>
> I wonder whether the unresponsiveness is actually just the result of the
> kernel spending most of the time in printf(), generating warning
> messages. vnode_pager_generic_putpages() doesn't return any error in
> case of a write failure, so the caller (syncer in this case) isn't aware
> that the paging out failed, that is, it is supposed to carry on as if
> nothing happened.
>
> So how about limiting the number of warnings to one per second? UFS has
> similar code in order to curb "file system full" and the like. Please
> consider trying the attached patch, which applies cleanly to 4-STABLE.
> It won't make the actual application causing these errors any happier,
> but it may eliminate the DoS aspect of the issue.
I have just tried your patch. To test I ran the program from
http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/67919
The patch allows me to login on machine while the system reports about
"vnode_pager_putpages: I/O error 28". However, the file system access is
very limited and after some time the system became unresponsible.
Igor Sysoev
http://sysoev.ru/en/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20041103191641.K63546>