Date: Wed, 25 Oct 2000 09:42:56 -0700 (PDT) From: Matt Dillon <dillon@earth.backplane.com> To: Terry Lambert <tlambert@primenet.com> Cc: bright@wintelcom.net (Alfred Perlstein), ps@FreeBSD.ORG, hackers@FreeBSD.ORG Subject: Re: vm_pageout_scan badness Message-ID: <200010251642.e9PGguj26737@earth.backplane.com> References: <200010251012.DAA19288@usr02.primenet.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:
:Consider that a file with a huge number of pages outstanding
:should probably be stealing pages from its own LRU list, and
:not the system, to satisfy new requests. This is particularly
:true of files that are demanding resources on a resource-bound
:system.
:...
: Terry Lambert
: terry@lambert.org
This isn't exactly what I was talking about. The issue in regards to
the filesystem syncer is that it fsync()'s an entire file. If
you have a big file (e.g. a USENET news history file) the
filesystem syncer can come along and exclusively lock it for
*seconds* while it is fsync()ing it, stalling all activity on
the file every 30 seconds.
The current VM system already does a good job in allowing files
to stealing pages from themselves. The sequential I/O detection
heuristic depresses the priority of pages as they are read making
it more likely for them to be reused. Since sequential I/O tends
to be the biggest abuser of file cache, the current FreeBSD
algorithms work well in real-life situations. We also have a few
other optimizations to reuse pages in there that I had added a year
or so ago (or fixed up, in the case of the sequential detection
heuristic).
One of the reasons why Yahoo uses MAP_NOSYNC so much (causing the problem
that Alfred has been talking about) is because the filesystem
syncer is 'broken' in regards to generating unnecessarily long stalls.
Personally speaking, I would much rather use MAP_NOSYNC anyway, even with
a fixed filesystem syncer. MAP_NOSYNC pages are not restricted by
the size of the filesystem buffer cache, so you can have a whole
lot more dirty pages in the system then you would normally be able to
have. This 'feature' has had the unfortunate side effect of screwing
up the pageout daemon's algorithms, but that's fixable.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200010251642.e9PGguj26737>