From owner-freebsd-hackers  Wed Oct 25  3:13: 5 2000
Delivered-To: freebsd-hackers@freebsd.org
Received: from smtp04.primenet.com (smtp04.primenet.com [206.165.6.134])
	by hub.freebsd.org (Postfix) with ESMTP
	id D9F1837B479; Wed, 25 Oct 2000 03:13:02 -0700 (PDT)
Received: (from daemon@localhost)
	by smtp04.primenet.com (8.9.3/8.9.3) id DAA16635;
	Wed, 25 Oct 2000 03:09:54 -0700 (MST)
Received: from usr02.primenet.com(206.165.6.202)
 via SMTP by smtp04.primenet.com, id smtpdAAANUaqxG; Wed Oct 25 03:09:47 2000
Received: (from tlambert@localhost)
	by usr02.primenet.com (8.8.5/8.8.5) id DAA19288;
	Wed, 25 Oct 2000 03:12:53 -0700 (MST)
From: Terry Lambert <tlambert@primenet.com>
Message-Id: <200010251012.DAA19288@usr02.primenet.com>
Subject: Re: vm_pageout_scan badness
To: bright@wintelcom.net (Alfred Perlstein)
Date: Wed, 25 Oct 2000 10:12:53 +0000 (GMT)
Cc: dillon@earth.backplane.com (Matt Dillon), ps@FreeBSD.ORG,
	hackers@FreeBSD.ORG
In-Reply-To: <20001024151414.P28123@fw.wintelcom.net> from "Alfred Perlstein" at Oct 24, 2000 03:14:14 PM
X-Mailer: ELM [version 2.5 PL2]
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Sender: owner-freebsd-hackers@FreeBSD.ORG
Precedence: bulk
X-Loop: FreeBSD.ORG

> > I'll take a crack at implementing the openbsd (or was it netbsd?) partial
> > fsync() code as well, to prevent the update daemon from locking up large
> > files that have lots of dirty pages for long periods of time.
> 
> Making the partial fsync would help some people but probably not
> these folks.

I think this would be better handled as a per file working set
quota, which could not be exceeded, unless changed by root.

Consider that a file with a huge number of pages outstanding
should probably be stealing pages from its own LRU list, and
not the system, to satisfy new requests.  This is particularly
true of files that are demanding resources on a resource-bound
system.


> The people getting hit by this are Yahoo! boxes, they have giant areas
> of NOSYNC mmap'd data, the issue is that for them the first scan through
> the loop always sees dirty data that needs to be written out.  I think
> they also need a _lot_ more than 32 pages cleaned per pass because all
> of thier pages need laundering.

First principles?

What are they doing, such that this situation arises in the
first place?  Having a clue to the problem they are trying to
resolve, which causes this problem as a side effect, would
both help to clarify if there were a better soloution for them,
as well as what FreeBSD should potentially act like they were
asking for instead, when/if the situation arose.


> It might be wise to switch to a 'launder mode' if this sort of
> usage pattern is detected and figure some better figure to use than
> 32, I was hoping you'd have some suggestions for a heuristic to
> detect this along the lines of what you have implemented in bufdaemon.

This is kind of evil.  You could do low and high watermarking,
as you suggest, but without any idea of the queue retention
time to expect, and how bursty the situation is, there's no
way to pick an appropriate algorithm.


					Terry Lambert
					terry@lambert.org
---
Any opinions in this posting are my own and not those of my present
or previous employers.


To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message