Date: Tue, 25 Mar 2003 09:06:43 -0800 From: Eivind Eklund <eivind@FreeBSD.org> To: David Schultz <das@FreeBSD.ORG> Cc: Garance A Drosihn <drosih@rpi.edu>, Dan Nelson <dnelson@allantgroup.com>, Poul-Henning Kamp <phk@phk.freebsd.dk>, Wes Peters <wes@softweyr.com>, freebsd-arch@FreeBSD.ORG Subject: Re: Patch to protect process from pageout killing Message-ID: <20030325090643.F20745@FreeBSD.org> In-Reply-To: <20030325075342.GA5450@HAL9000.homeunix.com>; from das@FreeBSD.ORG on Mon, Mar 24, 2003 at 11:53:42PM -0800 References: <200303240823.48262.wes@softweyr.com> <7019.1048523782@critter.freebsd.dk> <20030324213519.GA63147@dan.emsphone.com> <20030325012844.GB4406@HAL9000.homeunix.com> <p05200f42baa5897e8dd8@[128.113.24.47]> <20030325075342.GA5450@HAL9000.homeunix.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Mar 24, 2003 at 11:53:42PM -0800, David Schultz wrote:
> Yes, I think the SIGDANGER idea makes sense. Essentially what
> you'd want is a higher threshhold above the ``red alert---start
> killing things'' threshhold where you can do smart things like
> send SIGDANGERs without worrying about running completely out of
> memory. But I'm trying to impress on people that SIGDANGER is
> orthogonal to what Wes is trying to do, before the whole thing
> gets bogged down in discussions again and nothing ever happens.
> Here's an example of what I mean in verbose pseudocode with
> fudged constants:
>
> if (free VM < 64 pages) {
> /* This is the part Wes is working on. */
> kill big processes EXCEPT the ones that
> are so important that there's no point
> in running the system without them;
> } else if (free VM < 256 pages) {
> /*
> * It takes additional memory to do this, but we're
> * hoping some processes will cooperate and the
> * shortage will go away.
> */
> start warning processes with SIGDANGER;
> }
As far as I understand, this problem was covered by the SIGDANGER proposal,
by having processes with a SIGDANGER handler not be killed in the
(free VM < 64 pages) case, at least until there are no processes without a
SIGDANGER handler.
The pseudo-code becomes something like (with 64, 128 and 256 being arbitrary
constants)
if (free VM < 256 pages) {
send SIGDANGER to all processes
}
while (free VM < 128 pages &&
we have processes without SIGDANGER handler) {
kill "worst" process without SIGDANGER handler
}
while (free VM < 64 pages) {
/*
* Only goes here if we are out of processes without
* SIGDANGER handler
*/
kill "worst" process
}
As you see, SIGDANGER says that the process wants to decide for itself if
it is important (kept until 64) or want to die/free up resources at 256.
I'm not 100% happy with the SIGDANGER API, for the following reasons:
(1) There are cases it does not cover. I can implement a process that
is not really significant, but does caching and can easily free up
memory. However, even though it frees up memory, it should not get
special priority in the killing sequence.
The most extreme example of this is if we add SIGDANGER awareness
to phkmalloc - in that case, all newly compiled programs (that use
libc and malloc) would gain priority, while all old binaries would
be prioritized lower.
(2) The use an API instead of an external configuration option (e.g. a sysctl
with a list of protected PIDs) makes it impossible to use this without
having recompiling software. It also means that priority is determined
at the time of software implementation, not when the software is
deployed, unless there are special options in the software to change
behaviour. And these options are likely to appear, which basically
sucks.
However, I still feel that we *should* support the SIGDANGER API. We need
an API to cover the case where a program has resources that it can easily
release, and the API should be cross-platform. By supporting the SIGDANGER
API on FreeBSD too, that API becomes aboutr 4x as "legitimate" as it is
today. If we implement another API, both SIGDANGER and that API will be
seen as less legitimate than SIGDANGER is today, unless that API is *much*
better than SIGDANGER. Thus, we lower the chance of ever getting a true
cross-platform solution.
I feel there also is room for a separate solution that lets the administrator
determine processes to keep - but this should not block for implementation
of SIGDANGER with the standard semantics, and that is IMO what would be
most important to have.
Also note that a SIGDANGER implementation might automatically be picked up
by autoconf for already existing programs, giving an immediate benefit.
Eivind, who realize he has no vote until he has patches.
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20030325090643.F20745>