Date: Wed, 29 Apr 1998 02:08:52 +0000 (GMT) From: Terry Lambert <tlambert@primenet.com> To: adrian@virginia.edu Cc: dec@phoenix.its.rpi.edu, freebsd-hackers@FreeBSD.ORG Subject: Re: SIGDANGER Message-ID: <199804290208.TAA07145@usr01.primenet.com> In-Reply-To: <Pine.SOL.3.96.980427231203.3622N-100000@mamba.cs.Virginia.EDU> from "Adrian T. Filipi-Martin" at Apr 27, 98 11:32:55 pm
next in thread | previous in thread | raw e-mail | index | archive | help
> When I ran this basic example on an AIX box years ago, a buch of > the sleepers get killed before the danger process does. Why kill them? > They have only minimal resources allocated. Because they needed pages which weren't available when the pages were needed. > I believe it also breaks the POSIX definition of malloc(3), if I > am not mistaken. Perhaps IBM had this ammended after introducing this > misfeature. Any malloc(3) that doesn't actually preallocate the memory to the process, instead of just creating empty page mappings that are backed by pages as needed, and any malloc(3) whose free(3) sbrk(2)'s memory back to the system are technically in violation of POSIX. Specifically, POSIX requires that if I free a segment, I should be able to malloc the same sized or smaller segment immediately following the free, and not have to check for failure. This is a bit of stupidity in POSIX. My main rant, which Jordan alluded to, is that I think it should be possible to preload all necessary pages into swap, and back them with swap rather than with the program image. The primary reason for doing this is so that your dataless machine or Network Computer does not freeze waiting for a pagein if the NFS server becomes unavailable for whatever reason. Having worked in an environment where most of engineering was running on dataless machines with local swap and local auxillary disk, but running most applciations from a central server (much easier to maintain 40+ engineers this way), and having had 40+ people become unproductive while the NFS server reboots... well, it became a design issue for me. Going a bit further down this road, one could imagine that certain processes should be preallocated real swap for clean data pages which may become dirty. This is a precopy rather than a copy-on-write policy for them. This would, I think, be the correct use of the "sticky" bit, in a memory overcommit environment. The "swap preload" case in the NFS environment should probably be a mount option which only applies to FS's that are flagged as remote. The real answer for the general case is that for allocations via sbrk(), you can't know ahead of time how much memory a process will use, unless it makes all of its memory demands up front. The "SIGDANGER", I think, would be sent to all processes in a low "memory + swap" condition, indicating to them that they are in danger of being killed if they ask for pages or dirty copy-on-write pages, or otherwise increase page use. Most "important" (system) processes would ignore the signal, or at most use it to try and free cached allcoations that they didn't really have in active use at the moment. I really would prefer some mechanism for targetting the offending process; ie: on a page demand that can't be satisfied, shoot the guy with the highest working set instead of the poor schmuck who has three dirty data pages and wants a fourth. In other words, delay the page demand satisfy in the hopes that you will be able to later satisfy the request instead of just shooting the guy. If the guy with the highest working set has a SIGDANGER (I prefer naming like SIGLOWMEM or something more descriptive of what the signal actually indicates) handler registered, and pages do not become available, then shoot the next highest working set guy, and so on. A probable optimization would be: 1) Shoot everyopne with a "low memory condition" handler 2) wait 3) if memory is still low, shoot the highest working set without a handler and go to 2 4) Satisfy outstanding unsatisfied requests. Terry Lambert terry@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199804290208.TAA07145>