Date: Sat, 23 Jan 1999 14:05:47 -0500 (EST) From: "John S. Dyson" <dyson@iquest.net> To: dillon@apollo.backplane.com (Matthew Dillon) Cc: dyson@iquest.net, hackers@FreeBSD.ORG Subject: Re: Error in vm_fault change Message-ID: <199901231905.OAA04877@y.dyson.net> In-Reply-To: <199901230456.UAA58423@apollo.backplane.com> from Matthew Dillon at "Jan 22, 99 08:56:07 pm"
next in thread | previous in thread | raw e-mail | index | archive | help
Matthew Dillon said: > :hoggy process. However, thrashing by a hoggy process should be managed by > :limits. :-(. You are breaking any notion of the pageout daemon keeping stats > :in normal low memory conditions, but thrashing is hopeless anyway without > :limits (am I repeating myself?) > > Being realistic, *nobody* manages the RSS limits this tightly. As > much as I like RSS limits, their lack of any sort of dynamic feedback > makes them mostly unuseable in my view. They were almost entirely > unuseable at BEST on every machine from the lowliest mail server to the > most heavily loaded shell/web machine. On early machines with > significantly less memory then the machines have now, all I got for my > efforts were user complaints. > > The problem is that by the time an RSS limit kicks in, the damage > has already been done. > > If RSS limits were metrics rather then absolute values, they would > be MUCH more useful. > The existing RSS limit code did have a terrible lag, and therefore (really evil) processes could do their damage before the RSS limit helps. The RSS limit code that I provided delays the limit until memory conditions are low (and perhaps that still isn't good enough.) The new (proposed) RSS limit code does have a slight negative feedback (in that the code doesn't do anything unless memory is low.) My guess is that the RSS (hard) limit needs to be larger than the soft limit. Also, the RSS limiting needs to come into play when the cache queue is being attacked too much by the *evil* process. My demo code doesn't do that, but it is easy to add in. By adding the cache queue into the current version of my code (that I am running), it would look even more like the code previously elided in some ways. (I am going to add cache/inactive queue damage sensing to my local codebase to see if it works well.) The RSS limit needs to be a limit, and not a guideline, and shouldn't be used as such (the way the code is today.) Maybe a calculation could be added (similar to the systat -vmstat info) for a guideline. It is important to compare "peak" info with a "peak" limit, along with "average" or "trend" info with an "average" or "trend" limit. By using a "peak" limit (with the RSS limit code as is) as an "average", that is misuse. I guarantee you that an average limit would be MUCH more costly to calculate with any time resolution. (Remember, I know numerical methods and DSP type design fairly well -- there has always been an attempt to avoid too much cost in the FreeBSD VM code, and DG has always been on the side of sanity in my algorithmic complexity tendancies.) On reflection, I think that keeping the code simple has been a benefit, because it could have been even more subtile. By trying to create "guidelines", then processes using memory heavily, will see degradation. That degradation will take more in-common resources. The resource limit mechanism will be costly to system performance, when it comes into play. It ends up being the lesser of two evils. Note also, there are two kinds of "evil" processes: DOS type processes as might be run on BEST shell machines. Just big processes as might be run on workstations. With a few mods (which I really don't have time to code), I suggest that the peak RSS limits with some sensing of cache queue damage will help control the damage from both kinds of proceses. The DOS type process is problematical, and the average (or trend) limiting would allow for better control, but would also cost more. Machines today can do lots of damage to their memory profile, very quickly. The peak limit is probably a reasonable tradeoff of cost to calculate vs. benefit. -- John | Never try to teach a pig to sing, dyson@iquest.net | it makes one look stupid jdyson@nc.com | and it irritates the pig. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199901231905.OAA04877>