Date: Fri, 22 Jan 1999 12:49:50 -0500 (EST) From: "John S. Dyson" <dyson@iquest.net> To: dillon@apollo.backplane.com (Matthew Dillon) Cc: dyson@iquest.net, hackers@FreeBSD.ORG Subject: Re: Error in vm_fault change Message-ID: <199901221749.MAA03968@y.dyson.net> In-Reply-To: <199901221146.DAA52510@apollo.backplane.com> from Matthew Dillon at "Jan 22, 99 03:46:13 am"
next in thread | previous in thread | raw e-mail | index | archive | help
Matthew Dillon said: > > Basically, what happens is that a few programs create a thrashing situation > The proper action is to *not* start penalizing the rest of the machine! > Your "fix" doesn't really do that, but works around it in a very rough and subobtimal way. It is kind of like a drug that isn't very specific and has lots of side effects. A more specific fix is needed. > > My personal opinion is that memory allocation needs some sort of > scheduler - something that works somewhat like the cpu scheduler but > schedules memory rather then cpu. This way the static processes are > still allowed to run while the heavy-memory processes ( in a low memory > situation ) get blocked more. With this in place, the pager gets a > view of the overall load on the machine that is not skewed by a small > number of memory hogging processes. > Please refer to the code that I supllied to you. It isn't perfect, but with only a few mods (cleanups), it does work very well. The key to a real fix is some hysteresis (which I am running nicely -- and it decreases CPU overhead significantly.) Both memory usage is controlled, AND the undesirable and unfortunate loss of the wait on low free memory queues is now retained. > > A scheduler is not an artificial block/sleep mechanism such as the > piece of code I ripped out -- it is scheduling available free memory > to processes needing memory based on their memory-hogging priority. > Bzzzt... Never, never, never schedule (or think) in terms of "priority." It is the wrong parameter. Priority is only a convienience term for comp-sci classes that unfortunately gets carried through into practice. Set your scheduling goals "max memory", time deadline, or somesuch. Then calculate processes to run, block, or available memory for the process based upon those factors (in the best case, bin packing -- but that isn't often practical.) The mod that I submitted is based upon a max memory goal. I have been through this over and over again, on OSes that try to resolve things into priority.... Yuck!!! "Priority" ends up being a function of too many variables to be meaningful. The only way that priority is useful is for a very short term scheduling parameter -- not useful in longer term resource mgmt though. One shouldn't really think in terms of anything but demand on and requirements of resources. Attempts to give "bonus points" to "nice processes" complicates matters, and obfuscates the issue of "goals" and "requirements." Those processes that are easy on the system will naturally be treated nicely by the system, given a proper scheme. (Actually, "bonus points" will severely negatively affect the attainment of "goals" and "requirements.") It is very important to have a focused goal of requirements, and solve them. Trying to fix a problem with something that is only borne out by a simple anecdotal example is really not a good thing in the long term. This will build upon itself until the system really performs poorly. Understanding is the key. Global policy needs to optimize for performance, and local policy needs to perform resource limiting. -- John | Never try to teach a pig to sing, dyson@iquest.net | it makes one look stupid jdyson@nc.com | and it irritates the pig. To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199901221749.MAA03968>