Date: Fri, 16 Apr 1999 19:43:39 +0900 From: "Daniel C. Sobral" <dcs@newsguy.com> To: David Schwartz <davids@webmaster.com> Cc: chat@FreeBSD.ORG Subject: Re: swap-related problems Message-ID: <3717145B.6B4AB69F@newsguy.com> References: <001101be87ad$94130900$021d85d1@whenever.youwant.to>
next in thread | previous in thread | raw e-mail | index | archive | help
David Schwartz wrote: > > The idea is to make malloc return NULL (or fork) when the alternative would > be to overcommit. So it has to do with malloc returning NULL _before_ the > operating system runs into a serious overcommit situation. That's full pre-allocation. We have been telling you that's a bad idea. If you have an AIX, turn on pre-allocation system-wide to see what happens. On FreeBSD, run 'ln -s Z /etc/malloc.conf'. It is not the same thing, but will give you a general idea of what we are talking about. Not a very good idea, because it does nothing about the copy-on-write behavior, but some idea. > We are talking about forcing malloc to return NULL. Not about what to do > when/if malloc returns NULL. Precisely. I was never talking about what the applications do. > The problem is, you can't solve a problem in X by fixing the entire rest of > the world. Not only that, but there is no sane algorithm for doing so. > Suppose you have 64Mb of real memory, 512Mb of swap, and 2,500 users. You > tell me what you limit them to. If you have that, you will *never* be able to run a pre-allocating system. What I have shown to you is that the problem of protecting a subset of processes from getting killed is equivalent to the problem of setting a limit to the datasize. As you said, there is no sane algorithm for doing so. > > As an aside, if P does not expand to fill all memory, but you are > > still facing a situation where processes get killed because of > > overcommit, it is simply that you don't have enough memory to run > > what you are trying to run. > > No, it's simply that the operating system allowed a dangerous situation it > could have avoided. It could *not* have been avoided by protecting a subset of programs. It could have been avoided by pre-allocating all memory, in which case you wouldn't have enough memory to run anywhere near as much as you was running in that scenario. > > Err, excuse me? Let's use a "numeric example", as we used to ask our > > algebra professor (only to have it say "Given three numbers, a, b > > and c, ..." :). > > > > We have 64 Mb of memory. We start running processes. At some point, > > they will try to allocate more than 64 Mb of memory. > > Umm, are you assuming no swap? Without swap this whole thing doesn't work. > Let's assume 512Mb of swap. Okay, so we try to allocate more than 64Mb of > memory. I'm assuming 64 Mb total. RAM, ROM, swap, EEPROM, dma buffers, whatever. All of it. Everything. The absolute value is irrelevant. Assume 640 Kb, assume 4 Tb, it doesn't matter. Since you missed this fundamental point, I'm requoting the whole argument. Please, read it. *** beginning > > If at this point we make the malloc calls fail, we are > > pre-allocating everything. This doesn't work because your > > applications get to use a lot less memory than they would otherwise. > > A few very specialized systems have use for that. Unix is not among > > these systems, so pre-allocating systems are not relevant to this > > discussion. > > Umm, we don't need to make mallocs fail. We can always swap dirty pages. So > we're fine. No, we can't swap, because the swap is part of this 64 Mb of memory. > > Thus, we let the applications allocate more than 64 Mb of memory > > before making mallocs fail. This is overcommitting. At this point it > > is theoretically possible that all memory will get touched, causing > > a process to die. In practice, that is unlikely. > > > > Ok, so let the applications grow some more. At some point the system > > will get dangerously close to actually facing a memory starvation > > situation. Before that happens, the system will start failing > > mallocs, so the situation does not worsen. Of course, the system > > must be smart enough to let enough free memory so that normal > > dirtying of pages already allocated won't cause the memory > > starvation, even without any new mallocs. > > > It is simple to decide what point is that. When you finish this as > > your Ph.D. thesis, please send us a reference to your algorithm. *** end > > If the operating system returns NULL, it is either using the > > algorithm you'll describe in your Ph.D. thesis, or the memory is > > exhausted. If the later, some other process, which is not using > > pre-allocated memory, might dirty a page causing something to get > > killed. > > Right, some other process might dirty a page, but that shouldn't cause my > well-behaved process to get killed. My process would prefer to have its > malloc fail. In fact, if my malloc fails, I can even un-mmap some > non-critical stuff (shrinking caches), saving the operating system from > having to kill _anything_. What should get killed then? Whatever it is, it is what I called N, a non-essential process. Go read that explanation again to see why this doesn't work. > When it hasn't enough swap to back all 'vital' process' copy-on-write or > fill-on-demand pages. The definition of vital is up to the implementation. > 'owned by root' would be one definition. 'That successfully executed a > certain system call' would be another. And what is left is N. Go back and read that explanation again. > > Much on the contrary. I'm assuming the kernel cannot arbitrarily > > choose something other than memory full condition to fail on a > > malloc(), because it doesn't know anything about what the process > > needs or not. > > That's an absolutely retarded assumption, and certainly not the assumption > that many UNIXes use. Sure. FreeBSD, for instance, doesn't assume that at all. That's what we call overcommitting. > > Well... ok, what would you have the kernel do? Please describe the > > entire scenario, not just one process. How much real memory exists? > > How much got allocated? When did the mallocs start to fail? How was > > each process memory allocated (pre-allocated or on-demand). > > For the 400th time, the mallocs start to fail in 'critical' processes as > soon as the kernel doesn't have enough physical memory plus swap to back all > their allocations. > > The tradeoff is as follows: the process agrees not to take as much memory > as the operating system might be willing to allocate to it and in exchange, > the operating system agrees not to kill off the process due to memory > conditions beyond that process' control. > > This gives the operating system the benefit of failing mallocs early and > perhaps preventing it from having to kill off processes at all. This gives > the vital process the benefit of not being killed off due to > overcommitment -- the idea is to accept fewer resources in exchange for > those resources being guaranteed. > > If the process is smart enough to do something sensible when malloc fails, > such as return non-critical memory to the operating system, everybody wins. > If it's not, nobody loses, the process simply gets less memory than it would > otherwise want. Odds are, if it's well behaved, it won't be the one the > kernel would have killed off anyway. > > Are you deliberately being dense? Or do you have some weird hidden agenda? No. Not at all. This you propose is, basically, pre-allocating everything. The idea of non-critical processes is flawed. Go see previous explanations on why. In fact, the system you describe will tend to run only critical processes, as non-critical processes will tend to get killed, because they are overcommitting and the critical processes never get killed. Realize that the memory can become full without any malloc being called, just by having non-critical processes use their overcommitted memory. When these non-critical processes get killed, their memory gets used by both critical and non-critical processes, reducing further the space available for non-critical processes. From time to time, critical processes will have their mallocs fail, will clamp down on memory consumption, and maybe free some memory. The freed memory will be taken by critical processes which did not have their mallocs fail, and non-critical processes. Eventually, the process which clamped down on memory usage will try a malloc again, it will succeed (eventually it will succeed, unless you get to a stable full memory state), and it will get it's business going again until another malloc fails. So, your system will eventually have almost only critical processes, unless you have enough memory to satisfy the need of all critical processes, and still have enough left for non-critical processes. Notice that, in this case, if you were not pre-allocating then no process would ever get killed, because there would be enough memory for all. (Not true in all cases, but in most enough cases.) If you only have critical processes (plus a few non-critical ones), that's equivalent of pre-allocating all memory. A number of people have told you this wastes too much resources. If you think otherwise, go do the experiments I mentioned at the beginning of this message. -- Daniel C. Sobral (8-DCS) dcs@newsguy.com dcs@freebsd.org "Well, Windows works, using a loose definition of 'works'..." To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-chat" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3717145B.6B4AB69F>