Date: Thu, 24 Jul 2003 20:16:35 -0400 From: Chuck Swiger <cswiger@mac.com> To: freebsd-stable@freebsd.org, Peter Jeremy <peter.jeremy@alcatel.com.au> Subject: Re: malloc does not return null when out of memory Message-ID: <3F2076E3.5080607@mac.com> In-Reply-To: <20030724215020.GV430@gsmx07.alcatel.com.au> References: <20030723173427.GA72876@vmunix.com> <20030723140329.C92624@carver.gumbysoft.com> <20030723221336.GA26555@pit.databus.com> <20030723223654.GA24008@moghedien.mukappabeta.net> <20030723224436.GD22166@Odin.AC.HMC.Edu> <20030724021726.GJ430@gsmx07.alcatel.com.au> <3F1F45E2.5080506@mukappabeta.de> <20030724215020.GV430@gsmx07.alcatel.com.au>
next in thread | previous in thread | raw e-mail | index | archive | help
Peter Jeremy wrote: > On 2003-Jul-24 04:35:14 +0200, Matthias Buelow <mkb@mukappabeta.de> wrote: > Cleanly recovering in all cases when there is no additional memory > available is a very hard problem. Most of the suggested fixes > generally rely on the kernel providing a warning when the VM system > is under stress but can still satisfy some requests. Unfortunately, > I don't think any of them included the necessary patches. [ ... ] I agree with this statement: handling system-wide resource shortages is a tricky, failure-prone, and dangerous environment to try and program for. I'm very familar with the Mach project, Mach memory management, and task/thread semantics; am I okay using that terminology, or am I going to confuse myself (or other people :-)? For instance, I know that FreeBSD's VM system includes some degree of copy-on-write semantics, but I don't believe there is support for user-mode pagers, right? Mach uses message-passing, heavily, and supported both kernel-mode and user-mode paging algorithms; it would let you set VM_NORMAL, VM_SEQUENTIAL, VM_ANOLMALOUS, or your own paging algorithm. There is a difference between "I want another page of VM" and "my process wants another frame of physical memory", but for those tasks using VM backing store which they can flush and make available to the rest of the system, being able to free up physical RAM can help make memory available for error-handling or whatever in the faulting task. In other words, Mach messaging served as a mechanism for the kernel to let tasks (processes) know that the VM system was under stress. -- Mach relies heavily on COW semantics and lazy allocation of resources, and I fully understand the benefits of VM overcommit in general, but Mach let you control allocation of VM very explicitly if you needed to. Mach also did some clever optimizations with regard to zero-filling pages: mapped but unallocated regions of memory read as zero, and the Mach exception handler which trapped writing to mapped but unreferenced memory, or memory shared copy-on-write, and would only allocate more VM if the write actually changed something. Writing zeros to an unallocated page did not cause the page to be allocated. Which is why my patch used SOME_JUNK, of course. :-) [ This is in answer to another message by Jeremy, which said: ] > This comes up every time this thread starts. As I said before - read > the archives. If you think you have a solution that works and avoids > at least the larger pitfalls (see the archives), you need to provide > patches (or show a willingness to pay someone else to write the code). Let me criticise the proposed patch I made: 1) To force pages to be allocated, you only need to write one byte per page: mine writes at least one byte per malloc(). malloc_pages() could use the existing stride algorithm, and malloc_bytes() should do something else appropriate. 2) I use malloc_pagesize rather than, what, getpagesize(), or god only knows what else is appropriate. 3) What to do with regard to error-handling, as discussed above and elsewhere. -- -Chuck
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3F2076E3.5080607>