Date: Thu, 26 Jun 2003 15:04:41 -0400 From: Chuck Swiger <cswiger@mac.com> To: freebsd-performance@freebsd.org Subject: Re: ten thousand small processes Message-ID: <3EFB43C9.2050106@mac.com> In-Reply-To: <20030626020455.51862.qmail@cr.yp.to> References: <20030623030004.40078.qmail@cr.yp.to> <20030624203536.D17881-100000@mail.chesapeake.net> <20030625060629.51087.qmail@cr.yp.to> <3EFA28EF.9050400@mac.com> <20030626020455.51862.qmail@cr.yp.to>
next in thread | previous in thread | raw e-mail | index | archive | help
D. J. Bernstein wrote: >> Remember that VMM hardware requires page-alignment > > When I ask why the stack and data aren't put on the same page, and you > say ``They aren't on the same page,'' you aren't answering the question. True. If A implies B, yet A is false, what have you shown about B? I didn't write the words "They aren't on the same page". What I wrote was "stack and data have different VM protections and, since VMM hardware requires page alignment, the stack and data need to be put on different pages." If you don't think the latter paraphrase is a valid answer to the question you asked, fine, say so: quoting a single phrase from the three paragraphs I wrote, and then adapting it to the point where you can complain that what I didn't say didn't answer your question has more to do with rhetorical strawmen. Thanks. > (As for adding an x bit to data: This obviously won't break anything.) Using VMM protection to forbid code execution within the DATA, BSS, heap, and stack (if one can) mitigates against a common class of problems-- "buffer overflows"-- which have lead to a vast number of security vulnerabilities. Well-written programs do not require VMM hardware to protect them from such issues, but that doesn't mean that a general-purpose OS like FreeBSD should assume that all programs are well-written. [ ... ] >> It's easy to write a memory allocator that performs a specific case well; >> writing a general purpose malloc is significantly more complicated, > > I'm not talking about replacing malloc() with a special-purpose > allocator. I'm talking about adding a tiny bit of code to malloc() to > magically take advantage of space that is being ignored right now. > > The savings in this situation go beyond those dozens of megabytes of > magically reacquired RAM. There's a nasty spike in memory usage as soon > as malloc() starts extending the heap; when a program's allocations fit > into the magically reacquired RAM, the program also avoids the spike. Calling malloc(1) for the first time causes a 16K spike in memory usage under FreeBSD 4.8; that's a factor of four larger than the minimum possible allocation of one 4K page. [ It's not clear whether all four VM pages are actually allocated, or whether one is and three are waiting for a write-fault before the system allocates resources for them. It's also worth noting that malloc() has to return memory properly aligned to whatever the local hardware requires for arbitrary data types: malloc(1), malloc(4), and even malloc(16) might all reserve the same amount of space. ] With regard to your magic suggestion, frankly, what you are doing is solving a much easier problem domain than the general purpose malloc is expected to be capable of solving. Linear memory allocation and no provision to handle internal fragmentation (because you'll never release or reuse in this buffer of memory, yes?) are wonderful decisions for a memory allocator tuned for a very small program. So long as the program doesn't repeatedly allocate and free this memory, anyway. -- -Chuck
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?3EFB43C9.2050106>