From owner-freebsd-arch@FreeBSD.ORG Sat Mar 25 17:40:52 2006 Return-Path: X-Original-To: freebsd-arch@FreeBSD.org Delivered-To: freebsd-arch@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EEEEF16A426; Sat, 25 Mar 2006 17:40:51 +0000 (UTC) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (apollo.backplane.com [216.240.41.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6850343D5C; Sat, 25 Mar 2006 17:40:48 +0000 (GMT) (envelope-from dillon@apollo.backplane.com) Received: from apollo.backplane.com (localhost [127.0.0.1]) by apollo.backplane.com (8.13.4.20060308/8.13.4) with ESMTP id k2PHeloE014341; Sat, 25 Mar 2006 09:40:47 -0800 (PST) Received: (from dillon@localhost) by apollo.backplane.com (8.13.4.20060308/8.13.4/Submit) id k2PHelT4014340; Sat, 25 Mar 2006 09:40:47 -0800 (PST) Date: Sat, 25 Mar 2006 09:40:47 -0800 (PST) From: Matthew Dillon Message-Id: <200603251740.k2PHelT4014340@apollo.backplane.com> To: Jason Evans References: <44247DF1.8000002@FreeBSD.org> <200603250806.k2P86YJU011861@apollo.backplane.com> <4424FDE9.3080707@FreeBSD.org> Cc: freebsd-arch@FreeBSD.org Subject: Re: Proposed addition of malloc_size_np() X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 25 Mar 2006 17:40:52 -0000 :... :have to store the precise allocation request size; it can instead round :the request size up internally, then treat the allocation as being of :this rounded up size. By vaguely specifying the return value of :malloc_size_np(), we grant the malloc implementation freedom as to :whether the size is precisely what was specified during allocation, or :some rounded up value. : :I had no intention of suggesting that malloc_size_np() should extend :existing allocations, nor change the return value depending on the :current state of memory following the allocation pointed to by ptr. I'm not sure what benefit this would have, though. When a program calls malloc it generally does so with a good idea as to the number of bytes it wants to allocate. malloc implementations have always been free to return more bytes then were requested. But why would any program want to complicate its life and use an API where the potential additional space (which is extremely algorithm-dependant) becomes visible to the program? I'm trying to imagine the number of mistakes programmers would make, with the resulting bugs winding up in the program(s), that an integrated implementation would cause. An API like malloc_size_np() would have a very limited usefullness, not to mention being inflexible. Programs are far more likely to want to resize an allocation then to use whatever extra slop space the malloc implementation might happened to have included in the original allocation. A reallocation API like realloc_try() makes a lot more sense. Not only does it cover all aspects of malloc_size_np(), it also gives the allocator implementation the flexibility to actually truncate or extend an allocation if that happens to be supported by the allocator, or not if it doesn't. :This actually does make some assumptions about the way memory is managed :by malloc. It would not be useful in the context of phkmalloc or :jemalloc to truncate an allocation via realloc_try(), because each page :of memory contains allocations of only one size class. Thus, there is :no way to re-use the trailing space (exception: multi-page allocations). : This would potentially be useful for dlmalloc (or ptmalloc) though. : :Thanks, :Jason No, it actually doesn't. Or I should say, uf a program is going to actually try to USE the potential extra space for something, something like realloc_try() makes no more assumptions about the malloc implementation than malloc_size_np(). The implementation is totally free to limit the functionality of realloc_try() to the same slop space (if any) that malloc_size_np() returns. The difference is that realloc_try() is far closer to what a programmer would likely need. Even for things like slab allocators which have no ability to extend or truncate small allocations beyond the chunk size of the slab they reside in, many malloc implementations still use a different mechanism for larger multi-page allocations and in those cases the implementation might actually be able to do something with the original allocation. The only real situation that I can think of where a program might want to have the option of (optimally) using more or less space then was originally requested is the situation where a large chunk of memory, such as 32KB, needs to be extended or truncated. For example, a preprocessor or parser (generating post-parsed output) or linker (relocation arrays), or any other program of that nature could benefit. I'm still wondering who in their right mind would actually malloc X bytes and then use an API call to obtain and actually use the (dynamic, ephermal, and completely non-deterministic) Y bytes that the underlying implementation reserved, verses simply allocating Y bytes in the first place. It sounds like a recipe for disaster to me. At least with a reallocation API such as realloc_try() the caller can specify a requirement that the malloc implementation is either able to meet or not able to meet. -Matt Matthew Dillon