Date: Mon, 27 Mar 2006 15:20:08 -0500 From: John Baldwin <jhb@freebsd.org> To: Jason Evans <jasone@freebsd.org> Cc: Vasil Dimov <vd@freebsd.org>, Robert Watson <rwatson@freebsd.org>, freebsd-arch@freebsd.org Subject: Re: Proposed addition of malloc_size_np() Message-ID: <200603271520.11381.jhb@freebsd.org> In-Reply-To: <44281421.3060401@FreeBSD.org> References: <44247DF1.8000002@FreeBSD.org> <200603271110.02917.jhb@freebsd.org> <44281421.3060401@FreeBSD.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Monday 27 March 2006 11:34, Jason Evans wrote: > John Baldwin wrote: > > On Sunday 26 March 2006 13:04, Jason Evans wrote: > > > >>Robert Watson wrote: > >> > >>>I wonder if the intuitive objection people are raising is actually with > >>>the name. Since malloc_size() is defined on at least one platform to > >>>return the requested size, maybe a name like malloc_allocated_size() (or > >>>something a bit more compact) would help avoid that confusion, and make > >>>it clear that the consumer is getting back a commitment and not a hint > >>>for future realloc(), etc. > >> > >>Maybe you're right. We could just call it malloc_usable_size() and be > >>compatible with Linux. > > > > It would help to know why such a function would be useful. :) Do you have > > a specific use-case? If the purpose is for a program to see that it really > > as Y >= X bytes when it did malloc(X) so that the program can use Y bytes, > > that would seem to only be a source of bugs and complexity. If the program > > wants Y bytes, it should malloc(Y). Many folks in the thread seem to think > > that this function would be used for a poor-man's realloc() wrapper or > > something, and I think such uses would be short-sighted at best. If there > > are other uses such as for having a debug malloc wrap the real one, then > > that might justify the API, but it is unclear what a useful use of this API > > would be. > > I can think of a few straightforward uses: > > 1) Suppose that we are about to write to a string that we know is > malloc()ed, and we want to be sure that we aren't overflowing the > allocated buffer. We can add an assertion that the buffer is indeed > large enough to contain what is about to be written to it. In that case though you really want to know the size you allocated, not necessarily the backing-store size, esp. if you are in a library since your caller may (correctly) assume that if it allocates X bytes for a string the library will only write up to X bytes. So, I think in that case you want the requested size, not the backing store size. > 2) Suppose that we are using some sort of dynamically scalable data > structure, such as a hash table that we double in size every time > fullness crosses some threshold. In order to do that, we need to know > how large the table is. We can store that information, or we could just > query the size. (In this example, performance constraints might dictate > that we actually store the size of the table, but there are certainly > similar examples that wouldn't be so concerned with performance.) I think this devolves into the poor-mans realloc. :) Instead, the code should just realloc() when necessary and rely on the system malloc to know when it is safe for the realloc() to be done in place. > 3) This is what I want malloc_usable_size() for: garbage collection. In > order for a garbage collector to know when it should run, there needs to > be some way of tracking how much memory is in use. By using > dlsym(RTLD_NEXT, ...), I can intercept all malloc/calloc/realloc/free > calls and track current memory usage. However, unless there is a way of > getting the size of each allocation, I don't know how much to subtract > from the count for realloc/free calls. This is a good reason, and is what I hoped you had in mind when you proposed it. :) I think that other than this type of code though (malloc reimplementations or wrappers), nothing else should use this function. :) > 4) Porting code from Linux. Case in point: Xara Xtreme, currently being > ported by Vasil Dimov. At the moment, he has to use dlmalloc. Does it contain a G/C of its own or some such? > Jason > > Following is what I've written for the malloc(3) man page: > ---- > The malloc_usable_size() function returns the usable size of the > allocation pointed to by ptr. The return value may be larger than the > size that was requested during allocation. malloc_usable_size() is not > intended as a mechanism for in-place realloc(), though it can be abused > that way; rather it is primarily provided as a tool for introspection > purposes. Any discrepancy between the requested allocation size and the > size reported by malloc_usable_size() should not be depended on, since > such behavior is entirely implementation-dependent. > ---- I would word it stronger: "malloc_usable_size() should not be used as a mechanism for in-place realloc(). It is provided solely as a tool for introspection purposes." :) -- John Baldwin <jhb@FreeBSD.org> <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200603271520.11381.jhb>