From owner-freebsd-arch@FreeBSD.ORG Mon Mar 27 20:19:32 2006 Return-Path: X-Original-To: freebsd-arch@freebsd.org Delivered-To: freebsd-arch@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A36616A401; Mon, 27 Mar 2006 20:19:32 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7DF5643D48; Mon, 27 Mar 2006 20:19:31 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from localhost (john@localhost [127.0.0.1]) by server.baldwin.cx (8.13.4/8.13.4) with ESMTP id k2RKJUUm063473; Mon, 27 Mar 2006 15:19:30 -0500 (EST) (envelope-from jhb@freebsd.org) From: John Baldwin To: Jason Evans Date: Mon, 27 Mar 2006 15:20:08 -0500 User-Agent: KMail/1.9.1 References: <44247DF1.8000002@FreeBSD.org> <200603271110.02917.jhb@freebsd.org> <44281421.3060401@FreeBSD.org> In-Reply-To: <44281421.3060401@FreeBSD.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200603271520.11381.jhb@freebsd.org> X-Virus-Scanned: ClamAV 0.87.1/1358/Mon Mar 27 11:12:27 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-3.8 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on server.baldwin.cx Cc: Vasil Dimov , Robert Watson , freebsd-arch@freebsd.org Subject: Re: Proposed addition of malloc_size_np() X-BeenThere: freebsd-arch@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Discussion related to FreeBSD architecture List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 27 Mar 2006 20:19:32 -0000 On Monday 27 March 2006 11:34, Jason Evans wrote: > John Baldwin wrote: > > On Sunday 26 March 2006 13:04, Jason Evans wrote: > > > >>Robert Watson wrote: > >> > >>>I wonder if the intuitive objection people are raising is actually with > >>>the name. Since malloc_size() is defined on at least one platform to > >>>return the requested size, maybe a name like malloc_allocated_size() (or > >>>something a bit more compact) would help avoid that confusion, and make > >>>it clear that the consumer is getting back a commitment and not a hint > >>>for future realloc(), etc. > >> > >>Maybe you're right. We could just call it malloc_usable_size() and be > >>compatible with Linux. > > > > It would help to know why such a function would be useful. :) Do you have > > a specific use-case? If the purpose is for a program to see that it really > > as Y >= X bytes when it did malloc(X) so that the program can use Y bytes, > > that would seem to only be a source of bugs and complexity. If the program > > wants Y bytes, it should malloc(Y). Many folks in the thread seem to think > > that this function would be used for a poor-man's realloc() wrapper or > > something, and I think such uses would be short-sighted at best. If there > > are other uses such as for having a debug malloc wrap the real one, then > > that might justify the API, but it is unclear what a useful use of this API > > would be. > > I can think of a few straightforward uses: > > 1) Suppose that we are about to write to a string that we know is > malloc()ed, and we want to be sure that we aren't overflowing the > allocated buffer. We can add an assertion that the buffer is indeed > large enough to contain what is about to be written to it. In that case though you really want to know the size you allocated, not necessarily the backing-store size, esp. if you are in a library since your caller may (correctly) assume that if it allocates X bytes for a string the library will only write up to X bytes. So, I think in that case you want the requested size, not the backing store size. > 2) Suppose that we are using some sort of dynamically scalable data > structure, such as a hash table that we double in size every time > fullness crosses some threshold. In order to do that, we need to know > how large the table is. We can store that information, or we could just > query the size. (In this example, performance constraints might dictate > that we actually store the size of the table, but there are certainly > similar examples that wouldn't be so concerned with performance.) I think this devolves into the poor-mans realloc. :) Instead, the code should just realloc() when necessary and rely on the system malloc to know when it is safe for the realloc() to be done in place. > 3) This is what I want malloc_usable_size() for: garbage collection. In > order for a garbage collector to know when it should run, there needs to > be some way of tracking how much memory is in use. By using > dlsym(RTLD_NEXT, ...), I can intercept all malloc/calloc/realloc/free > calls and track current memory usage. However, unless there is a way of > getting the size of each allocation, I don't know how much to subtract > from the count for realloc/free calls. This is a good reason, and is what I hoped you had in mind when you proposed it. :) I think that other than this type of code though (malloc reimplementations or wrappers), nothing else should use this function. :) > 4) Porting code from Linux. Case in point: Xara Xtreme, currently being > ported by Vasil Dimov. At the moment, he has to use dlmalloc. Does it contain a G/C of its own or some such? > Jason > > Following is what I've written for the malloc(3) man page: > ---- > The malloc_usable_size() function returns the usable size of the > allocation pointed to by ptr. The return value may be larger than the > size that was requested during allocation. malloc_usable_size() is not > intended as a mechanism for in-place realloc(), though it can be abused > that way; rather it is primarily provided as a tool for introspection > purposes. Any discrepancy between the requested allocation size and the > size reported by malloc_usable_size() should not be depended on, since > such behavior is entirely implementation-dependent. > ---- I would word it stronger: "malloc_usable_size() should not be used as a mechanism for in-place realloc(). It is provided solely as a tool for introspection purposes." :) -- John Baldwin <>< http://www.FreeBSD.org/~jhb/ "Power Users Use the Power to Serve" = http://www.FreeBSD.org