Date: Mon, 13 Sep 1999 01:18:26 -0700 From: "Ronald F. Guilmette" <rfg@monkeys.com> To: Alfred Perlstein <bright@wintelcom.net> Cc: questions@FreeBSD.ORG Subject: Re: Need persistant heap code. Message-ID: <29573.937210706@monkeys.com> In-Reply-To: Your message of Mon, 13 Sep 1999 01:33:43 -0000. <Pine.BSF.4.05.9909130129230.6392-100000@fw.wintelcom.net>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <Pine.BSF.4.05.9909130129230.6392-100000@fw.wintelcom.net>, you wrot e: > >On Sun, 12 Sep 1999, Ronald F. Guilmette wrote: > >> >> I need to find (or, if necessary, build) some package that will provide >> essentially the same functionality as malloc and free, except for the >> fact that data placed into the allocated regions should be persistant, >> i.e. stored in a disk file, even after the program using these primitives >> terminates. >> >> Ideally, the code should do allocation very efficiently as, for example, >> the GNU malloc package does. >> >> It is also essential that the code be able to expand the disk file, as >> necessary, to accomodate new allocation requests. But the total amount >> of space that will be allocated in this persistant heap at any given time >> will never exceed 2GB. >> >> And of course, I'd like this as C source code and I need it to run on >> FreeBSD. >> >> Anybody ever heard of such a thing? If so, where might I be able to >> get a copy? >> >> Please understand, I don't need any extraordinarily fancy persistance >> package... just persistant versions of malloc and free, with maybe >> calloc thrown in, just for good measure. >> >> Any help would be appreciated. > >The problem is that you didn't define a "restart" proceedure, >ie, how is one to identify or get a handle to what is actually >stored in this on disk heap? Humm... Good point. I guess that I would be happy to just treat this "persistant heap" thing as sort of an independent address space. Whatever routine filled the role of "malloc" for this persistant heap would just return a 32-bit offset, which would be taken as being relative to the start of the heap file. >It would be an interesting project to setup such an allocation >mechanism, I imagine one would need a special pointer type to >distinguish between real memory and on disk memory. I don't think so. I mean yea sure, if we were talking C++ code, then yes, you would want to create a new pointer type... a "persistant heap pointer". But I have gotten rather un-enamored of C++ and I much prefer the simplicity of C these days. In good old C, you could just use void* and/or char* types for these new persistant heap pointers. Then you would just have to remember to always add the base address of the place where the persistant file got mmapped into the current process... assuming that mmap() is used to implement this whole scheme somehow... before you ever try to dereference one of these things. If you screw up and forget to do that add, then yea, all hell breaks loose and you are hosed. But if you're careful, then you don't need a new type. >Perhaps if you put up a page explaining the API and restart proceedure >someone would take an interest in it, then again maybe something is >already out there that I am unaware of. Yes, I should probably do that, but I'm somewhat flexible about the API at this point, and I'm just hoping that _something_ will turn up like what I want... something that is already built. (I don't want to *preclude* any possible solutions just yet.) I suppose that I could just grab GNU malloc and then munge it awhile so as to come up with a version that would just try to do allocations out of some mmapped region (associated with some disk file) but here is the problem: How do you EXPAND an mmapped region? I didn't see any such capability on the mmap() man page and thus, I assume that there is no such capability. (Should I put in an enhancement request to the kernel developers?) The only way I see to expand an mmapped region is to munmap it, extend the file (using seek/write) and then re-mmap the file again. Seems rather messy, but I guess it would work. So I could do that, and I could just hack on GNU malloc until I came up with a version that knew how to do allocations against (and also how to expand) and mmapped file. I _could_ do all of this, but I'm lazy, and I was just vaguely hoping that someone else had already done it for me. (1/2 :-) To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-questions" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?29573.937210706>