Date: Tue, 27 Aug 1996 20:02:57 +0200 From: Poul-Henning Kamp <phk@critter.tfs.com> To: Terry Lambert <terry@lambert.org> Cc: stesin@gu.kiev.ua, angio@aros.net, squid-users@nlanr.net, current@FreeBSD.org Subject: Re: FreeBSD malloc.c, -lmalloc, and squid. Message-ID: <1056.841168977@critter.tfs.com> In-Reply-To: Your message of "Tue, 27 Aug 1996 09:53:42 PDT." <199608271653.JAA24972@phaeton.artisoft.com>
next in thread | previous in thread | raw e-mail | index | archive | help
In message <199608271653.JAA24972@phaeton.artisoft.com>, Terry Lambert writes: >> As a result of a pre-2.1.5-release discussion, >> I was under impression that libmalloc in FreeBSD-2.1.5 was going >> to be a "libPHKmalloc" really, with old libmalloc going elsewhere. >> From your message I got an impression that I was wrong. :( libmalloc is gone. >> As for me, I strongly suggest using phkmalloc with Squid >> on any platform. Or gnumalloc at least. Default (standard?) >> BSD malloc isn't usable. > >More on this topic... > >I would like to see the use of an mmap() of /dev/zero replace the >use of sbrk() in malloc. For one thing, it would allow us to >seperate allocation by thread by establishing sparse allocation >zones in the overall process address space. Been there, done that. sbrk() is about 5..10 times faster, probably because it knows much more at the onset than mmap does. And by the way, you can munmap pages you got with sbrk as far as I know :-) >Secondly, the sbrk()-based system recovery of memory assumes too >much allocation locality: the recovery of memory (via sbrk() back >to the system) can only occur at the end of the allocation region. Well, I'm playing with an neat little hack, which I can send you a diff for if you want, (all of one line :-) that uses madvise to tell the kernel that the contents of free pages can be thrown away, and a dmz be done on next access. This potentially saves a pageout and a pagein, possibly more, if you have plenty of ram, nothing. It's cheaper than to unmap, but still a little to expensive if done conseqently. The trick would be to have the kernel tell the processes when it's low on memory, which leads to the Real Solution: Use madvise to keep an counter updated in the process structure "int pages_i_can_give_back" and use that as a criterion for sending a "SIGNOMEM" to processes when memory gets short. This signal is used to ask the process to kindly mark anything it doesn't use as unused with madvise or free it. I've tried this, it's very easy, and very little code, but I never got to the signal sending stuff because it was rather obvious that none of my test-systems would benefit significantly, and consequently, the added complexity was not worth it. If you have a system that pages out a lot, you would gain something, but who has that ? Not me. RAM is cheaper than frying disks by swapping heavily. And that brings me to the conclusion: Only if you are paging a lot can you really gain something by avoiding paging pages out that nobody loves the contents of. If your paging rate is low, there is not much to be gained anywhere. I guess that the pragmatic solution would be to add a couple of options to MALLOC_OPTIONS and leave the matter of policy in the hands of the local sysad. Except of course that you can't set it system-wide in any meaningfull way at present. I have thought about adding a global default to malloc (/etc/default/malloc) to solve that problem. Comments ? Anybody wants to test the code ? -- Poul-Henning Kamp | phk@FreeBSD.ORG FreeBSD Core-team. whois: [PHK] | phk@ref.tfs.com TRW Financial Systems, Inc. Future will arrive by its own means, progress not so.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1056.841168977>