From owner-freebsd-hackers Tue Apr 11 12:11:26 1995 Return-Path: hackers-owner Received: (from majordom@localhost) by freefall.cdrom.com (8.6.10/8.6.6) id MAA29692 for hackers-outgoing; Tue, 11 Apr 1995 12:11:26 -0700 Received: from vinkku.hut.fi (vode@vinkku.hut.fi [130.233.245.1]) by freefall.cdrom.com (8.6.10/8.6.6) with ESMTP id MAA29682 for ; Tue, 11 Apr 1995 12:11:20 -0700 Received: (from vode@localhost) by vinkku.hut.fi (8.6.11/8.6.7) id WAA08335; Tue, 11 Apr 1995 22:11:13 +0300 Date: Tue, 11 Apr 1995 22:11:13 +0300 From: Kai Vorma Message-Id: <199504111911.WAA08335@vinkku.hut.fi> To: hackers@FreeBSD.org Subject: A better malloc() for FreeBSD Reply-to: Kai.Vorma@hut.fi Sender: hackers-owner@FreeBSD.org Precedence: bulk As I wrote about two weeks ago the BSD malloc isn't very space-efficient but wastes memory quite liberally. This is because of two main reasons 1) It allocates blocks that are 2^n-4 bytes long. So if you say malloc(1024) you actually get 2044 bytes. 2) It cannot coalesc freed blocks: break is 0x1fbffc (2072032 bytes sbrked) after 1000 allocations of 1024 break is 0x1fbffc (2072032 bytes sbrked) after freeing all allocations break is 0x27cffc (2600416 bytes sbrked) after allocating 512000 I searched for better malloc implementations and found two good candidates: o CSRI malloc v1.17 by Mark Moraes o malloc-2.5.3b by Doug Lea Both mallocs are very space-efficient and performs coalescing on frees. The CSRI malloc is a bit slow and somewhat bloated (it does have hooks for debugging and some helper functions like strdup, emalloc, evalloc etc.). Doug Lea's malloc is very fast and simple so I choosed it. I replaced the BSD malloc in libc with Doug Lea's malloc about week ago and recompiled all static programs in /bin and /sbin. So far I have had only two problems: 1) sed crashes in some situations. At the end of this message is a patch (commited to freebsd-current a few days ago) that fixes sed. 2) Originally I installed Doug Lea's malloc as stdlib/dlmalloc.c and just commented out old malloc (gen/valloc.c, stdlib/malloc.c and stdlib/calloc.c) from makefiles. Everything worked smoothly until I tried to link something with static gnumalloc (linker complained about multiple defined symbols _malloc etc. Why?) so I had to replace the original stdlib/malloc.c with dlmalloc.c and comment out calloc() and valloc() from it. Does it help? Perhaps :-) I have 16MB of memory so things go smoothly anyway. However, the virtual size of XF_86 dropped from 7MB to 4MB and xv no longer grows until it hits datasize limit. I would suppose that the new malloc causes less TLB and page faults, too. If you want to try your luck then fetch malloc-2.5.3b.c from gee.cs.oswego.edu:/pub/misc, apply the following patch to it and install it as /usr/src/lib/libc/stdlib/malloc.c. Then recompile and install libc (take a backup of your old libc.so first! Actually take a full backup of your system first - I managed to destroy my cnews installation before I found and fixed a bug in sed). I'd like to see Doug Lea's malloc integrated into libc at some point. I don't think it should go into 2.1 but how about -current after 2.1 is out? ..vode PS. The comments says that it is a preliminary working version and unreleased. I asked the author and he said that it is stable and he has just forgotten to remove those comments. --------------------------------------------------------------------------- *** usr/src/usr.bin/sed/compile.c.old Tue Apr 11 10:44:45 1995 --- usr/src/usr.bin/sed/compile.c Thu Apr 6 05:15:37 1995 *************** *** 217,222 **** --- 217,224 ---- if (!*p) p = NULL; cmd2 = xmalloc(sizeof(struct s_command)); + cmd2->nonsel = cmd2->inrange = 0; + cmd2->a1 = cmd2->a2 = NULL; cmd2->code = '}'; *compile_stream("}", &cmd->u.c, p) = cmd2; cmd->next = cmd2; --------------------------------------------------------------------------- *** malloc-2.5.3b.c Sun Mar 26 10:50:54 1995 --- malloc.c Fri Apr 7 19:21:29 1995 *************** *** 271,276 **** --- 271,278 ---- /* preliminaries */ + #include + #ifndef __STD_C #ifdef __STDC__ #define __STD_C 1 *************** *** 1365,1373 **** } - /* Derivatives */ #if __STD_C Void_t* valloc(size_t bytes) #else --- 1367,1375 ---- } /* Derivatives */ + #if 0 #if __STD_C Void_t* valloc(size_t bytes) #else *************** *** 1394,1400 **** while (sz-- > 0) *q++ = 0; return p; } ! #if __STD_C void cfree(Void_t *mem) #else --- 1396,1402 ---- while (sz-- > 0) *q++ = 0; return p; } ! #endif #if __STD_C void cfree(Void_t *mem) #else