Date: Fri, 3 Oct 1997 14:17:17 -0400 (EDT) From: Thomas David Rivers <rivers@dignus.com> To: jkh@time.cdrom.com, phk@critter.freebsd.dk Cc: freebsd-bugs@FreeBSD.ORG, gram@cdsec.com, hackers@FreeBSD.ORG Subject: Electric Fence info (was Re: Bug in malloc/free (was: Memory leak in getservbyXXX?)) Message-ID: <199710031817.OAA16814@lakes.dignus.com>
next in thread | raw e-mail | index | archive | help
>
> > This would indicate a bug of the class where memory is written to after
> > being free()'ed, a kind of bug which phkmalloc makes no attempt to catch.
>
> Man, I sure wish there was a copy of purify available for FreeBSD.
> It's great at catching stuff like this! :(
>
> Maybe you could hack free() to do an mprotect(addr, len, PROT_NONE) on
> free'd pages, unprotecting them again as necessary when the malloc
> routines themselves need to frob that memory. Or, since we're just
> testing, do it from an internally registered SIGBUS handler which
> figures out the right thing to do. :-)
>
> BTW, how *do* you get the faulting memory location from a SIGBUS
> handler? I was just playing around with this a bit and noted that it
> wasn't immediately obvious how you'd get that info from the signal
> handler.
>
> Jordan
Ok - there seems to be enough interest... Here's the man page
for electric fence (which does the mprotect() trick Jordan mentions):
I can make the source available to anyone who wants it.
- Dave Rivers -
efence(3) efence(3)
NAME
efence - Electric Fence Malloc Debugger
SYNOPSIS
#include <stdlib.h>
void * malloc (size_t size);
void free (void *ptr);
void * realloc (void *ptr, size_t size);
void * calloc (size_t nelem, size_t elsize);
void * memalign (size_t alignment, size_t size);
void * valloc (size_t size);
extern int EF_ALIGNMENT;
extern int EF_PROTECT_BELOW;
extern int EF_PROTECT_FREE;
DESCRIPTION
Electric Fence helps you detect two common programming
bugs: software that overruns the boundaries of a malloc()
memory allocation, and software that touches a memory
allocation that has been released by free(). Unlike other
malloc() debuggers, Electric Fence will detect read
accesses as well as writes, and it will pinpoint the exact
instruction that causes an error. It has been in use at
Pixar since 1987, and at many other sites for years.
Electric Fence uses the virtual memory hardware of your
computer to place an inaccessible memory page immediately
after (or before, at the user's option) each memory allo-
cation. When software reads or writes this inaccessible
page, the hardware issues a segmentation fault, stopping
the program at the offending instruction. It is then triv-
ial to find the erroneous statement using your favorite
debugger. In a similar manner, memory that has been
released by free() is made inaccessible, and any code that
touches it will get a segmentation fault.
Simply linking your application with libefence.a will
allow you to detect most, but not all, malloc buffer over-
runs and accesses of free memory. If you want to be rea-
sonably sure that you've found all bugs of this type,
you'll have to read and understand the rest of this man
page.
USAGE
Link your program with the library libefence.a . Make
27-April-1993 1
efence(3) efence(3)
sure you are not linking with -lmalloc, -lmallocdebug, or
with other malloc-debugger or malloc-enhancer libraries.
You can only use one at a time. If your system adminis-
trator has installed Electric Fence for public use, you'll
be able to use the -lefence argument to the linker, other-
wise you'll have to put the path-name for libefence.a in
the linker's command line. Some systems will require spe-
cial arguments to the linker to assure that you are using
the Electric Fence malloc() and not the one from your C
library. On AIX systems, you may have to use the flags
-bnso -bnodelcsect -bI:/lib/syscalls.exp
On Sun systems running SunOS 4.X, you'll probably have to
use -Bstatic.
Run your program using a debugger. It's easier to work
this way than to create a core file and post-mortem debug
it. Electric Fence can create huge core files, and some
operating systems will thus take minutes simply to dump
core! Some operating systems will not create usable core
files from programs that are linked with Electric Fence.
If your program has one of the errors detected by Electric
Fence, it will get a segmentation fault (SIGSEGV) at the
offending instruction. Use the debugger to locate the
erroneous statement, and repair it.
GLOBAL AND ENVIRONMENT VARIABLES
Electric Fence has four configuration switches that can be
enabled via the shell environment, or by setting the value
of global integer variables using a debugger. These
switches change what bugs Electric Fence will detect, so
it's important that you know how to use them.
EF_ALIGNMENT
This is an integer that specifies the alignment for
any memory allocations that will be returned by
malloc(), calloc(), and realloc(). The value is
specified in bytes, thus a value of 4 will cause
memory to be aligned to 32-bit boundaries unless
your system doesn't have a 8-bit characters.
EF_ALIGNMENT is set to sizeof(int) by default,
since that is generally the word-size of your CPU.
If your program requires that allocations be
aligned to 64-bit boundaries and you have a 32-bit
int you'll have to set this value to 8. This is the
case when compiling with the -mips2 flag on MIPS-
based systems such as those from SGI. The memory
allocation that is returned by Electric Fence mal-
loc() is aligned using the value in EF_ALIGNMENT,
and its size the multiple of that value that is
greater than or equal to the requested size. For
this reason, you will sometimes want to set
EF_ALIGNMENT to 0 (no alignment), so that you can
detect overruns of less than your CPU's word size.
Be sure to read the section WORD-ALIGNMENT AND
27-April-1993 2
efence(3) efence(3)
OVERRUN DETECTION in this manual page before you
try this. To change this value, set EF_ALIGNMENT
in the shell environment to an integer value, or
assign to the global integer variable EF_ALIGNMENT
using a debugger.
EF_PROTECT_BELOW
Electric Fence usually places an inaccessible page
immediately after each memory allocation, so that
software that runs past the end of the allocation
will be detected. Setting EF_PROTECT_BELOW to 1
causes Electric Fence to place the inaccessible
page before the allocation in the address space, so
that under-runs will be detected instead of over-
runs. When EF_PROTECT_BELOW is set, the EF_ALIGN-
MENT parameter is ignored. All allocations will be
aligned to virtual-memory-page boundaries, and
their size will be the exact size that was
requested. To change this value, set EF_PRO-
TECT_BELOW in the shell environment to an integer
value, or assign to the global integer variable
EF_PROTECT_BELOW using a debugger.
EF_PROTECT_FREE
Electric Fence usually returns free memory to a
pool from which it may be re-allocated. If you sus-
pect that a program may be touching free memory,
set EF_PROTECT_FREE to 1. This will cause Electric
Fence to never re-allocate memory once it has been
freed, so that any access to free memory will be
detected. Some programs will use tremendous amounts
of memory when this parameter is set. To change
this value, set EF_PROTECT_FREE in the shell envi-
ronment to an integer value, or assign to the
global integer variable EF_PROTECT_FREE using a
debugger.
EF_ALLOW_MALLOC_0
By default, Electric Fence traps calls to malloc()
with a size of zero, because they are often the
result of a software bug. If EF_ALLOW_MALLOC_0 is
non-zero, the software will not trap calls to mal-
loc() with a size of zero. To change this value,
set EF_ALLOC_MALLOC_0 in the shell environment to
an integer value, or assign to the global integer
variable EF_ALLOC_MALLOC_0 using a debugger.
WORD-ALIGNMENT AND OVERRUN DETECTION
There is a conflict between the alignment restrictions
that malloc() operates under and the debugging strategy
used by Electric Fence. When detecting overruns, Electric
Fence malloc() allocates two or more virtual memory pages
for each allocation. The last page is made inaccessible in
such a way that any read, write, or execute access will
27-April-1993 3
efence(3) efence(3)
cause a segmentation fault. Then, Electric Fence malloc()
will return an address such that the first byte after the
end of the allocation is on the inaccessible page. Thus,
any overrun of the allocation will cause a segmentation
fault.
It follows that the address returned by malloc() is the
address of the inaccessible page minus the size of the
memory allocation. Unfortunately, malloc() is required to
return word-aligned allocations, since many CPUs can only
access a word when its address is aligned. The conflict
happens when software makes a memory allocation using a
size that is not a multiple of the word size, and expects
to do word accesses to that allocation. The location of
the inaccessible page is fixed by hardware at a word-
aligned address. If Electric Fence malloc() is to return
an aligned address, it must increase the size of the allo-
cation to a multiple of the word size. In addition, the
functions memalign() and valloc() must honor explicit
specifications on the alignment of the memory allocation,
and this, as well can only be implemented by increasing
the size of the allocation. Thus, there will be situa-
tions in which the end of a memory allocation contains
some padding space, and accesses of that padding space
will not be detected, even if they are overruns.
Electric Fence provides the variable EF_ALIGNMENT so that
the user can control the default alignment used by mal-
loc(), calloc(), and realloc(). To debug overruns as
small as a single byte, you can set EF_ALIGNMENT to zero.
This will result in Electric Fence malloc() returning
unaligned addresses for allocations with sizes that are
not a multiple of the word size. This is not a problem in
most cases, because compilers must pad the size of objects
so that alignment restrictions are honored when storing
those objects in arrays. The problem surfaces when soft-
ware allocates odd-sized buffers for objects that must be
word-aligned. One case of this is software that allocates
a buffer to contain a structure and a string, and the
string has an odd size (this example was in a popular TIFF
library). If word references are made to un-aligned
buffers, you will see a bus error (SIGBUS) instead of a
segmentation fault. The only way to fix this is to re-
write the offending code to make byte references or not
make odd-sized allocations, or to set EF_ALIGNMENT to the
word size.
Another example of software incompatible with EF_ALIGNMENT
< word-size is the strcmp() function and other string
functions on SunOS (and probably Solaris), which make
word-sized accesses to character strings, and may attempt
to access up to three bytes beyond the end of a string.
These result in a segmentation fault (SIGSEGV). The only
way around this is to use versions of the string functions
27-April-1993 4
efence(3) efence(3)
that perform byte references instead of word references.
INSTRUCTIONS FOR DEBUGGING YOUR PROGRAM
1. Link with libefence.a as explained above.
2. Run your program in a debugger and fix any overruns
or accesses to free memory.
3. Quit the debugger.
4. Set EF_PROTECT_BELOW = 1 in the shell environment.
5. Repeat step 2, this time repairing underruns if
they occur.
6. Quit the debugger.
7. Read the restrictions in the section on WORD-ALIGN-
MENT AND OVERRUN DETECTION. See if you can set
EF_ALIGNMENT to 0 and repeat step 2. Sometimes this
will be too much work, or there will be problems
with library routines for which you don't have the
source, that will prevent you from doing this.
MEMORY USAGE AND EXECUTION SPEED
Since Electric Fence uses at least two virtual memory
pages for each of its allocations, it's a terrible memory
hog. I've sometimes found it necessary to add a swap file
using swapon(8) so that the system would have enough vir-
tual memory to debug my program. Also, the way we manipu-
late memory results in various cache and translation
buffer entries being flushed with each call to malloc or
free. The end result is that your program will be much
slower and use more resources while you are debugging it
with Electric Fence.
Don't leave libefence.a linked into production software!
Use it only for debugging.
PORTING
Electric Fence is written for ANSI C. You should be able
to port it with simple changes to the Makefile and to
page.c, which contains the memory management primitives .
Many POSIX platforms will require only a re-compile. The
operating system facilities required to port Electric
Fence are:
A way to allocate memory pages
A way to make selected pages inaccessible.
A way to make the pages accessible again.
A way to detect when a program touches an inacces-
sible page.
A way to print messages.
27-April-1993 5
efence(3) efence(3)
Please e-mail me a copy of any changes you have to make,
so that I can merge them into the distribution.
AUTHOR
Bruce Perens
WARNINGS
I have tried to do as good a job as I can on this soft-
ware, but I doubt that it is even theoretically possible
to make it bug-free. This software has no warranty. It
will not detect some bugs that you might expect it to
detect, and will indicate that some non-bugs are bugs.
Bruce Perens and/or Pixar will not be liable to any claims
resulting from the use of this software or the ideas
within it. The entire responsibility for its use must be
assumed by the user. If you use it and it results in loss
of life and/or property, tough. If it leads you on a wild
goose chase and you waste two weeks debugging something,
too bad. If you can't deal with the above, please don't
use the software! I've written this in an attempt to help
other people, not to get myself sued or prosecuted.
LICENSE
Copyright 1987,1988,1989,1990,1991,1992,1993 Bruce Perens.
All Rights Reserved except for those granted in this
notice.
Permission is granted for you to use this software to
debug other programs. You may re-distribute it the origi-
nal form in which I released it to you. If you make modi-
fications to it, please send them to me for distribution -
you may not re-distribute modified versions. You may not
sell this software. You may not sell support for this
software. If you use ideas from this software in your own
product, please pay me for them.
CONTACTING THE AUTHOR
Bruce Perens
c/o Pixar
1001 West Cutting Blvd., Suite 200
Richmond, CA 94804
Telephone: 510-215-3502
Fax: 510-236-0388
Internet: Bruce@Pixar.com
FILES
/dev/zero: Source of memory pages (via mmap(2)).
SEE ALSO
malloc(3), mmap(2), mprotect(2), swapon(8)
DIAGNOSTICS
Segmentation Fault: Examine the offending statement for
violation of the boundaries of a memory allocation.
27-April-1993 6
efence(3) efence(3)
Bus Error: See the section on WORD-ALIGNMENT AND OVERRUN
DETECTION. in this manual page.
BUGS
My explanation of the alignment issue could be improved.
Some Sun systems running SunOS 4.1 are reported to signal
an access to a protected page with SIGBUS rather than
SIGSEGV, I suspect this is an undocumented feature of a
particular Sun hardware version, not just the operating
system. On these systems, eftest will fail with a bus
error until you modify the Makefile to define PAGE_PROTEC-
TION_VIOLATED_SIGNAL as SIGBUS.
There are, without doubt, other bugs and porting issues.
Please contact me via e-mail if you have any bug reports,
ideas, etc.
WHAT'S BETTER
PURIFY, from Purify Systems, does a much better job than
Electric Fence, and does much more. It's available at this
writing on SPARC systems only, soon on HP. I'm not affili-
ated with Purify, I just think it's a wonderful product
and you should check it out.
27-April-1993 7
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?199710031817.OAA16814>