Date: Sun, 11 Feb 2001 22:10:41 -0800 (PST) From: Matt Dillon <dillon@earth.backplane.com> To: "Forrest W. Christian" <forrestc@imach.com> Cc: "Michael C . Wu" <keichii@iteration.net>, freebsd-small@FreeBSD.ORG Subject: Re: Sans-Swap VM Subsystem Questions Message-ID: <200102120610.f1C6AfO02706@earth.backplane.com> References: <Pine.BSF.4.21.0102111925480.22351-100000@workhorse.iMach.com>
next in thread | previous in thread | raw e-mail | index | archive | help
:In my instance, I crunchgen everything into one huge binary and hard link
:the names... Which now I think about it brings up another question, but
:I'll mention it below.
:
:The reason that I got thinking about this is that my crunchgen binary is
:getting rather large, and I'm sure that there are clean pages which may or
:may not be in use at a given time. I'd rather it free the clean pages
:than blow up due to lack of memory.
That sounds reasonable. I'm assuming you statically link it as well.
There is a trade off here. If you have one big whopping static binary
you will get a good memory footprint, but if you combine a lot
of programs in the crunch you will wind up bringing in most of libc
anyway. On the positive side, there's just one copy of it. On the
negative side, any given program instance will be accessing procedures
strewn all over the image and tend to cause most of the link library
portion of the image to be in-core.
The other alternative is to statically link the individual binaries
separately. This will result in a larger amount of space taken up in
the flash, but if you are only running one or two binaries predominantly
the memory footprint will be excellent because the memory footprint for
the 'idle' programs or the ones not being run will be zip.
Most UNIX systems, including FreeBSD, have no problem throwing away
clean pages which are not being used at any given moment. I don't
think you have to worry about the clean pages. It's the dirty ones
that are the gotcha (which is why static linking is better then
using shared libraries which require run-time glueing).
:Assuming you have one big homogenous program which does everything you
:want (including init), and hard link multiple references to that one
:inode, does the kernel understand that it should share the code? Or
:better put, does the kernel actually look at the block/inode number on
:disk when it tries to figure out what it can and cannot share/free/etc?
Yes. The kernel will do the right thing. The kernel caches by vnode.
Hardlinks are nothing more then multiple directory reference to the
same vnode.
:One additional kinda-related question.. In traditional PicoBSD, a memory
:disk is used to store the code which is actually copied off of the
:floppy. In this case, it would be cool if FreeBSD understood that the
:code actually resided in memory, and didn't make a second copy of it. Do
:you know if this is by chance the case? If it is, I'm seriously impressed
:with whoever wrote that code...I'm still thinking about the cans
:(cases?) of worms that could be.
I'm not sure what picobsd is using, but its probably MFS. With MFS every
active page will take up 2x the memory and every inactive page will take
up 1x the memory (verses no memory if your backing store is some native
device, like a flash rom). Compression has the same effect -- if
something is compressed on the filesystem, the system eats dirty memory
uncompressing it (compression is something that used to work with a.out
binaries. I don't think it works for ELF binaries anyway).
:> The kernel has no problem freeing up clean pages (backed by the original
:> binary images in the flash memory), whereas without swap the kernel has
:> no way to free up dirty pages.
:
:Ok, then this brings us back to the following options:
:
: options NO_SWAPPING (in the kernel config file)
: vm.swap_enabled (sysctl)
: vm.defer_swapspace_pageouts
: vm.disable_swapspace_pageouts
: (and probably a whole slew of others).
:
:In regards to options NO_SWAPPING, lint says it removes the swapping code
:from the kernel. From the looks of things, it takes a *LOT* of the VM
:subsystem with it. I realize there is some controversy over if this is
:safe or not, but past experience shows that it doesn't seem to make
:anything less stable. The question is whether freeing a clean page is
:considered swapping in this context, or whether it just means actual
:swapping of a dirty page.
NO_SWAPPING should be fine, though I don't think I've ever used the
option myself so you should make sure that the kernel still operates
properly with it set. From my read of the code, NO_SWAPPING removes
the pieces of the kernel that deal with deactivating dirty pages and
leaves the pieces of the kernel that deal with freeing clean pages. So
it should do what you want.
:For the others, I mainly want to make sure that they generally don't
:really matter in the context of not even having a swap partition on disk.
:
:Are the sysctl options *really* documented anywhere?
Not really. If you set NO_SWAPPING I don't think there is anything
special you have to do with the sysctls. It's roughly similar to
a system with swapping enabled, but with no swap devices mounted.
:> I also strongly recommend running with the 'H' malloc.conf option
:> (see 'man malloc').
:
:Am I reading the man page correctly that the way to set this is:
:
: ln -s H /etc/malloc.conf
Correct. This will allow the kernel to be more agressive in reusing
free()d pages that were previously malloc()'d. Normally such pages
are left dirty, even though they are completely free. This option
will cause the memory allocator to inform the kernel when such pages
can be marked clean. It does not have any significant overhead.
:> Beyond that, you have to be very, very, very careful in how you write
:> the programs that will run under such a system. Remember, clean pages
:> are essentially free, dirty pages cost memory.
:
:I have a feeling that the next stage of this project is nailing memory
:leaks.....
:
:- Forrest W. Christian (forrestc@imach.com) AC7DE
Yes. That's usually the biggest issue. What I recommend is that you
wrap malloc() and free() without your own routines. e.g. for all of
my projects I have a routine called zalloc() and zfree() (where my
zalloc() allocates and zeros the memory, similar to calloc(), but taking
only a single number-of-bytes argument like malloc()). In anycase,
the real reason to wrap the routines is that then you can have a debug
mode that tracks the allocations and frees by source file and line number.
/* someheader.h */
#ifdef MEMDEBUG
#define zalloc(bytes) _zalloc_debug(bytes, __FILE__, __LINE__)
#define zfree(ptr, bytes) _zfree_debug(ptr, bytes, __FILE__, __LINE__)
#else
#define zalloc _zalloc
#define zfree _zfree
#endif
If you write the wrapper to track the allocations and frees, then
install a signal handler on SIGUSR1 or something like that to 'dump'
a histogram of the count of the number of allocations made from
any given file/line, it makes finding memory leaks really easy.
-Matt
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-small" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200102120610.f1C6AfO02706>