From owner-freebsd-small Sun Feb 11 22:11:24 2001 Delivered-To: freebsd-small@freebsd.org Received: from earth.backplane.com (earth-nat-cw.backplane.com [208.161.114.67]) by hub.freebsd.org (Postfix) with ESMTP id E9E3D37B401 for ; Sun, 11 Feb 2001 22:11:18 -0800 (PST) Received: (from dillon@localhost) by earth.backplane.com (8.11.2/8.9.3) id f1C6AfO02706; Sun, 11 Feb 2001 22:10:41 -0800 (PST) (envelope-from dillon) Date: Sun, 11 Feb 2001 22:10:41 -0800 (PST) From: Matt Dillon Message-Id: <200102120610.f1C6AfO02706@earth.backplane.com> To: "Forrest W. Christian" Cc: "Michael C . Wu" , freebsd-small@FreeBSD.ORG Subject: Re: Sans-Swap VM Subsystem Questions References: Sender: owner-freebsd-small@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG :In my instance, I crunchgen everything into one huge binary and hard link :the names... Which now I think about it brings up another question, but :I'll mention it below. : :The reason that I got thinking about this is that my crunchgen binary is :getting rather large, and I'm sure that there are clean pages which may or :may not be in use at a given time. I'd rather it free the clean pages :than blow up due to lack of memory. That sounds reasonable. I'm assuming you statically link it as well. There is a trade off here. If you have one big whopping static binary you will get a good memory footprint, but if you combine a lot of programs in the crunch you will wind up bringing in most of libc anyway. On the positive side, there's just one copy of it. On the negative side, any given program instance will be accessing procedures strewn all over the image and tend to cause most of the link library portion of the image to be in-core. The other alternative is to statically link the individual binaries separately. This will result in a larger amount of space taken up in the flash, but if you are only running one or two binaries predominantly the memory footprint will be excellent because the memory footprint for the 'idle' programs or the ones not being run will be zip. Most UNIX systems, including FreeBSD, have no problem throwing away clean pages which are not being used at any given moment. I don't think you have to worry about the clean pages. It's the dirty ones that are the gotcha (which is why static linking is better then using shared libraries which require run-time glueing). :Assuming you have one big homogenous program which does everything you :want (including init), and hard link multiple references to that one :inode, does the kernel understand that it should share the code? Or :better put, does the kernel actually look at the block/inode number on :disk when it tries to figure out what it can and cannot share/free/etc? Yes. The kernel will do the right thing. The kernel caches by vnode. Hardlinks are nothing more then multiple directory reference to the same vnode. :One additional kinda-related question.. In traditional PicoBSD, a memory :disk is used to store the code which is actually copied off of the :floppy. In this case, it would be cool if FreeBSD understood that the :code actually resided in memory, and didn't make a second copy of it. Do :you know if this is by chance the case? If it is, I'm seriously impressed :with whoever wrote that code...I'm still thinking about the cans :(cases?) of worms that could be. I'm not sure what picobsd is using, but its probably MFS. With MFS every active page will take up 2x the memory and every inactive page will take up 1x the memory (verses no memory if your backing store is some native device, like a flash rom). Compression has the same effect -- if something is compressed on the filesystem, the system eats dirty memory uncompressing it (compression is something that used to work with a.out binaries. I don't think it works for ELF binaries anyway). :> The kernel has no problem freeing up clean pages (backed by the original :> binary images in the flash memory), whereas without swap the kernel has :> no way to free up dirty pages. : :Ok, then this brings us back to the following options: : : options NO_SWAPPING (in the kernel config file) : vm.swap_enabled (sysctl) : vm.defer_swapspace_pageouts : vm.disable_swapspace_pageouts : (and probably a whole slew of others). : :In regards to options NO_SWAPPING, lint says it removes the swapping code :from the kernel. From the looks of things, it takes a *LOT* of the VM :subsystem with it. I realize there is some controversy over if this is :safe or not, but past experience shows that it doesn't seem to make :anything less stable. The question is whether freeing a clean page is :considered swapping in this context, or whether it just means actual :swapping of a dirty page. NO_SWAPPING should be fine, though I don't think I've ever used the option myself so you should make sure that the kernel still operates properly with it set. From my read of the code, NO_SWAPPING removes the pieces of the kernel that deal with deactivating dirty pages and leaves the pieces of the kernel that deal with freeing clean pages. So it should do what you want. :For the others, I mainly want to make sure that they generally don't :really matter in the context of not even having a swap partition on disk. : :Are the sysctl options *really* documented anywhere? Not really. If you set NO_SWAPPING I don't think there is anything special you have to do with the sysctls. It's roughly similar to a system with swapping enabled, but with no swap devices mounted. :> I also strongly recommend running with the 'H' malloc.conf option :> (see 'man malloc'). : :Am I reading the man page correctly that the way to set this is: : : ln -s H /etc/malloc.conf Correct. This will allow the kernel to be more agressive in reusing free()d pages that were previously malloc()'d. Normally such pages are left dirty, even though they are completely free. This option will cause the memory allocator to inform the kernel when such pages can be marked clean. It does not have any significant overhead. :> Beyond that, you have to be very, very, very careful in how you write :> the programs that will run under such a system. Remember, clean pages :> are essentially free, dirty pages cost memory. : :I have a feeling that the next stage of this project is nailing memory :leaks..... : :- Forrest W. Christian (forrestc@imach.com) AC7DE Yes. That's usually the biggest issue. What I recommend is that you wrap malloc() and free() without your own routines. e.g. for all of my projects I have a routine called zalloc() and zfree() (where my zalloc() allocates and zeros the memory, similar to calloc(), but taking only a single number-of-bytes argument like malloc()). In anycase, the real reason to wrap the routines is that then you can have a debug mode that tracks the allocations and frees by source file and line number. /* someheader.h */ #ifdef MEMDEBUG #define zalloc(bytes) _zalloc_debug(bytes, __FILE__, __LINE__) #define zfree(ptr, bytes) _zfree_debug(ptr, bytes, __FILE__, __LINE__) #else #define zalloc _zalloc #define zfree _zfree #endif If you write the wrapper to track the allocations and frees, then install a signal handler on SIGUSR1 or something like that to 'dump' a histogram of the count of the number of allocations made from any given file/line, it makes finding memory leaks really easy. -Matt To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-small" in the body of the message