Date: Sat, 29 Sep 2001 23:29:53 +0000 From: Vladimir Dozen <vladimir-dozen@mail.ru> To: Matt Dillon <dillon@earth.backplane.com> Cc: Wilko Bulte <wkb@freebie.xs4all.nl>, Alfred Perlstein <bright@mu.org>, hackers@FreeBSD.ORG Subject: Re: VM: dynamic swap remapping (patch) Message-ID: <20010929232953.B341@eix.do-labs.spb.ru> In-Reply-To: <200109291653.f8TGrRR37689@earth.backplane.com>; from dillon@earth.backplane.com on Sat, Sep 29, 2001 at 09:53:27AM -0700 References: <20010929155941.A291@eix.do-labs.spb.ru> <20010929071024.Q59854@elvis.mu.org> <20010929141349.A80876@freebie.xs4all.nl> <200109291653.f8TGrRR37689@earth.backplane.com>
next in thread | previous in thread | raw e-mail | index | archive | help
ehlo.
> You also cannot guarentee that your process will have time to
> cleanup prior to the system killing, nor can you guarentee that all the
> standard system utilities and daemons will be able to gracefully handle
> the out of memory condition. In otherwords, you could implement
> the signal and even have the program use it, but you will still likely
> leave gaping holes in the implementation that will result in lost data.
Actually, the things as I coded them better suited namely for poorly
written daemons that never check for malloc result. Precommit will just
kill them as soon as malloc() will return NULL, and they dereference it.
Killproc() will kill them too. Remapping will save them. Disk space
now is large enough to make them live till root will notice that
they grow to much and do something (kill them manually, probably ;).
> It is much easier to manage memory manually. For example, if these
> programs require 1G of independant memory to run it ought to be a
> fairly simple matter to simply create a 1GB file for each process
> (using dd rather then ftruncate() to create the file so the blocks are
> preallocated), mmap() it using PROT_READ|PROT_WRITE,MAP_SHARED|MAP_NOSYNC,
> and do your memory management out of that.
First at all, it is NOT easier. Doing own memory management is not too
simple, especially for threads and SMP -- we seen 50% performance impact
when two threads on two processors were doing intensive allocations
(it was not FreeBSD, and these was kernel threads).
Second, application not always grows to 1G, most of the time it keeps
as small as 500M ;). Why should we precommit 1G for 500M data? Doing
multi-mmap memory management is additional pain.
Third, swapping to device is faster, and, while we have enough swap,
I would prefer to swap there. Even a few percent for 5-day computation
make sense.
> Problem solved.
If I'm the developer -- probably, yes. What if I'm system administrator,
and has to run something large _and important_? The day I'll notice
that monster creates swap files I'll know I have to add RAM. I will
have time since it still works, it was not killed.
P.S. Anyway, I do NOT insist my solution is better, and even that it
is good for anything at all. It was fun for me to hack in BSD kernel,
and it was interesting challenge, and I feel need to share results
with others. At worst, I will recommend our customer to setup
processing farm under FreeBSD with applied patch.
--
dozen @ home
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-hackers" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20010929232953.B341>