Date: Sun, 04 Nov 2012 07:59:01 -0700 From: Ian Lepore <freebsd@damnhippie.dyndns.org> To: Dylan Cochran <a134qaed@gmail.com> Cc: freebsd-hackers@freebsd.org, freebsd-embedded@freebsd.org Subject: Re: watchdogd, jemalloc, and mlockall Message-ID: <1352041141.1120.146.camel@revolution.hippie.lan> In-Reply-To: <CA%2B8JZ0cRhHft7FsL=wO7e-1yOz4c-Whq=UXKokmK1ZWBOGepqA@mail.gmail.com> References: <1351967919.1120.102.camel@revolution.hippie.lan> <50957793.8060709@delphij.net> <1351973531.1120.118.camel@revolution.hippie.lan> <CA%2B8JZ0cRhHft7FsL=wO7e-1yOz4c-Whq=UXKokmK1ZWBOGepqA@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, 2012-11-04 at 09:16 -0500, Dylan Cochran wrote: > Have you already tried something like opt.lg_chunk? This, combined > with other options for the library (man 3 jemalloc), should reduce the > space from 8MB down to 16K, or so. (approximation, I'm being liberal > for jemalloc's internal bookkeeping size). For a special case like > watchdogd, this would make more sense in general (given it should be > designed to do no allocations/deletions during normal operation > anyway). For other programs, this would be as unwise as statically > linking them. > I had completely missed the fact that jemalloc had its own manpage, thank you. Given that new information I think the pieces are in place to put watchdogd on a memory diet. I'll work up a patch in the next couple days -- Ian > The 'perfect' solution would obviously be improving the library > manager (rtld) to only mmap() function pages it needs, though I will > admit I'm not sure if the ELF format is even capable of supporting > something like that, what other problems it would cause down the road, > or if it even attempts to do this already (I haven't looked at the > runtime linker code since 7.0). > > By the way, remember that when you compare static v dynamic, that the > runtime linker does allocate private memory to handle the resolving of > symbols to virtual memory addresses. That may skew your memory usage > figures a bit. > > On Sat, Nov 3, 2012 at 4:12 PM, Ian Lepore > <freebsd@damnhippie.dyndns.org> wrote: > > > > On Sat, 2012-11-03 at 12:59 -0700, Xin Li wrote: > > > -----BEGIN PGP SIGNED MESSAGE----- > > > Hash: SHA256 > > > > > > On 11/3/12 11:38 AM, Ian Lepore wrote: > > > > In an attempt to un-hijack the thread about memory usage increase > > > > between 6.4 and 9.x, I'm starting a new thread here related to my > > > > recent discovery that watchdogd uses a lot more memory since it > > > > began using mlockall(2). > > > > > > > > I tried statically linking watchdogd and it made a small difference > > > > in RSS, presumably because it doesn't wire down all of libc and > > > > libm. > > > > > > Speaking for this, the last time I brought this up, someone (can't > > > remember, I think it was phk@) argued that the shared library would > > > use only one copy of memory, while statically linked ones would be > > > duplicated and thus use more memory. I haven't yet tried to prove or > > > challenge that, though. > > > > That sounds right to me... if 3 or 4 daemons were to eventually be > > statically linked because of mlockall(), then each of them would have > > its own private copy of strlen(), and malloc(), and so on; we'd be back > > to the bad old days before shared libs came along. Each program would > > contain its own copy of only the routines from the library that it uses, > > not the entire library in each program. > > > > On the other hand, if even one daemon linked with shared libc uses > > mlockall(), then all of libc gets wired. As I understand it, only one > > physical copy of libc would exist in memory, still shared by almost all > > running apps. The entire contents of the library would continuously > > occupy physical memory, even the parts that no apps are using. > > > > It's hard to know how to weigh the various tradeoffs. I suspect there's > > no one correct answer. > > > > -- Ian > > > > > > _______________________________________________ > > freebsd-hackers@freebsd.org mailing list > > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org"
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?1352041141.1120.146.camel>