Date: Sun, 6 Mar 2011 14:06:45 +0200 From: Vlad Galu <dudu@dudu.ro> To: Eric Anderson <anderson@ttel.com> Cc: Deomid Ryabkov <myself@rojer.pp.ru>, freebsd-hackers@freebsd.org Subject: Re: Mem leak : malloc/free + pthreads = leakage? Message-ID: <AANLkTikd22VT3CVtvkGdw6zos%2Biq4Ve8y1W_WLr0qctS@mail.gmail.com> In-Reply-To: <570723AB-7011-4185-89E1-E02F018DC0C3@ttel.com> References: <40A52D4A-9397-4406-A7EC-E7CBBEFADD55@freebsd.org> <4D726887.5080800@rojer.pp.ru> <570723AB-7011-4185-89E1-E02F018DC0C3@ttel.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Mar 6, 2011 at 6:53 AM, Eric Anderson <anderson@ttel.com> wrote: > On Mar 5, 2011, at 10:44 AM, Deomid Ryabkov wrote: > > > On 03/05/2011 04:02 AM, Eric Anderson wrote: > >> Hi all, > >> > >> I have a moderately threaded userland program (all C) I am working on > (using pthreads, freebsd 8.1 64bit). It seems to leak memory (using > standard malloc/free) badly. > > as opposed to what? OpenBSD? Linux? Windows? why do you think your > problem is specific to FreeBSD (as evidenced by your post to a > FreeBSD-specific list) or is related to threaded programs? > > OpenSolaris and Mac OS X. I didn't really assume or state it was specific > to FreeBSD, just that this scenario was on FreeBSD. I happen to do most of > the development and testing on OS X and FreeBSD, and I've enjoyed the > FreeBSD community for a very long time. > > > > > > >> I am using pcap to capture packets and process them. I have a handful > of libs statically linked in (pcap is one, the rest don't seem to matter - I > can remove them and still see the leak). > >> > >> Does anyone know of issues regarding malloc/free on multithreaded > userland apps? > > hell yeah. it goes like this: you malloc() then forget to free() - boom, > you have a memory leak. > > > > you're welcome. > > > Thanks, very insightful. > > > > > > > > sarcasm aside, those questions still remain: why do you think > os/libraries are the problem and not your code? > > Because I am tracking all malloc and free calls within the application code > (aside from libraries) and I can account for all malloc'ed memory and freed > memory in both count and by bytes, yet looking at ps output shows a very > different story, and if I leave it run long enough, will consume all memory > and swap in the system and then be killed off. I wrapped malloc/free in a > function, and record all memory alloc'ed and free'd. The only memory I > cannot track is memory alloced and freed by libraries I am pulling in (well, > can't track easily anyway without hacking through all of their source code). > > > > > you can't post all of it, ok, and we don't want all of it either. can you > isolate a specific example of where valid usage of a library causes a leak? > > > Not really - if I could, I would have fixed it by now. It's a non-trivial > issue - which is why I am beginning to suspect something more complicated > than a "oops I forgot to free" kind of error. Plus, I have seen a few > people elsewhere on various forums/mailing lists with similar issues > claiming that switching to the Hoard allocator fixed the problem (which > doesn't seem to be happy with 32bit FreeBSD - tried it). I have also seen > various comments about pthreads and memory allocators having apparent leaks > at some threading level, but not sure. > > Thanks, > Eric > > Had there been a memleak in jemalloc, I'm sure more people would have spotted it by now. How many pcap_t structs do you use in your app? libpcap is not threadsafe. FWIW I've been running a pcap/threaded application continuously for the past couple of years and the memory usage has been constant. Also, put a couple of printf()s before and after pcap_dispatch()/pcap_loop() (if you use any of them), to make sure they don't block waiting for your callback to return (on some platforms it doesn't, so you never get to free memory in outer frames). This might not necessarily be your case, but it's worth taking a look at. > > > _______________________________________________ > freebsd-hackers@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-hackers > To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org" > -- Good, fast & cheap. Pick any two.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTikd22VT3CVtvkGdw6zos%2Biq4Ve8y1W_WLr0qctS>