Date: Sun, 6 Mar 2011 14:09:16 +0200 From: Vlad Galu <dudu@dudu.ro> To: Eric Anderson <anderson@ttel.com> Cc: Deomid Ryabkov <myself@rojer.pp.ru>, freebsd-hackers@freebsd.org Subject: Re: Mem leak : malloc/free + pthreads = leakage? Message-ID: <AANLkTi=K9QuK%2BbyxF1t7HzG9wKxXek804tNOgzM3Dp_b@mail.gmail.com> In-Reply-To: <AANLkTikd22VT3CVtvkGdw6zos%2Biq4Ve8y1W_WLr0qctS@mail.gmail.com> References: <40A52D4A-9397-4406-A7EC-E7CBBEFADD55@freebsd.org> <4D726887.5080800@rojer.pp.ru> <570723AB-7011-4185-89E1-E02F018DC0C3@ttel.com> <AANLkTikd22VT3CVtvkGdw6zos%2Biq4Ve8y1W_WLr0qctS@mail.gmail.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Sun, Mar 6, 2011 at 2:06 PM, Vlad Galu <dudu@dudu.ro> wrote: > > > On Sun, Mar 6, 2011 at 6:53 AM, Eric Anderson <anderson@ttel.com> wrote: > >> On Mar 5, 2011, at 10:44 AM, Deomid Ryabkov wrote: >> >> > On 03/05/2011 04:02 AM, Eric Anderson wrote: >> >> Hi all, >> >> >> >> I have a moderately threaded userland program (all C) I am working on >> (using pthreads, freebsd 8.1 64bit). It seems to leak memory (using >> standard malloc/free) badly. >> > as opposed to what? OpenBSD? Linux? Windows? why do you think your >> problem is specific to FreeBSD (as evidenced by your post to a >> FreeBSD-specific list) or is related to threaded programs? >> >> OpenSolaris and Mac OS X. I didn't really assume or state it was specific >> to FreeBSD, just that this scenario was on FreeBSD. I happen to do most of >> the development and testing on OS X and FreeBSD, and I've enjoyed the >> FreeBSD community for a very long time. >> >> >> >> > >> >> I am using pcap to capture packets and process them. I have a handful >> of libs statically linked in (pcap is one, the rest don't seem to matter - I >> can remove them and still see the leak). >> >> >> >> Does anyone know of issues regarding malloc/free on multithreaded >> userland apps? >> > hell yeah. it goes like this: you malloc() then forget to free() - boom, >> you have a memory leak. >> > >> > you're welcome. >> >> >> Thanks, very insightful. >> >> >> > >> > >> > sarcasm aside, those questions still remain: why do you think >> os/libraries are the problem and not your code? >> >> Because I am tracking all malloc and free calls within the application >> code (aside from libraries) and I can account for all malloc'ed memory and >> freed memory in both count and by bytes, yet looking at ps output shows a >> very different story, and if I leave it run long enough, will consume all >> memory and swap in the system and then be killed off. I wrapped malloc/free >> in a function, and record all memory alloc'ed and free'd. The only memory I >> cannot track is memory alloced and freed by libraries I am pulling in (well, >> can't track easily anyway without hacking through all of their source code). >> >> >> >> > you can't post all of it, ok, and we don't want all of it either. can >> you isolate a specific example of where valid usage of a library causes a >> leak? >> >> >> Not really - if I could, I would have fixed it by now. It's a non-trivial >> issue - which is why I am beginning to suspect something more complicated >> than a "oops I forgot to free" kind of error. Plus, I have seen a few >> people elsewhere on various forums/mailing lists with similar issues >> claiming that switching to the Hoard allocator fixed the problem (which >> doesn't seem to be happy with 32bit FreeBSD - tried it). I have also seen >> various comments about pthreads and memory allocators having apparent leaks >> at some threading level, but not sure. >> >> Thanks, >> Eric >> >> > Had there been a memleak in jemalloc, I'm sure more people would have > spotted it by now. How many pcap_t structs do you use in your app? libpcap > is not threadsafe. FWIW I've been running a pcap/threaded application > continuously for the past couple of years and the memory usage has been > constant. > > Also, put a couple of printf()s before and after > pcap_dispatch()/pcap_loop() (if you use any of them), to make sure they > don't block waiting for your callback to return (on some platforms it > doesn't, so you never get to free memory in outer frames). This might not > necessarily be your case, but it's worth taking a look at. > > That should've been pcap_dispatch()/pcap_next(). > >> >> _______________________________________________ >> freebsd-hackers@freebsd.org mailing list >> http://lists.freebsd.org/mailman/listinfo/freebsd-hackers >> To unsubscribe, send any mail to "freebsd-hackers-unsubscribe@freebsd.org >> " >> > > > > -- > Good, fast & cheap. Pick any two. > -- Good, fast & cheap. Pick any two.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?AANLkTi=K9QuK%2BbyxF1t7HzG9wKxXek804tNOgzM3Dp_b>