Date: Thu, 19 Feb 2015 16:04:53 -0600 From: Matthew Grooms <mgrooms@shrew.net> To: freebsd-stable@freebsd.org Subject: pthread leaky with resources? Message-ID: <54E65E05.2040101@shrew.net>
next in thread | raw e-mail | index | archive | help
This is a multi-part message in MIME format. --------------010708070805010203040003 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit All, I have a multi-threaded program that runs on 10.1-RELEASE-p5. It starts out with a reasonable footprint but there is obviously a resource leak as the program uses substantially more memory over time ... PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 51560 rdj 3 20 0 46676K 7500K select 1 0:00 0.00% dialyd ... 24h later ... PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND 51560 rdj 3 20 0 131M 27064K select 3 1:45 0.00% dialyd After a bit of debugging, I determined that it only happens when threads are created and then later destroyed. Valgrind thinks that the resources are being leaked from libthr itself ... First I tried joining the threads, then I tried detaching them using either pthread_detach or pthread_attr_setdetachstate. Whatever method I use, I still see resources leaked. I wrote a simple test program to demonstrate this ( attached ). Ten threads show 16,472 bytes leaked where 250 threads show 275,672 bytes. Here is an example of the valgrind output ... ==69449== 254,000 bytes in 250 blocks are still reachable in loss record 12 of 12 ==69449== at 0x1007221: calloc (in /usr/local/lib/valgrind/vgpreload_memcheck-amd64-freebsd.so) ==69449== by 0x16F8BD9: ??? (in /lib/libthr.so.3) ==69449== by 0x16F0E3D: pthread_create (in /lib/libthr.so.3) ==69449== by 0x4012B4: main (in /usr/home/mgrooms/threads/threads) ... and example of the summary shown for each run of the test program ... valgrind --leak-check=full ./threads 10 ==69310== Memcheck, a memory error detector ==69310== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==69310== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==69310== Command: ./threads 10 ==69310== main: waiting for condition to be signaled ... thread 5: signaling condition main: condition signaled, exiting ==69310== ==69310== HEAP SUMMARY: ==69310== in use at exit: 16,472 bytes in 30 blocks ==69310== total heap usage: 44 allocs, 14 frees, 17,368 bytes allocated ==69310== ==69310== LEAK SUMMARY: ==69310== definitely lost: 0 bytes in 0 blocks ==69310== indirectly lost: 0 bytes in 0 blocks ==69310== possibly lost: 0 bytes in 0 blocks ==69310== still reachable: 16,472 bytes in 30 blocks ==69310== suppressed: 0 bytes in 0 blocks ==69310== Reachable blocks (those to which a pointer was found) are not shown. ==69310== To see them, rerun with: --leak-check=full --show-reachable=yes ==69310== ==69310== For counts of detected and suppressed errors, rerun with: -v ==69310== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) valgrind --leak-check=full ./threads 100 ==69311== Memcheck, a memory error detector ==69311== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==69311== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==69311== Command: ./threads 100 ==69311== main: waiting for condition to be signaled ... thread 99: signaling condition main: condition signaled, exiting ==69311== ==69311== HEAP SUMMARY: ==69311== in use at exit: 113,672 bytes in 210 blocks ==69311== total heap usage: 314 allocs, 104 frees, 121,048 bytes allocated ==69311== ==69311== LEAK SUMMARY: ==69311== definitely lost: 0 bytes in 0 blocks ==69311== indirectly lost: 0 bytes in 0 blocks ==69311== possibly lost: 0 bytes in 0 blocks ==69311== still reachable: 113,672 bytes in 210 blocks ==69311== suppressed: 0 bytes in 0 blocks ==69311== Reachable blocks (those to which a pointer was found) are not shown. ==69311== To see them, rerun with: --leak-check=full --show-reachable=yes ==69311== ==69311== For counts of detected and suppressed errors, rerun with: -v ==69311== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) valgrind --leak-check=full ./threads 250 ==69315== Memcheck, a memory error detector ==69315== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==69315== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==69315== Command: ./threads 250 ==69315== main: waiting for condition to be signaled ... thread 241: signaling condition main: condition signaled, exiting ==69315== ==69315== HEAP SUMMARY: ==69315== in use at exit: 275,672 bytes in 510 blocks ==69315== total heap usage: 764 allocs, 254 frees, 293,848 bytes allocated ==69315== ==69315== LEAK SUMMARY: ==69315== definitely lost: 0 bytes in 0 blocks ==69315== indirectly lost: 0 bytes in 0 blocks ==69315== possibly lost: 0 bytes in 0 blocks ==69315== still reachable: 275,672 bytes in 510 blocks ==69315== suppressed: 0 bytes in 0 blocks ==69315== Reachable blocks (those to which a pointer was found) are not shown. ==69315== To see them, rerun with: --leak-check=full --show-reachable=yes ==69315== ==69315== For counts of detected and suppressed errors, rerun with: -v ==69315== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0) This doesn't seem right to me. Anyone have any ideas? Thanks, -Matthew --------------010708070805010203040003 Content-Type: text/plain; charset=windows-1252; name="threads.cpp" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="threads.cpp" Ly8NCi8vCW1haW4uY3BwOiB0aHJlYWQgdGVzdA0KLy8NCg0KI2luY2x1ZGUgPGVycm5vLmg+ DQojaW5jbHVkZSA8dW5pc3RkLmg+DQojaW5jbHVkZSA8c3RkaW8uaD4NCiNpbmNsdWRlIDxz dGRsaWIuaD4NCiNpbmNsdWRlIDxwdGhyZWFkLmg+DQojaW5jbHVkZSA8bWFjaGluZS9hdG9t aWMuaD4NCg0KZW51bSBUSFJFQURfVFlQRQ0Kew0KCVRIUkVBRF9UWVBFX0pPSU5FRCA9IDAs DQoJVEhSRUFEX1RZUEVfREVUQUNIRUQsDQoJVEhSRUFEX1RZUEVfREVUQUNIRURfQVRUUklC DQoNCn07DQoNCmludCB0aHJlYWRfaW5kZXggPSAwOw0KDQpwdGhyZWFkX211dGV4X3QgbXV0 ZXg7DQpwdGhyZWFkX2NvbmRfdCBjb25kaXRpb247DQoNCnZvaWQgcHRocmVhZF9lcnJvcigg Y29uc3QgY2hhciAqIG5hbWUsIGludCByZXN1bHQgKQ0Kew0KCXN3aXRjaCggcmVzdWx0ICkN Cgl7DQoJCWNhc2UgMDoNCgkJCXJldHVybjsNCg0KCQljYXNlIEVBR0FJTjoNCgkJCXByaW50 ZiggIiVzIGZhaWxlZCAoIEVBR0FJTjogUmVzb3VyY2UgdGVtcG9yYXJpbHkgdW5hdmFpbGFi bGUgKVxuIiwgbmFtZSApOw0KCQkJYnJlYWs7DQoJCWNhc2UgRVBFUk06DQoJCQlwcmludGYo ICIlcyBmYWlsZWQgKCBFUEVSTTogT3BlcmF0aW9uIG5vdCBwZXJtaXR0ZWQgKVxuIiwgbmFt ZSApOw0KCQkJYnJlYWs7DQoJCWNhc2UgRUlOVkFMOg0KCQkJcHJpbnRmKCAiJXMgZmFpbGVk ICggRUlOVkFMOiBJbnZhbGlkIGFyZ3VtZW50IClcbiIsIG5hbWUgKTsNCgkJCWJyZWFrOw0K CQljYXNlIEVOT01FTToNCgkJCXByaW50ZiggIiVzIGZhaWxlZCAoIEVOT01FTTogQ2Fubm90 IGFsbG9jYXRlIG1lbW9yeSApXG4iLCBuYW1lICk7DQoJCQlicmVhazsNCgkJY2FzZSBFREVB RExLOg0KCQkJcHJpbnRmKCAiJXMgZmFpbGVkICggRURFQURMSzogUmVzb3VyY2UgZGVhZGxv Y2sgYXZvaWRlZCApXG4iLCBuYW1lICk7DQoJCQlicmVhazsNCgkJZGVmYXVsdDoNCgkJCXBy aW50ZiggIiVzIGZhaWxlZCggdW5rbm93biBlcnJvciApXG4iLCBuYW1lICk7DQoJfQ0KDQoJ ZXhpdCggMSApOw0KfQ0KDQp2b2lkICogdGhyZWFkX3Byb2MoIHZvaWQgKiBhcmcgKQ0Kew0K CS8vDQoJLy8gaW5jcmVtZW50IHRoZSB0aHJlYWQgaW5kZXggdW5kZXIgbXV0ZXggcHJvdGVj dGlvbg0KCS8vDQoNCglpbnQgcmVzdWx0ID0gcHRocmVhZF9tdXRleF9sb2NrKCAmbXV0ZXgg KTsNCglpZiggcmVzdWx0ICE9IDAgKQ0KCQlwdGhyZWFkX2Vycm9yKCAidGhyZWFkOiBwdGhy ZWFkX211dGV4X2xvY2siLCByZXN1bHQgKTsNCg0KCWludCBsb2NhbF9pbmRleCA9IHRocmVh ZF9pbmRleCsrOw0KCXByaW50ZiggInRocmVhZCAlaTogY3JlYXRlZFxuIiwgbG9jYWxfaW5k ZXggKTsNCg0KCXJlc3VsdCA9IHB0aHJlYWRfbXV0ZXhfdW5sb2NrKCAmbXV0ZXggKTsNCglp ZiggcmVzdWx0ICE9IDAgKQ0KCQlwdGhyZWFkX2Vycm9yKCAidGhyZWFkOiBwdGhyZWFkX211 dGV4X3VubG9jayIsIHJlc3VsdCApOw0KDQoJLy8NCgkvLyBzbGVlcCBmb3IgYSBiaXQNCgkv Lw0KDQoJc2xlZXAoIDMgKTsNCg0KCS8vDQoJLy8gZGVjcmVtZW50IHRoZSB0aHJlYWQgaW5k ZXggdW5kZXIgbXV0ZXggcHJvdGVjdGlvbg0KCS8vIGFuZCBzaWduYWwgdGhlIGNvbmRpdGlv biBpZiB0aGUgdGhyZWFkIGluZGV4IGhhcw0KCS8vIHJlYWNoZWQgemVybw0KCS8vDQoNCgly ZXN1bHQgPSBwdGhyZWFkX211dGV4X2xvY2soICZtdXRleCApOw0KCWlmKCByZXN1bHQgIT0g MCApDQoJCXB0aHJlYWRfZXJyb3IoICJ0aHJlYWQ6IHB0aHJlYWRfbXV0ZXhfbG9jayIsIHJl c3VsdCApOw0KDQoJcHJpbnRmKCAidGhyZWFkICVpOiBleGl0aW5nXG4iLCBsb2NhbF9pbmRl eCApOw0KCWlmKCAtLXRocmVhZF9pbmRleCA9PSAwICkNCgl7DQoJCXByaW50ZiggInRocmVh ZCAlaTogc2lnbmFsaW5nIGNvbmRpdGlvblxuIiwgbG9jYWxfaW5kZXggKTsNCgkJcmVzdWx0 ID0gcHRocmVhZF9jb25kX3NpZ25hbCggJmNvbmRpdGlvbiApOw0KCQlpZiggcmVzdWx0ID09 IDAgKQ0KCQkJcHRocmVhZF9lcnJvciggInRocmVhZDogcHRocmVhZF9jb25kX3NpZ25hbCIs IHJlc3VsdCApOw0KCX0NCg0KCXJlc3VsdCA9IHB0aHJlYWRfbXV0ZXhfdW5sb2NrKCAmbXV0 ZXggKTsNCglpZiggcmVzdWx0ICE9IDAgKQ0KCQlwdGhyZWFkX2Vycm9yKCAidGhyZWFkOiBw dGhyZWFkX211dGV4X3VubG9jayIsIHJlc3VsdCApOw0KDQoJcmV0dXJuIE5VTEw7DQp9DQoN CmludCBtYWluKCBpbnQgYXJnYywgY2hhciAqIGFyZ3ZbXSApDQp7DQoJLy8NCgkvLyB0eXBl IG9mIHRocmVhZCBjbGVhbnVwDQoJLy8NCgkvLyBUSFJFQURfVFlQRV9KT0lORUQ6IHRocmVh ZCB3aWxsIGJlIGpvaW5lZCB3aXRoIHB0aHJlYWRfam9pbg0KCS8vIFRIUkVBRF9UWVBFX0RF VEFDSEVEOiB0aHJlYWQgd2lsbCBiZSBkZXRhY2hlZCB1c2luZyBwdGhyZWFkX2RldGFjaA0K CS8vIFRIUkVBRF9UWVBFX0RFVEFDSEVEX0FUVFJJQjogdGhyZWFkIHdpbGwgYmUgZGV0YWNo ZWQgdXNpbmcgcHRocmVhZF9hdHRyX3NldGRldGFjaHN0YXRlDQoJLy8NCg0KCVRIUkVBRF9U WVBFIHRocmVhZF90eXBlID0gVEhSRUFEX1RZUEVfREVUQUNIRUQ7DQoNCgkvLw0KCS8vIHJl YWQgdGhyZWFkIGNvdW50DQoJLy8NCg0KCWludCB0aHJlYWRfY291bnQgPSAxMDsNCglpZigg YXJnYyA9PSAyICkNCgkJdGhyZWFkX2NvdW50ID0gYXRvaSggYXJndlsxXSApOw0KDQoJaWYo IHRocmVhZF9jb3VudCA8IDEgKQ0KCXsNCgkJcHJpbnRmKCAidXNhZ2U6IHRocmVhZHMgW3Ro cmVhZGNvdW50XVxuIiApOw0KCQlleGl0KCAyICk7DQoJfQ0KDQoJLy8NCgkvLyBjcmVhdGUg b3VyIHB0aHJlYWQgbXV0ZXgNCgkvLw0KDQoJaW50IHJlc3VsdCA9IHB0aHJlYWRfbXV0ZXhf aW5pdCggJm11dGV4LCBOVUxMICk7DQoJaWYoIHJlc3VsdCAhPSAwICkNCgkJcHRocmVhZF9l cnJvciggIm1haW46IHB0aHJlYWRfbXV0ZXhfaW5pdCIsIHJlc3VsdCApOw0KDQoJLy8NCgkv LyBjcmVhdGUgb3VyIHB0aHJlYWQgY29uZGl0aW9uDQoJLy8NCg0KCXJlc3VsdCA9IHB0aHJl YWRfY29uZF9pbml0KCAmY29uZGl0aW9uLCBOVUxMICk7DQoJaWYoIHJlc3VsdCAhPSAwICkN CgkJcHRocmVhZF9lcnJvciggIm1haW46IHB0aHJlYWRfY29uZF9pbml0IiwgcmVzdWx0ICk7 DQoNCgkvLw0KCS8vIGFsb2NhdGUgc3RvcmFnZSBmb3Igb3VyIHB0aHJlYWQgaGFuZGxlcw0K CS8vDQoNCglwdGhyZWFkX3QgKiBwdGhyZWFkcyA9IG5ldyBwdGhyZWFkX3RbIHRocmVhZF9j b3VudCBdOw0KCWlmKCBwdGhyZWFkcyA9PSBOVUxMICkNCgl7DQoJCXByaW50ZiggImZhaWxl ZCB0byBhbGxvY2F0ZSB0aHJlYWQgaGFuZGxlIHN0b3JhZ2VcbiIgKTsNCgkJcmV0dXJuIC0x Ow0KCX0NCg0KCS8vDQoJLy8gbG9vcCBmb3IgbnVtYmVyIG9mIHRocmVhZHMgcmVxdWVzdGVk DQoJLy8NCg0KCWZvciggaW50IGluZGV4ID0gMDsgaW5kZXggPCB0aHJlYWRfY291bnQ7IGlu ZGV4KysgKQ0KCXsNCgkJLy8NCgkJLy8gaW5pdGlhbGl6ZSBkZWZhdWx0IHRocmVhZCBhdHRy aWJ1dGVzDQoJCS8vDQoNCgkJcHRocmVhZF9hdHRyX3QgYXR0cjsNCgkJcmVzdWx0ID0gcHRo cmVhZF9hdHRyX2luaXQoICZhdHRyICk7DQoNCgkJLy8NCgkJLy8gc2V0IGRldGFjaCB0aHJl YWQgYXR0cmlidXRlIGlmIG5lY2Nlc3NhcnkNCgkJLy8NCg0KCQlpZiggdGhyZWFkX3R5cGUg PT0gVEhSRUFEX1RZUEVfREVUQUNIRURfQVRUUklCICkNCgkJCXJlc3VsdCA9IHB0aHJlYWRf YXR0cl9zZXRkZXRhY2hzdGF0ZSggJmF0dHIsIFBUSFJFQURfQ1JFQVRFX0RFVEFDSEVEICk7 DQoNCgkJLy8NCgkJLy8gY3JlYXRlIGEgbmV3IHRocmVhZA0KCQkvLw0KDQoJCXJlc3VsdCA9 IHB0aHJlYWRfY3JlYXRlKCAmcHRocmVhZHNbIGluZGV4IF0sICZhdHRyLCAmdGhyZWFkX3By b2MsIE5VTEwgKTsNCgkJaWYoIHJlc3VsdCAhPSAwICkNCgkJCXB0aHJlYWRfZXJyb3IoICJt YWluOiBwdGhyZWFkX2NyZWF0ZSIsIHJlc3VsdCApOw0KDQoJCS8vDQoJCS8vIGNsZWFuIHVw IHRocmVhZCBhdHRyaWJ1dGVzDQoJCS8vDQoNCgkJcHRocmVhZF9hdHRyX2Rlc3Ryb3koICZh dHRyICk7DQoNCgkJLy8NCgkJLy8gZXhwbGljaXRseSBkZXRhY2ggdGhlIHRocmVhZCBpZiBu ZWNjZXNzYXJ5DQoJCS8vDQoNCgkJaWYoIHRocmVhZF90eXBlID09IFRIUkVBRF9UWVBFX0RF VEFDSEVEICkNCgkJew0KCQkJcmVzdWx0ID0gcHRocmVhZF9kZXRhY2goIHB0aHJlYWRzWyBp bmRleCBdICk7DQoJCQlpZiggcmVzdWx0ICE9IDAgKQ0KCQkJCXB0aHJlYWRfZXJyb3IoICJt YWluOiBwdGhyZWFkX2RldGFjaCIsIHJlc3VsdCApOw0KCQl9DQoJfQ0KDQoJLy8NCgkvLyB3 YWl0IGZvciBvdXIgY29uZGl0aW9uIHRvIGJlIHNpZ25hbGVkDQoJLy8NCg0KCXByaW50Zigg Im1haW46IHdhaXRpbmcgZm9yIGNvbmRpdGlvbiB0byBiZSBzaWduYWxlZCAuLi5cbiIgKTsN Cg0KCXJlc3VsdCA9IHB0aHJlYWRfbXV0ZXhfbG9jayggJm11dGV4ICk7DQoJaWYoIHJlc3Vs dCAhPSAwICkNCgkJcHRocmVhZF9lcnJvciggIm1haW46IHB0aHJlYWRfbXV0ZXhfbG9jayIs IHJlc3VsdCApOw0KDQoJcmVzdWx0ID0gcHRocmVhZF9jb25kX3dhaXQoICZjb25kaXRpb24s ICZtdXRleCApOw0KCWlmKCByZXN1bHQgIT0gMCApDQoJCXB0aHJlYWRfZXJyb3IoICJtYWlu OiBwdGhyZWFkX2NvbmRfd2FpdCIsIHJlc3VsdCApOw0KDQoJcHJpbnRmKCAibWFpbjogY29u ZGl0aW9uIHNpZ25hbGVkLCBleGl0aW5nXG4iICk7DQoNCglyZXN1bHQgPSBwdGhyZWFkX211 dGV4X3VubG9jayggJm11dGV4ICk7DQoJaWYoIHJlc3VsdCAhPSAwICkNCgkJcHRocmVhZF9l cnJvciggIm1haW46IHB0aHJlYWRfbXV0ZXhfdW5sb2NrIiwgcmVzdWx0ICk7DQoNCgkvLw0K CS8vIGpvaW4gYWxsIHRocmVhZHMgaWYgbmVjY2Vzc2FyeQ0KCS8vDQoNCglpZiggdGhyZWFk X3R5cGUgPT0gVEhSRUFEX1RZUEVfSk9JTkVEICkNCgl7DQoJCWZvciggaW50IGluZGV4ID0g MDsgaW5kZXggPCB0aHJlYWRfY291bnQ7IGluZGV4KysgKQ0KCQl7DQoJCQlyZXN1bHQgPSBw dGhyZWFkX2pvaW4oIHB0aHJlYWRzWyBpbmRleCBdLCBOVUxMICk7DQoJCQlpZiggcmVzdWx0 ICE9IDAgKQ0KCQkJCXB0aHJlYWRfZXJyb3IoICJtYWluOiBwdGhyZWFkX2pvaW4iLCByZXN1 bHQgKTsNCgkJfQ0KDQoJCXByaW50ZiggImFsbCB0aHJlYWRzIGpvaW5lZFxuIiApOw0KCX0N Cg0KCS8vDQoJLy8gY2xlYW4gdXANCgkvLw0KDQoJZGVsZXRlIFtdIHB0aHJlYWRzOw0KDQoJ cHRocmVhZF9jb25kX2Rlc3Ryb3koICZjb25kaXRpb24gKTsNCglwdGhyZWFkX211dGV4X2Rl c3Ryb3koICZtdXRleCApOw0KDQoJcmV0dXJuIDA7DQp9DQo= --------------010708070805010203040003--
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?54E65E05.2040101>