From owner-freebsd-stable@FreeBSD.ORG Fri Feb 20 15:05:04 2015 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 5F124826 for ; Fri, 20 Feb 2015 15:05:04 +0000 (UTC) Received: from mx1.shrew.net (mx1.shrew.net [38.97.5.131]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 14DFF88B for ; Fri, 20 Feb 2015 15:05:03 +0000 (UTC) Received: from mail.shrew.net (mail.shrew.prv [10.24.10.20]) by mx1.shrew.net (8.14.7/8.14.7) with ESMTP id t1KF3Z76018307 for ; Fri, 20 Feb 2015 09:03:35 -0600 (CST) (envelope-from mgrooms@shrew.net) Received: from [10.22.200.30] (cpe-72-177-96-36.austin.res.rr.com [72.177.96.36]) by mail.shrew.net (Postfix) with ESMTPSA id 721FC18A86A for ; Fri, 20 Feb 2015 09:03:30 -0600 (CST) Message-ID: <54E74D17.6020209@shrew.net> Date: Fri, 20 Feb 2015 09:04:55 -0600 From: Matthew Grooms User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:31.0) Gecko/20100101 Thunderbird/31.4.0 MIME-Version: 1.0 To: freebsd-stable@freebsd.org Subject: Re: pthread leaky with resources? References: <54E65E05.2040101@shrew.net> <20150220100446.GL34251@kib.kiev.ua> In-Reply-To: <20150220100446.GL34251@kib.kiev.ua> Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.4.3 (mx1.shrew.net [10.24.10.10]); Fri, 20 Feb 2015 09:03:35 -0600 (CST) X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Feb 2015 15:05:04 -0000 On 2/20/2015 4:04 AM, Konstantin Belousov wrote: > On Thu, Feb 19, 2015 at 04:04:53PM -0600, Matthew Grooms wrote: >> All, >> >> I have a multi-threaded program that runs on 10.1-RELEASE-p5. It starts >> out with a reasonable footprint but there is obviously a resource leak >> as the program uses substantially more memory over time ... >> >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 51560 rdj 3 20 0 46676K 7500K select 1 0:00 0.00% dialyd >> >> ... 24h later ... >> >> PID USERNAME THR PRI NICE SIZE RES STATE C TIME WCPU COMMAND >> 51560 rdj 3 20 0 131M 27064K select 3 1:45 0.00% dialyd >> >> After a bit of debugging, I determined that it only happens when threads >> are created and then later destroyed. Valgrind thinks that the resources >> are being leaked from libthr itself ... > Threading library caches thread structures and some related objects. > This is needed to correctly handle kernel notifications about threads > exit and to avoid thread id reuse, besides usual argument for using cache > to improve creation speed. > > Also, the thread stacks are cached, each stack being 2MB probably accounts > for most of the memory usage columns in the top output above. Konstantin, Thanks for the reply. That seems reasonable. But surely these resources need to be reclaimed at some point after a thread gracefully exits. Otherwise any software that creates short lived threads will eventually run out of system resources and periodically require a restart. The test program I included does nothing but create threads that gracefully exit. Do you have another explanation as to why a program would indefinitely grow in size that way? Thanks, -Matthew