From owner-freebsd-stable@FreeBSD.ORG  Fri Feb 20 10:04:53 2015
Return-Path: <owner-freebsd-stable@FreeBSD.ORG>
Delivered-To: freebsd-stable@freebsd.org
Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115])
 (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits))
 (No client certificate requested)
 by hub.freebsd.org (Postfix) with ESMTPS id 3E2AF813
 for <freebsd-stable@freebsd.org>; Fri, 20 Feb 2015 10:04:53 +0000 (UTC)
Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1])
 (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
 (Client did not present a certificate)
 by mx1.freebsd.org (Postfix) with ESMTPS id A6E5012A
 for <freebsd-stable@freebsd.org>; Fri, 20 Feb 2015 10:04:52 +0000 (UTC)
Received: from tom.home (kostik@localhost [127.0.0.1])
 by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t1KA4ls9069306
 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO);
 Fri, 20 Feb 2015 12:04:47 +0200 (EET)
 (envelope-from kostikbel@gmail.com)
DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t1KA4ls9069306
Received: (from kostik@localhost)
 by tom.home (8.14.9/8.14.9/Submit) id t1KA4lii069305;
 Fri, 20 Feb 2015 12:04:47 +0200 (EET)
 (envelope-from kostikbel@gmail.com)
X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com
 using -f
Date: Fri, 20 Feb 2015 12:04:47 +0200
From: Konstantin Belousov <kostikbel@gmail.com>
To: Matthew Grooms <mgrooms@shrew.net>
Subject: Re: pthread leaky with resources?
Message-ID: <20150220100446.GL34251@kib.kiev.ua>
References: <54E65E05.2040101@shrew.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <54E65E05.2040101@shrew.net>
User-Agent: Mutt/1.5.23 (2014-03-12)
X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00,
 DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no
 autolearn_force=no version=3.4.0
X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home
Cc: freebsd-stable@freebsd.org
X-BeenThere: freebsd-stable@freebsd.org
X-Mailman-Version: 2.1.18-1
Precedence: list
List-Id: Production branch of FreeBSD source code <freebsd-stable.freebsd.org>
List-Unsubscribe: <http://lists.freebsd.org/mailman/options/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=unsubscribe>
List-Archive: <http://lists.freebsd.org/pipermail/freebsd-stable/>
List-Post: <mailto:freebsd-stable@freebsd.org>
List-Help: <mailto:freebsd-stable-request@freebsd.org?subject=help>
List-Subscribe: <http://lists.freebsd.org/mailman/listinfo/freebsd-stable>,
 <mailto:freebsd-stable-request@freebsd.org?subject=subscribe>
X-List-Received-Date: Fri, 20 Feb 2015 10:04:53 -0000

On Thu, Feb 19, 2015 at 04:04:53PM -0600, Matthew Grooms wrote:
> All,
> 
> I have a multi-threaded program that runs on 10.1-RELEASE-p5. It starts 
> out with a reasonable footprint but there is obviously a resource leak 
> as the program uses substantially more memory over time ...
> 
>    PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME WCPU COMMAND
> 51560 rdj           3  20    0 46676K  7500K select  1   0:00 0.00% dialyd
> 
> ... 24h later ...
> 
>    PID USERNAME    THR PRI NICE   SIZE    RES STATE   C   TIME WCPU COMMAND
> 51560 rdj           3  20    0   131M 27064K select  3   1:45 0.00% dialyd
> 
> After a bit of debugging, I determined that it only happens when threads 
> are created and then later destroyed. Valgrind thinks that the resources 
> are being leaked from libthr itself ...
Threading library caches thread structures and some related objects.
This is needed to correctly handle kernel notifications about threads
exit and to avoid thread id reuse, besides usual argument for using cache
to improve creation speed.

Also, the thread stacks are cached, each stack being 2MB probably accounts
for most of the memory usage columns in the top output above.