From owner-freebsd-threads@FreeBSD.ORG Wed Aug 30 15:19:23 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2F59B16A4E0; Wed, 30 Aug 2006 15:19:23 +0000 (UTC) (envelope-from eischen@vigrid.com) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9FD2B43D46; Wed, 30 Aug 2006 15:19:22 +0000 (GMT) (envelope-from eischen@vigrid.com) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.8/8.13.8/NETPLEX) with ESMTP id k7UFJGw1016129; Wed, 30 Aug 2006 11:19:16 -0400 (EDT) Date: Wed, 30 Aug 2006 11:19:16 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Andriy Gapon In-Reply-To: <44F59F3C.3040303@icyb.net.ua> Message-ID: References: <44F59F3C.3040303@icyb.net.ua> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Greylist: Message whitelisted by DRAC access database, not delayed by milter-greylist-2.0.2 (mail.ntplx.net [204.213.176.10]); Wed, 30 Aug 2006 11:19:17 -0400 (EDT) X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: ace-users@cse.wustl.edu, Sergey Matveychuk , freebsd-threads@freebsd.org Subject: Re: ace/freebsd: THR_NEW_LWP problem with libpthread/pthread_setconcurrency X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2006 15:19:23 -0000 On Wed, 30 Aug 2006, Andriy Gapon wrote: > > ACE VERSION: > 5.5.1 (from FreeBSD port: ace-5.5.1) > > HOST MACHINE and OPERATING SYSTEM: > FreeBSD 6.1-RELEASE-p2 i386 (uniprocessor) > > COMPILER NAME AND VERSION (AND PATCHLEVEL): > gcc version 3.4.4 [FreeBSD] 20050518 (system compiler) > libpthread is used as threading library > > AREA/CLASS/EXAMPLE AFFECTED: > ACE threads > > DOES THE PROBLEM AFFECT: > EXECUTION - this is a run-time problem > > SYNOPSIS: > Can not create threads with THR_NEW_LWP flag. > > DESCRIPTION: > ACE_Thread_Manager::spawn(..., THR_NEW_LWP) fails with EAGAIN. It seems > that this happens because when creating such threads ACE calls > pthread_setconcurrency with incrementally increasing concurrency levels. > pthread_setconcurrency of libpthread in turn seems to call kse_create() > which fails with EPROCLIM. As I understand this happens because kernel > does not allow to have more KSE-s per process as there are processors. pthread_setconcurrency() should return EAGAIN, not EPROCLIM. kse_create() may set errno to EPROCLIM, but I don't see how the code in libpthread can return anything other than EAGAIN if kse_create() fails. POSIX states that pthread_setconcurrency() can return EAGAIN, so I think ACE is at fault if it is not handling that properly. I would also think that THR_NEW_LWP should not increment the concurrency, but create a new system scope thread since that seems more in the spirit of "spawning a new thread with a new LWP". > This problem is quite noticeable with ACE because THR_NEW_LWP is present > in default flags in many places. > > REPEAT BY: > Running any ACE-based program that spawns threads with THR_NEW_LWP and > linked to libpthead. > > SAMPLE FIX/WORKAROUND: > Straightforward workaround is to explicitly pass thread flags everywhere > and not use THR_NEW_LWP. Maybe this could be patched in ACE sources as a > part of FreeBSD port patch step. > IMHO, better fixes would be: > 1. make ACE_Thread_Manager::spawn() more robust with respect to > pthread_setconcurrency() failing with EAGAIN. > 2. make FreeBSD libpthread dumb-happy in pthread_setconcurrency(), i.e > pretend to always succeed. AFAIK, POSIX leaves it up to implementations > to interpret concurrency levels, so making any level be equivalent to > default level should be OK. Yes to 1, No to 2, and the third option should be to treat THR_NEW_LWP as creating a new system scope thread. -- DE