From owner-freebsd-threads@FreeBSD.ORG Wed Aug 30 15:56:11 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 10B9B16A4DF; Wed, 30 Aug 2006 15:56:11 +0000 (UTC) (envelope-from avg@icyb.net.ua) Received: from citadel.icyb.net.ua (citadel.icyb.net.ua [212.40.38.140]) by mx1.FreeBSD.org (Postfix) with ESMTP id C96D143D58; Wed, 30 Aug 2006 15:56:09 +0000 (GMT) (envelope-from avg@icyb.net.ua) Received: from [212.40.38.87] (oddity-e.topspin.kiev.ua [212.40.38.87]) by citadel.icyb.net.ua (8.8.8p3/ICyb-2.3exp) with ESMTP id SAA24866; Wed, 30 Aug 2006 18:56:02 +0300 (EEST) (envelope-from avg@icyb.net.ua) Message-ID: <44F5B512.5060708@icyb.net.ua> Date: Wed, 30 Aug 2006 18:56:02 +0300 From: Andriy Gapon User-Agent: Thunderbird 1.5.0.5 (X11/20060801) MIME-Version: 1.0 To: Daniel Eischen References: <44F59F3C.3040303@icyb.net.ua> In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1 Content-Transfer-Encoding: 7bit Cc: ace-users@cse.wustl.edu, Sergey Matveychuk , freebsd-threads@freebsd.org Subject: Re: ace/freebsd: THR_NEW_LWP problem with libpthread/pthread_setconcurrency X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Aug 2006 15:56:11 -0000 on 30/08/2006 18:19 Daniel Eischen said the following: > On Wed, 30 Aug 2006, Andriy Gapon wrote: >> DESCRIPTION: >> ACE_Thread_Manager::spawn(..., THR_NEW_LWP) fails with EAGAIN. It seems >> that this happens because when creating such threads ACE calls >> pthread_setconcurrency with incrementally increasing concurrency levels. >> pthread_setconcurrency of libpthread in turn seems to call kse_create() >> which fails with EPROCLIM. As I understand this happens because kernel >> does not allow to have more KSE-s per process as there are processors. > > pthread_setconcurrency() should return EAGAIN, not EPROCLIM. kse_create() > may set errno to EPROCLIM, but I don't see how the code in libpthread > can return anything other than EAGAIN if kse_create() fails. That's exactly what I actually was trying to say, sorry for unclear wording. > POSIX states that pthread_setconcurrency() can return EAGAIN, so I > think ACE is at fault if it is not handling that properly. I > would also think that THR_NEW_LWP should not increment the > concurrency, but create a new system scope thread since that > seems more in the spirit of "spawning a new thread with a new > LWP". Yes, indeed, it seems like a good approach, at least for FreeBSD+libpthread. On the other hand, ACE also defines the following thread flags: THR_SCOPE_SYSTEM, THR_SCOPE_PROCESS. The whole THR_NEW_LWP seems to be inspired by Solaris threads (of old?): http://docs.sun.com/app/docs/doc/805-5080/6j4q7emhv?a=view THR_NEW_LWP--Increases the concurrency level for unbound threads by one. The effect is similar to incrementing concurrency by one with thr_setconcurrency(3T), although THR_NEW_LWP does not affect the level set through the thr_setconcurrency() function. Typically, THR_NEW_LWP adds a new LWP to the pool of LWPs running unbound threads. >> SAMPLE FIX/WORKAROUND: >> Straightforward workaround is to explicitly pass thread flags everywhere >> and not use THR_NEW_LWP. Maybe this could be patched in ACE sources as a >> part of FreeBSD port patch step. >> IMHO, better fixes would be: >> 1. make ACE_Thread_Manager::spawn() more robust with respect to >> pthread_setconcurrency() failing with EAGAIN. >> 2. make FreeBSD libpthread dumb-happy in pthread_setconcurrency(), i.e >> pretend to always succeed. AFAIK, POSIX leaves it up to implementations >> to interpret concurrency levels, so making any level be equivalent to >> default level should be OK. > > Yes to 1, No to 2, and the third option should be to treat THR_NEW_LWP > as creating a new system scope thread. Yes, I also feel that ACE should not try to mimic Solaris threads THR_NEW_LWP and instead treat it as an alias for THR_SCOPE_SYSTEM (or no-flag). At least on FreeBSD. Whether this is to be done through changes in ACE itself or through a patch in the FreeBSD port, I am not sure. -- Andriy Gapon