From owner-freebsd-threads@FreeBSD.ORG Mon Aug 7 14:19:29 2006 Return-Path: X-Original-To: threads@FreeBSD.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C39F716A503 for ; Mon, 7 Aug 2006 14:19:29 +0000 (UTC) (envelope-from rwatson@FreeBSD.org) Received: from cyrus.watson.org (cyrus.watson.org [209.31.154.42]) by mx1.FreeBSD.org (Postfix) with ESMTP id 60E6F43D45 for ; Mon, 7 Aug 2006 14:19:29 +0000 (GMT) (envelope-from rwatson@FreeBSD.org) Received: from fledge.watson.org (fledge.watson.org [209.31.154.41]) by cyrus.watson.org (Postfix) with ESMTP id F034E46D0E for ; Mon, 7 Aug 2006 10:19:28 -0400 (EDT) Date: Mon, 7 Aug 2006 15:19:28 +0100 (BST) From: Robert Watson X-X-Sender: robert@fledge.watson.org To: threads@FreeBSD.org Message-ID: <20060807151829.T45647@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed Cc: Subject: GHC 6.4.3 on FreeBSD (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2006 14:19:29 -0000 I've been talking to the Haskell folk about problems they've had getting their threaded runtime running on FreeBSD. Here's one of their problem reports. Since I'm not up on how threads and signals are supposed to act, I can't confirm the below is a bug, of course, but figured you (experts) would be able to respond better. Robert N M Watson Computer Laboratory University of Cambridge ---------- Forwarded message ---------- Date: Mon, 7 Aug 2006 15:12:19 +0100 From: Simon Marlow To: Volker Stolz , Robert Watson , Kip Macy Cc: Simon Peyton-Jones , Tony Finch , haskell@freebsd.org, glasgow-haskell-users@haskell.org Subject: GHC 6.4.3 on FreeBSD An update on the GHC/FreeBSD front: I didn't manage to reproduce the reported threading bugs on a UP, will be trying on a MP shortly. However, I did discover one odd case that libpthread doesn't appear to handle properly, but libthr does. This arose from a test in GHC's test suite, but I've transliterated the code from Haskell to C: $ cat thr1.c #include #include #include int main(int argc, char *argv[]) { sigset_t s; sigemptyset (&s); sigaddset(&s, SIGUSR1); sigprocmask(SIG_BLOCK, &s, NULL); pthread_kill(pthread_self(), SIGUSR1); sigpending(&s); printf("%x\n", s); exit(0); } $ gcc -pthread thr1.c $ ./a.out 0 $ gcc -lthr thr1.c $ ./a.out 20000000 $ This might (or might not) be related to the other threading issues with GHC on FreeBSD. Do you think we should link with -lthr by default on FreeBSD? This would be a trivial change to make, and given that GHC has its own lightweight threading runtime, we probably aren't getting much from having the KSE-based threading implementation anyway. Cheers, Simon From owner-freebsd-threads@FreeBSD.ORG Mon Aug 7 14:51:24 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3B4D816A4DD; Mon, 7 Aug 2006 14:51:24 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5488343D6E; Mon, 7 Aug 2006 14:51:23 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from [10.0.1.106] (nat-outside.atlanta.corp.yahoo.com [63.172.193.57]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id k77EpMU6035301; Mon, 7 Aug 2006 10:51:22 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: David Xu Date: Mon, 7 Aug 2006 10:33:11 -0400 User-Agent: KMail/1.9.1 References: <20060804140657.GK4498@obiwan.tataz.chchile.org> <200608041134.43979.john@baldwin.cx> <200608050806.24502.davidxu@freebsd.org> In-Reply-To: <200608050806.24502.davidxu@freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200608071033.12277.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [66.23.211.162]); Mon, 07 Aug 2006 10:51:22 -0400 (EDT) X-Virus-Scanned: ClamAV 0.87.1/1639/Mon Aug 7 09:34:09 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.2 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on server.baldwin.cx Cc: freebsd-threads@freebsd.org Subject: Re: system scope vs. process scope X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2006 14:51:24 -0000 On Friday 04 August 2006 20:06, David Xu wrote: > On Friday 04 August 2006 23:34, John Baldwin wrote: > > > Suppose you have a system with 2 processes. One process is single-threaded > > and is CPU bound. The other process has 2 threads both of which are also > > CPU bound. If the threads in the second process are PTHREAD_SCOPE_SYSTEM, > > then each thread will get 33% of the CPU. If the threads in the second > > process are PTHREAD_SCOPE_PROCESS, then the each process will get 50% of > > the CPU. The second process will then use its own algorithm to split it's > > 50% of the CPU up between it's two threads. If it divides it evenly, then > > each of its' threads will end up with 25% of the CPU whereas the thread for > > the first process has 50% of the CPU. The idea for this is that if you > > have a system with several single-threaded processes and one process with > > 1000 threads, you don't want that process to starve out all the other > > processes. > > > I can make my application fork 800 processes and starve out all the other > user. Yes, but then you aren't sharing the same address space, which your application might need if it wants to do threading. :) And, your statement aside, I believe the above is why the difference exists. Your statement is more about the usefulness of the distinction, not about the justification for when it was created. -- John Baldwin From owner-freebsd-threads@FreeBSD.ORG Mon Aug 7 17:15:23 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B129F16A4DF; Mon, 7 Aug 2006 17:15:23 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3E4B343D45; Mon, 7 Aug 2006 17:15:23 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.7/8.13.7/NETPLEX) with ESMTP id k77HFMFD020878; Mon, 7 Aug 2006 13:15:22 -0400 (EDT) Date: Mon, 7 Aug 2006 13:15:21 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Robert Watson In-Reply-To: <20060807151829.T45647@fledge.watson.org> Message-ID: References: <20060807151829.T45647@fledge.watson.org> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: threads@freebsd.org Subject: Re: GHC 6.4.3 on FreeBSD (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2006 17:15:23 -0000 On Mon, 7 Aug 2006, Robert Watson wrote: > > I've been talking to the Haskell folk about problems they've had getting > their threaded runtime running on FreeBSD. Here's one of their problem > reports. Since I'm not up on how threads and signals are supposed to act, I > can't confirm the below is a bug, of course, but figured you (experts) would > be able to respond better. Try this patch. -- DE Index: thread/thr_sig.c =================================================================== RCS file: /opt/FreeBSD/cvs/src/lib/libpthread/thread/thr_sig.c,v retrieving revision 1.85 diff -u -r1.85 thr_sig.c --- thread/thr_sig.c 9 Jun 2006 14:23:40 -0000 1.85 +++ thread/thr_sig.c 7 Aug 2006 16:37:50 -0000 @@ -1095,6 +1095,11 @@ struct pthread *curthread = _get_curthread(); struct kse_mailbox *kmbx; + if ((curthread->kse->k_flags & KF_STARTED) == 0) { + (void)__sys_kill(getpid(), sig); + return; + } + if (pthread->attr.flags & PTHREAD_SCOPE_SYSTEM) { kse_thr_interrupt(&pthread->tcb->tcb_tmbx, KSE_INTR_SENDSIG, sig); return; From owner-freebsd-threads@FreeBSD.ORG Mon Aug 7 21:39:31 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0911416A4DA for ; Mon, 7 Aug 2006 21:39:31 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (smtp5-g19.free.fr [212.27.42.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3894C43D46 for ; Mon, 7 Aug 2006 21:39:20 +0000 (GMT) (envelope-from tataz@tataz.chchile.org) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp5-g19.free.fr (Postfix) with ESMTP id D930724BD0; Mon, 7 Aug 2006 23:39:19 +0200 (CEST) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id 1F2949B458; Mon, 7 Aug 2006 21:39:46 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id 03121405F; Mon, 7 Aug 2006 23:39:45 +0200 (CEST) Date: Mon, 7 Aug 2006 23:39:45 +0200 From: Jeremie Le Hen To: John Baldwin Message-ID: <20060807213945.GQ4498@obiwan.tataz.chchile.org> References: <20060804140657.GK4498@obiwan.tataz.chchile.org> <200608041134.43979.john@baldwin.cx> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <200608041134.43979.john@baldwin.cx> User-Agent: Mutt/1.5.11 Cc: freebsd-threads@freebsd.org Subject: Re: [fbsd] Re: system scope vs. process scope X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 07 Aug 2006 21:39:31 -0000 Hi John, On Fri, Aug 04, 2006 at 11:34:43AM -0400, John Baldwin wrote: > On Friday 04 August 2006 10:06, Jeremie Le Hen wrote: > > Hi, > > > > occasionally, I saw environnement variables LIBPTHREAD_SYSTEM_SCOPE and > > LIBPTHREAD_PROCESS_SCOPE mentionned hither and thither. I grep(1)'ed > > through the source tree for some documentation, but I wasn't able to > > find any. Read the source Luke... > > > > It seems that the PTHREAD_SCOPE_SYSTEM thread attribute is the default. > > Intuitivelely, I would say that setting the PTHREAD_SCOPE_PROCESS > > attribute creates a new process with its address space being shared with > > the other userland threads (a la Linux) whereas the default behaviour is > > to create a new kernel thread. > > > > Am I right ? What are the pros and cons of either methods ? > > That's not what it means. :) It has to do with scheduling. A > PTHREAD_SCOPE_SYSTEM will compete for the CPU with other "system" threads > (aka kernel threads). That is, each userland thread has a direct kernel > thread that is visible to the system (kernel). A PTHREAD_SCOPE_PROCESS > thread competes for the CPU just within the current process. That is, each > group of PTHREAD_SCOPE_PROCESS thread's all share (in theory) a single kernel > thread that competes for CPU with other system threads. An example might > explain the theory more completely. > > Suppose you have a system with 2 processes. One process is single-threaded > and is CPU bound. The other process has 2 threads both of which are also CPU > bound. If the threads in the second process are PTHREAD_SCOPE_SYSTEM, then > each thread will get 33% of the CPU. If the threads in the second process > are PTHREAD_SCOPE_PROCESS, then the each process will get 50% of the CPU. > The second process will then use its own algorithm to split it's 50% of the > CPU up between it's two threads. If it divides it evenly, then each of its' > threads will end up with 25% of the CPU whereas the thread for the first > process has 50% of the CPU. The idea for this is that if you have a system > with several single-threaded processes and one process with 1000 threads, you > don't want that process to starve out all the other processes. Thank you very much for your explanation. I have some difficulties though to understand why this is a thread attribute, not process-wide. This implies that threads inside one process may be either PTHREAD_SCOPE_SYSTEM or PTHREAD_SCOPE_PROCESS. I don't really get the point in doing this, except if this how libpthread manages to achieve M:N threading. > In practice things get much hairier, but suffice it to say that libpthread > manages the scheduling of PTHREAD_SCOPE_PROCESS threads in userland, whereas > libthr would have to depend on the kernel managing that. How d those two attributes modify libpthread behaviour in regards to M:N scheduling ? I am quite puzzled about this. > (To some extent > libpthread needs some help from the kernel to provide this as well.) Scheduler Activations ? Thank you very much. Best regards, -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From owner-freebsd-threads@FreeBSD.ORG Tue Aug 8 18:02:00 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 80CA516A4DD for ; Tue, 8 Aug 2006 18:02:00 +0000 (UTC) (envelope-from jhb@freebsd.org) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id 841AA43D45 for ; Tue, 8 Aug 2006 18:01:59 +0000 (GMT) (envelope-from jhb@freebsd.org) Received: from [10.0.1.106] (nat-outside.atlanta.corp.yahoo.com [63.172.193.57]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id k78I1uOx051913; Tue, 8 Aug 2006 14:01:57 -0400 (EDT) (envelope-from jhb@freebsd.org) From: John Baldwin To: Jeremie Le Hen Date: Tue, 8 Aug 2006 11:33:53 -0400 User-Agent: KMail/1.9.1 References: <20060804140657.GK4498@obiwan.tataz.chchile.org> <200608041134.43979.john@baldwin.cx> <20060807213945.GQ4498@obiwan.tataz.chchile.org> In-Reply-To: <20060807213945.GQ4498@obiwan.tataz.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200608081133.54490.jhb@freebsd.org> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [66.23.211.162]); Tue, 08 Aug 2006 14:01:57 -0400 (EDT) X-Virus-Scanned: ClamAV 0.87.1/1640/Mon Aug 7 21:11:04 2006 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.1 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on server.baldwin.cx Cc: freebsd-threads@freebsd.org Subject: Re: [fbsd] Re: system scope vs. process scope X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 08 Aug 2006 18:02:00 -0000 On Monday 07 August 2006 17:39, Jeremie Le Hen wrote: > Hi John, >=20 > On Fri, Aug 04, 2006 at 11:34:43AM -0400, John Baldwin wrote: > > On Friday 04 August 2006 10:06, Jeremie Le Hen wrote: > > > Hi, > > >=20 > > > occasionally, I saw environnement variables LIBPTHREAD_SYSTEM_SCOPE a= nd > > > LIBPTHREAD_PROCESS_SCOPE mentionned hither and thither. I grep(1)'ed > > > through the source tree for some documentation, but I wasn't able to > > > find any. Read the source Luke... > > >=20 > > > It seems that the PTHREAD_SCOPE_SYSTEM thread attribute is the defaul= t. > > > Intuitivelely, I would say that setting the PTHREAD_SCOPE_PROCESS > > > attribute creates a new process with its address space being shared w= ith > > > the other userland threads (a la Linux) whereas the default behaviour= is > > > to create a new kernel thread. > > >=20 > > > Am I right ? What are the pros and cons of either methods ? > >=20 > > That's not what it means. :) It has to do with scheduling. A=20 > > PTHREAD_SCOPE_SYSTEM will compete for the CPU with other "system" threa= ds=20 > > (aka kernel threads). That is, each userland thread has a direct kerne= l=20 > > thread that is visible to the system (kernel). A PTHREAD_SCOPE_PROCESS= =20 > > thread competes for the CPU just within the current process. That is, = each=20 > > group of PTHREAD_SCOPE_PROCESS thread's all share (in theory) a single = kernel=20 > > thread that competes for CPU with other system threads. An example mig= ht=20 > > explain the theory more completely. > >=20 > > Suppose you have a system with 2 processes. One process is single-thre= aded=20 > > and is CPU bound. The other process has 2 threads both of which are al= so CPU=20 > > bound. If the threads in the second process are PTHREAD_SCOPE_SYSTEM, = then=20 > > each thread will get 33% of the CPU. If the threads in the second proc= ess=20 > > are PTHREAD_SCOPE_PROCESS, then the each process will get 50% of the CP= U. =20 > > The second process will then use its own algorithm to split it's 50% of= the=20 > > CPU up between it's two threads. If it divides it evenly, then each of= its'=20 > > threads will end up with 25% of the CPU whereas the thread for the firs= t=20 > > process has 50% of the CPU. The idea for this is that if you have a sy= stem=20 > > with several single-threaded processes and one process with 1000 thread= s, you=20 > > don't want that process to starve out all the other processes. >=20 > Thank you very much for your explanation. >=20 > I have some difficulties though to understand why this is a thread > attribute, not process-wide. This implies that threads inside one > process may be either PTHREAD_SCOPE_SYSTEM or PTHREAD_SCOPE_PROCESS. > I don't really get the point in doing this, except if this how > libpthread manages to achieve M:N threading. You can mix the two. You might have a dedicated SYSTEM_SCOPE thread for doing some task like running a CORBA ORB server over in a 3rd party library and then create a pool of PROCESS_SCOPE worker threads for your actual application. > > In practice things get much hairier, but suffice it to say that libpthr= ead=20 > > manages the scheduling of PTHREAD_SCOPE_PROCESS threads in userland, wh= ereas=20 > > libthr would have to depend on the kernel managing that. >=20 > How d those two attributes modify libpthread behaviour in regards > to M:N scheduling ? I am quite puzzled about this. >=20 > > (To some extent=20 > > libpthread needs some help from the kernel to provide this as well.) >=20 > Scheduler Activations ? It's not really related to SAs directly. It's hard to explain w/o pictures. However, try to imagine how you would actually implement this stuff. =46irst, let's look at the M:M case where each userland thread has its own kernel thread. Suppose you have a multithreaded process where all the threads are PROCESS_SCOPE and you have a multithreaded process where all the threads are SYSTEM_SCOPE. You have a fixed quantum of, say, 10ms. What that tells you is that each SYSTEM_SCOPE thread should run for 10ms before being preempted (assume no I/O or blocking for now), but that the PROCESS_SCOPE threads all have to share a single 10ms quantum. How do you guarantee that? Well, you can create a thread group type structure that the actual quantum gets assigned to. A SYSTEM_SCOPE thread has its own private group (so it gets the full quantum), whereas the PROCESS_SCOPE threads in a process share a single group. The theory then is that you schedule thread groups to run for a quantum, and the group splits that up among its threads. In the case of a SYSTEM_SCOPE thread, there's only 1 thread in the group, so you just run the 1 thread. In the case of PROCESS_SCOPE, you now have to come up with an algorithm for determining how to schedule the various threads during that quantum. All of this stuff so far has been done in the kernel, and in theory the 'ksegroup' structure in the kernel serves as the thread group type entity mentioned above. In practice I believe it is a lot more complex than the above simple version. Now, as far as Scheduler Activations, they are primarily a way to optimize the PROCESS_SCOPE case a bit and allow for userland to participate in scheduling the threads. With SA, when you create a new PROCESS_SCOPE thread, it doesn't create a kernel thread (assume for simplicity the UP case for now). Instead, the kernel thread for your process is "tied" to a specific PROCESS_SCOPE thread. You can sort of think of the kernel thread as being a CPU for your process at at this point, and libpthread has its own scheduler that chooses which userland thread to run. When a userland thread makes a system call, the kernel thread it is bound to goes into the kernel to do the work. If the kernel thread has to block for I/O, then the kernel creates a new kernel thread that goes back into userland to let another userland thread continue to use the quantum. If all of the userland threads are blocked, then the new kernel thread has nothing to do, so it can commit suicide. Thus, you only have kernel threads for any threads blocked in the kernel, plus 1 kernel thread to run any non-blocked threads. This can result in the kernel only having 1 runnable thread in its thread group at a given time (like it would for SYSTEM_SCOPE threads) thus making the kernel scheduler not have to do as much work. With SMP there is some extra complication in that you will actually give a process up to N kernel threads (where N =3D=3D number of CPUs) to run non-blocked threads to allow for concurrent work, but many of the basics are the same. =46rom this, you can see that KSE isn't just about scheduler activations (which many people think it is), but it also provides the framework for managing PROCESS_SCOPE vs SYSTEM_SCOPE threads, even for the M:M case. These two parts of KSE are fairly orthogonal. You could have SA (and thus M:N) without supporting PROCESS_SCOPE vs SYSTEM_SCOPE, and you could have scheduler fairness (PROCESS_SCOPE vs SYSTEM_SCOPE) without SA. =2D-=20 John Baldwin From owner-freebsd-threads@FreeBSD.ORG Thu Aug 10 12:47:11 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D42F416A4DD; Thu, 10 Aug 2006 12:47:11 +0000 (UTC) (envelope-from simonmar@microsoft.com) Received: from mail-eur.microsoft.com (mail-eur.microsoft.com [213.199.128.145]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3875C43D45; Thu, 10 Aug 2006 12:47:11 +0000 (GMT) (envelope-from simonmar@microsoft.com) Received: from EUR-MSG-20.europe.corp.microsoft.com ([65.53.210.18]) by mail-eur.microsoft.com with Microsoft SMTPSVC(6.0.3790.1830); Thu, 10 Aug 2006 13:47:07 +0100 X-MimeOLE: Produced By Microsoft Exchange V6.5 Content-class: urn:content-classes:message MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable Date: Thu, 10 Aug 2006 13:47:06 +0100 Message-ID: <2E9B33CE230409489A7ED37E5E34090F05148C80@EUR-MSG-20.europe.corp.microsoft.com> in-reply-to: <20060807182923.F45647@fledge.watson.org> X-MS-Has-Attach: X-MS-TNEF-Correlator: Thread-Topic: GHC 6.4.3 on FreeBSD (fwd) thread-index: Aca6R0tTIvzngMy0SPSsiRlMH1N3BwCMrJIw From: "Simon Marlow" To: "Robert Watson" X-OriginalArrivalTime: 10 Aug 2006 12:47:07.0457 (UTC) FILETIME=[16B49310:01C6BC7B] Cc: glasgow-haskell-users@haskell.org, haskell@freebsd.org, Volker Stolz , Simon Peyton-Jones , Tony Finch , threads@freebsd.org Subject: RE: GHC 6.4.3 on FreeBSD (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 12:47:12 -0000 Dear threads@freebsd.org (and others), This is relating to the problems we're experiencing with GHC's threaded runtime on FreeBSD (see context below). GHC itself, which is a Haskell program, when compiled with itself and linked against GHC's threaded runtime, sometimes hangs eating 100% CPU. This happens when it is linked with -pthreads, I haven't seen it when linking with -lthr. If I attach using gdb to a hung process, I see this: (gdb) where #0 0x2897146b in pthread_testcancel () from /usr/lib/libpthread.so.2 #1 0x28972499 in __error () from /usr/lib/libpthread.so.2 #2 0x2895e6ba in sigaction () from /usr/lib/libpthread.so.2 #3 0x087ea6b4 in runProcess (args=3D0x29091758, workingDirectory=3D0x0, environment=3D0x0,=20 fdStdInput=3D0, fdStdOutput=3D1, fdStdError=3D2, set_inthandler=3D1, inthandler=3D0,=20 set_quithandler=3D1, quithandler=3D0) at runProcess.c:61 #4 0x0879b3d0 in s3ja_info () Now, runProcess() is part of our libraries. It is basically doing a fork/exec combo, with some sigaction()s and other stuff in between. As I understand the POSIX spec, it should be safe to do sigaction() in the child of a fork() in a multithreaded process. So I'm surprised to see pthread_testcancel on the stack there. This is about all I've managed to extract. gdb is having difficulty looking at the other threads: (gdb) info thr * 5 LWP 100331 0x2897146b in pthread_testcancel () from /usr/lib/libpthread.so.2 4 Thread 0x8920000 (LWP 100278) Cannot get lwp 100278 registers: Operation not permitted for unknown reasons. Any help would be appreciated... Cheers, Simon On 07 August 2006 18:31, Robert Watson wrote: > Dear many, >=20 > I pinged the FreeBSD threads list about the previously mentioned > problem, and was sent this patch by Dan Eischen. The FreeBSD threads > mailing list is extremely responsive to bug reports; my > recommendation is that we direct future specific bug reports, as well > as any feedback on this patch, to that list. >=20 > Thanks, >=20 > Robert N M Watson > Computer Laboratory > University of Cambridge >=20 > ---------- Forwarded message ---------- > Date: Mon, 7 Aug 2006 13:15:21 -0400 (EDT) > From: Daniel Eischen > To: Robert Watson > Cc: threads@freebsd.org > Subject: Re: GHC 6.4.3 on FreeBSD (fwd) >=20 > On Mon, 7 Aug 2006, Robert Watson wrote: >=20 >>=20 >> I've been talking to the Haskell folk about problems they've had >> getting their threaded runtime running on FreeBSD. Here's one of >> their problem reports. Since I'm not up on how threads and signals >> are supposed to act, I can't confirm the below is a bug, of course, >> but figured you (experts) would be able to respond better. >=20 > Try this patch. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 10 20:26:36 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DAAF816A4DE; Thu, 10 Aug 2006 20:26:36 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F12643D5A; Thu, 10 Aug 2006 20:26:33 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.7/8.13.7/NETPLEX) with ESMTP id k7AKQIiN005169; Thu, 10 Aug 2006 16:26:18 -0400 (EDT) Date: Thu, 10 Aug 2006 16:26:18 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Simon Marlow In-Reply-To: <2E9B33CE230409489A7ED37E5E34090F05148C80@EUR-MSG-20.europe.corp.microsoft.com> Message-ID: References: <2E9B33CE230409489A7ED37E5E34090F05148C80@EUR-MSG-20.europe.corp.microsoft.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: glasgow-haskell-users@haskell.org, haskell@freebsd.org, Volker Stolz , Simon Peyton-Jones , Tony Finch , Robert Watson , threads@freebsd.org Subject: RE: GHC 6.4.3 on FreeBSD (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 10 Aug 2006 20:26:36 -0000 On Thu, 10 Aug 2006, Simon Marlow wrote: > Dear threads@freebsd.org (and others), > > This is relating to the problems we're experiencing with GHC's threaded > runtime on FreeBSD (see context below). > > GHC itself, which is a Haskell program, when compiled with itself and > linked against GHC's threaded runtime, sometimes hangs eating 100% CPU. > This happens when it is linked with -pthreads, I haven't seen it when > linking with -lthr. > > If I attach using gdb to a hung process, I see this: > > (gdb) where > #0 0x2897146b in pthread_testcancel () from /usr/lib/libpthread.so.2 > #1 0x28972499 in __error () from /usr/lib/libpthread.so.2 > #2 0x2895e6ba in sigaction () from /usr/lib/libpthread.so.2 > #3 0x087ea6b4 in runProcess (args=0x29091758, workingDirectory=0x0, > environment=0x0, > fdStdInput=0, fdStdOutput=1, fdStdError=2, set_inthandler=1, > inthandler=0, > set_quithandler=1, quithandler=0) at runProcess.c:61 > #4 0x0879b3d0 in s3ja_info () That trace isn't possible. I've found that you can't trust these traces unless you compile libpthread with "-g". -- DE From owner-freebsd-threads@FreeBSD.ORG Fri Aug 11 04:01:25 2006 Return-Path: X-Original-To: threads@freebsd.org Delivered-To: freebsd-threads@FreeBSD.ORG Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7331516A4DA; Fri, 11 Aug 2006 04:01:25 +0000 (UTC) (envelope-from fullermd@over-yonder.net) Received: from mail.localelinks.com (web.localelinks.com [64.39.75.54]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1D39343D46; Fri, 11 Aug 2006 04:01:24 +0000 (GMT) (envelope-from fullermd@over-yonder.net) Received: from draco.over-yonder.net (adsl-072-148-013-213.sip.jan.bellsouth.net [72.148.13.213]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mail.localelinks.com (Postfix) with ESMTP id 007C6FD; Thu, 10 Aug 2006 23:01:23 -0500 (CDT) Received: by draco.over-yonder.net (Postfix, from userid 100) id 1DCC361C5E; Thu, 10 Aug 2006 23:01:23 -0500 (CDT) Date: Thu, 10 Aug 2006 23:01:23 -0500 From: "Matthew D. Fuller" To: Simon Marlow Message-ID: <20060811040123.GS65866@over-yonder.net> References: <20060807182923.F45647@fledge.watson.org> <2E9B33CE230409489A7ED37E5E34090F05148C80@EUR-MSG-20.europe.corp.microsoft.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <2E9B33CE230409489A7ED37E5E34090F05148C80@EUR-MSG-20.europe.corp.microsoft.com> X-Editor: vi X-OS: FreeBSD User-Agent: Mutt/1.5.11-fullermd.3 Cc: glasgow-haskell-users@haskell.org, haskell@freebsd.org, Volker Stolz , Simon Peyton-Jones , Tony Finch , Robert Watson , threads@freebsd.org Subject: Re: GHC 6.4.3 on FreeBSD (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 11 Aug 2006 04:01:25 -0000 On Thu, Aug 10, 2006 at 01:47:06PM +0100 I heard the voice of Simon Marlow, and lo! it spake thus: > > If I attach using gdb to a hung process, I see this: > > (gdb) where > #0 0x2897146b in pthread_testcancel () from /usr/lib/libpthread.so.2 FWIW, when I was trying (with no knowledge of what I was doing ;) to get the ghc port to build on -CURRENT, using both 6.x and 5.x bootstrap binaries, they would lock themselves up at 100% with a ktrace showing them calling pthread_testcancel()'s as fast as my disk could fill up with the ktrace log. I'd actually assumed it was some mismatch with trying to run the 6/5 binaries on the 7 system, but I guess not... -- Matthew Fuller (MF4839) | fullermd@over-yonder.net Systems/Network Administrator | http://www.over-yonder.net/~fullermd/ On the Internet, nobody can hear you scream.