From owner-freebsd-threads@FreeBSD.ORG Mon Jul 31 11:03:48 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9CC7016A547 for ; Mon, 31 Jul 2006 11:03:48 +0000 (UTC) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 334AC43D77 for ; Mon, 31 Jul 2006 11:03:23 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k6VB3M2Z051987 for ; Mon, 31 Jul 2006 11:03:22 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k6VB3Jg6051983 for freebsd-threads@freebsd.org; Mon, 31 Jul 2006 11:03:19 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 31 Jul 2006 11:03:19 GMT Message-Id: <200607311103.k6VB3Jg6051983@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Cc: Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 31 Jul 2006 11:03:48 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- s [2005/01/26] threads/76690threads fork hang in child for (-lc_r & -lthr) 1 problem total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can s [2001/01/20] threads/24472threads libc_r does not honor SO_SNDTIMEO/SO_RCVT s [2001/01/25] threads/24632threads libc_r delicate deviation from libc in ha s [2001/11/26] bin/32295 threads pthread dont dequeue signals s [2002/02/01] threads/34536threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe s [2002/06/27] threads/39922threads [threads] [patch] Threaded applications e s [2003/03/02] threads/48856threads Setting SIGCHLD to SIG_IGN still leaves z s [2003/03/10] threads/49087threads Signals lost in programs linked with libc s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/08/26] threads/70975threads unexpected and unreliable behaviour when o [2004/10/05] threads/72353threads Assertion fails in /usr/src/lib/libpthrea o [2004/10/07] threads/72429threads threads blocked in stdio (fgets, etc) are o [2004/10/21] threads/72953threads fork() unblocks blocked signals w/o PTHRE o [2004/12/19] threads/75273threads FBSD 5.3 libpthread (KSE) bug o [2004/12/21] threads/75374threads pthread_kill() ignores SA_SIGINFO flag s [2005/01/26] threads/76694threads fork cause hang in dup()/close() function o [2005/04/08] threads/79683threads svctcp_create() fails if multiple threads o [2005/04/28] threads/80435threads panic on high loads o [2005/07/22] threads/83914threads [libc] popen() doesn't work in static thr s [2005/08/02] threads/84483threads problems with devel/nspr and -lc_r on 4.x o [2005/08/20] threads/85160threads [libthr] [patch] libobjc + libpthread/lib p [2005/11/19] threads/89262threads [kernel] [patch] multi-threaded process h o [2005/12/12] threads/90278threads libthr, ULE and -current produces >100% W o [2006/01/03] kern/91266 threads [threads] Trying sleep, but thread marked s [2006/03/15] threads/94467threads send(), sendto() and sendmsg() are not co f [2006/06/01] threads/98256threads gnome-system-monitor core dumps from pthr s [2006/07/25] threads/100815threads FBSD 5.5 broke nanosleep in libc_r 28 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- s [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything s [2000/10/21] kern/22190 threads A threaded read(2) from a socketpair(2) f s [2001/09/09] threads/30464threads pthread mutex attributes -- pshared s [2002/05/02] threads/37676threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] threads/40671threads pthread_cancel doesn't remove thread from s [2004/07/13] threads/69020threads pthreads library leaks _gc_mutex o [2004/11/21] threads/74180threads KSE problem. Applications those riched ma o [2005/04/13] threads/79887threads [patch] freopen() isn't thread-safe o [2005/05/13] threads/80992threads abort() sometimes not caught by gdb depen o [2005/05/26] threads/81534threads [libc_r] [patch] libc_r close() will fail 10 problems total. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 17:40:29 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4766216A4DE for ; Thu, 3 Aug 2006 17:40:29 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1AE5E43D7E for ; Thu, 3 Aug 2006 17:40:16 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73HeFc2008805 for ; Thu, 3 Aug 2006 17:40:15 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73HeFxg008801; Thu, 3 Aug 2006 17:40:15 GMT (envelope-from gnats) Resent-Date: Thu, 3 Aug 2006 17:40:15 GMT Resent-Message-Id: <200608031740.k73HeFxg008801@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Poul-Henning Kamp Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9E99616A4DD for ; Thu, 3 Aug 2006 17:36:35 +0000 (UTC) (envelope-from phk@critter.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB61043D45 for ; Thu, 3 Aug 2006 17:36:32 +0000 (GMT) (envelope-from phk@critter.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id C7A6C170C5 for ; Thu, 3 Aug 2006 17:36:31 +0000 (UTC) Received: from critter.freebsd.dk (localhost [127.0.0.1]) by critter.freebsd.dk (8.13.7/8.13.7) with ESMTP id k739Osdr016407 for ; Thu, 3 Aug 2006 09:24:54 GMT (envelope-from phk@critter.freebsd.dk) Received: (from phk@localhost) by critter.freebsd.dk (8.13.7/8.13.7/Submit) id k739OrJ1016406; Thu, 3 Aug 2006 09:24:53 GMT (envelope-from phk) Message-Id: <200608030924.k739OrJ1016406@critter.freebsd.dk> Date: Thu, 3 Aug 2006 09:24:53 GMT From: Poul-Henning Kamp To: FreeBSD-gnats-submit@FreeBSD.org X-Send-Pr-Version: 3.113 Cc: Subject: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Poul-Henning Kamp List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 17:40:29 -0000 >Number: 101323 >Category: threads >Synopsis: fork(2) in threaded programs broken. >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Thu Aug 03 17:40:15 GMT 2006 >Closed-Date: >Last-Modified: >Originator: Poul-Henning Kamp >Release: FreeBSD 7.0-CURRENT i386 >Organization: >Environment: System: FreeBSD critter.freebsd.dk 7.0-CURRENT FreeBSD 7.0-CURRENT #5: Wed Jul 5 22:14:32 UTC 2006 root@critter.freebsd.dk:/critter/obj/critter/src/sys/TP41P i386 >Description: Forking a threaded process and starting a thread in the child process does not work. Tested on -current and releng_6 and both fails with more or less the same error message: Fatal error 'mutex is on list' at line 540 in file /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) >How-To-Repeat: #include #include void * thr_sleeper(void *arg) { (void)arg; while (1) { printf("%d\n", getpid()); sleep (1); } } int main(int argc, char **argv) { pthread_t t1, t2; (void)argc; (void)argv; pthread_create(&t1, NULL, thr_sleeper, NULL); if (!fork()) { pthread_create(&t2, NULL, thr_sleeper, NULL); while(1) sleep(1); } while(1) sleep(1); } >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 17:51:54 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A119B16A4DA; Thu, 3 Aug 2006 17:51:54 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 51E9B43D69; Thu, 3 Aug 2006 17:51:48 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.7/8.13.7/NETPLEX) with ESMTP id k73HpjvS016409; Thu, 3 Aug 2006 13:51:45 -0400 (EDT) Date: Thu, 3 Aug 2006 13:51:45 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Poul-Henning Kamp In-Reply-To: <200608030924.k739OrJ1016406@critter.freebsd.dk> Message-ID: References: <200608030924.k739OrJ1016406@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 17:51:54 -0000 On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > Forking a threaded process and starting a thread in the > child process does not work. > > Tested on -current and releng_6 and both fails with more or > less the same error message: > > Fatal error 'mutex is on list' at line 540 in file > /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) fork()ing from a threaded process and making calls to functions other than those defined as async-signal-safe is not allowed by POSIX. libpthread intentionally doesn't support this. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:00:35 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 50BA616A4E0 for ; Thu, 3 Aug 2006 18:00:35 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3DF7B43D49 for ; Thu, 3 Aug 2006 18:00:34 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73I0YWr010021 for ; Thu, 3 Aug 2006 18:00:34 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73I0YQR010020; Thu, 3 Aug 2006 18:00:34 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 18:00:34 GMT Message-Id: <200608031800.k73I0YQR010020@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Daniel Eischen Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:00:35 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: Daniel Eischen To: Poul-Henning Kamp Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 3 Aug 2006 13:51:45 -0400 (EDT) On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > Forking a threaded process and starting a thread in the > child process does not work. > > Tested on -current and releng_6 and both fails with more or > less the same error message: > > Fatal error 'mutex is on list' at line 540 in file > /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) fork()ing from a threaded process and making calls to functions other than those defined as async-signal-safe is not allowed by POSIX. libpthread intentionally doesn't support this. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:12:21 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4DED016A4E8; Thu, 3 Aug 2006 18:12:21 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id AE8B043D9C; Thu, 3 Aug 2006 18:11:39 +0000 (GMT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 0D59F170C5; Thu, 3 Aug 2006 18:11:29 +0000 (UTC) To: Daniel Eischen From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 03 Aug 2006 13:51:45 -0400." Date: Thu, 03 Aug 2006 18:11:28 +0000 Message-ID: <49426.1154628688@critter.freebsd.dk> Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:12:21 -0000 In message , Daniel Eischen writes: >On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > >> Forking a threaded process and starting a thread in the >> child process does not work. >> >> Tested on -current and releng_6 and both fails with more or >> less the same error message: >> >> Fatal error 'mutex is on list' at line 540 in file >> /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) > >fork()ing from a threaded process and making calls to functions >other than those defined as async-signal-safe is not allowed >by POSIX. libpthread intentionally doesn't support this. As you may probably be aware, I don't hold great respect for POSIX when it comes to writing useful APIs :-) First of all, there is a difference between POSIX explicitly disallowing something and POSIX not guaranteeing that something works. I belive we are in the second range here. Both Solaris and Linux support the usage which FreeBSD fails and as far as I can tell from our source-code, we also go a long way to make it work, just not long enough. As far as I can tell, all that's need to make it work correctly is the attached patch (which I've sent to Jason Evans since it's malloc related.) Poul-Henning Index: thr_fork.c =================================================================== RCS file: /home/ncvs/src/lib/libpthread/thread/thr_fork.c,v retrieving revision 1.37 diff -u -r1.37 thr_fork.c --- thr_fork.c 13 Mar 2006 00:59:51 -0000 1.37 +++ thr_fork.c 3 Aug 2006 18:09:44 -0000 @@ -93,10 +93,10 @@ } /* Fork a new process: */ - if (_kse_isthreaded() != 0) { - _malloc_prefork(); - } - if ((ret = __sys_fork()) == 0) { + _malloc_prefork(); + ret = __sys_fork(); + _malloc_postfork(); + if (ret == 0) { /* Child process */ errsave = errno; @@ -110,9 +110,6 @@ } _thr_mutex_reinit(&_thr_atfork_mutex); } else { - if (_kse_isthreaded() != 0) { - _malloc_postfork(); - } errsave = errno; if (curthread->attr.flags & PTHREAD_SCOPE_SYSTEM) { __sys_sigprocmask(SIG_SETMASK, &oldset, NULL); -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:22:23 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6CD9416A4F3 for ; Thu, 3 Aug 2006 18:22:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4EDA743D9E for ; Thu, 3 Aug 2006 18:20:24 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73IKFVX011173 for ; Thu, 3 Aug 2006 18:20:15 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73IKFLm011172; Thu, 3 Aug 2006 18:20:15 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 18:20:15 GMT Message-Id: <200608031820.k73IKFLm011172@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: "Poul-Henning Kamp" Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Poul-Henning Kamp List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:22:23 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: "Poul-Henning Kamp" To: Daniel Eischen Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 03 Aug 2006 18:11:28 +0000 In message , Daniel Eischen writes: >On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > >> Forking a threaded process and starting a thread in the >> child process does not work. >> >> Tested on -current and releng_6 and both fails with more or >> less the same error message: >> >> Fatal error 'mutex is on list' at line 540 in file >> /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) > >fork()ing from a threaded process and making calls to functions >other than those defined as async-signal-safe is not allowed >by POSIX. libpthread intentionally doesn't support this. As you may probably be aware, I don't hold great respect for POSIX when it comes to writing useful APIs :-) First of all, there is a difference between POSIX explicitly disallowing something and POSIX not guaranteeing that something works. I belive we are in the second range here. Both Solaris and Linux support the usage which FreeBSD fails and as far as I can tell from our source-code, we also go a long way to make it work, just not long enough. As far as I can tell, all that's need to make it work correctly is the attached patch (which I've sent to Jason Evans since it's malloc related.) Poul-Henning Index: thr_fork.c =================================================================== RCS file: /home/ncvs/src/lib/libpthread/thread/thr_fork.c,v retrieving revision 1.37 diff -u -r1.37 thr_fork.c --- thr_fork.c 13 Mar 2006 00:59:51 -0000 1.37 +++ thr_fork.c 3 Aug 2006 18:09:44 -0000 @@ -93,10 +93,10 @@ } /* Fork a new process: */ - if (_kse_isthreaded() != 0) { - _malloc_prefork(); - } - if ((ret = __sys_fork()) == 0) { + _malloc_prefork(); + ret = __sys_fork(); + _malloc_postfork(); + if (ret == 0) { /* Child process */ errsave = errno; @@ -110,9 +110,6 @@ } _thr_mutex_reinit(&_thr_atfork_mutex); } else { - if (_kse_isthreaded() != 0) { - _malloc_postfork(); - } errsave = errno; if (curthread->attr.flags & PTHREAD_SCOPE_SYSTEM) { __sys_sigprocmask(SIG_SETMASK, &oldset, NULL); -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:34:17 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E198916A4DF; Thu, 3 Aug 2006 18:34:17 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id D6F7843D93; Thu, 3 Aug 2006 18:34:05 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.7/8.13.7/NETPLEX) with ESMTP id k73IY4pt029324; Thu, 3 Aug 2006 14:34:04 -0400 (EDT) Date: Thu, 3 Aug 2006 14:34:04 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Poul-Henning Kamp In-Reply-To: <49426.1154628688@critter.freebsd.dk> Message-ID: References: <49426.1154628688@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:34:18 -0000 On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > In message , Daniel Eischen writes: >> On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: >> >>> Forking a threaded process and starting a thread in the >>> child process does not work. >>> >>> Tested on -current and releng_6 and both fails with more or >>> less the same error message: >>> >>> Fatal error 'mutex is on list' at line 540 in file >>> /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) >> >> fork()ing from a threaded process and making calls to functions >> other than those defined as async-signal-safe is not allowed >> by POSIX. libpthread intentionally doesn't support this. > > As you may probably be aware, I don't hold great respect for > POSIX when it comes to writing useful APIs :-) > > First of all, there is a difference between POSIX explicitly > disallowing something and POSIX not guaranteeing that something > works. > > I belive we are in the second range here. You should read section 2.4.3 of "Signal Concepts" in the POSIX spec. > Both Solaris and Linux support the usage which FreeBSD fails and > as far as I can tell from our source-code, we also go a long way > to make it work, just not long enough. > > As far as I can tell, all that's need to make it work correctly is > the attached patch (which I've sent to Jason Evans since it's malloc related.) No, that's not nearly enough. This has been discussed in -threads before. Forking from a multi-threaded program is just like an asynchronous signal in an unthreaded program. You have no idea what state any of the libraries or application data is in. Other threads may have taken either library or application locks and left things in an inconsistent state. The only sure way to safely fork() from a threaded process is to suspend threads while assuring they are not in critical regions, then fork() and reinitialize any necessary library global data in the child, and resume the suspended threads in the parent after the fork(). It may seem simple to #define NOTYET in src/lib/libpthread/thread/thr_kern.c but that isn't going to guarantee that libc or other library data is left in a consistent state and that their locks are reinitialized. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:40:18 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 74C3316A4DE for ; Thu, 3 Aug 2006 18:40:18 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1296943D45 for ; Thu, 3 Aug 2006 18:40:18 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73IeH5P013688 for ; Thu, 3 Aug 2006 18:40:17 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73IeHMA013687; Thu, 3 Aug 2006 18:40:17 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 18:40:17 GMT Message-Id: <200608031840.k73IeHMA013687@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Daniel Eischen Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:40:18 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: Daniel Eischen To: Poul-Henning Kamp Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 3 Aug 2006 14:34:04 -0400 (EDT) On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > In message , Daniel Eischen writes: >> On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: >> >>> Forking a threaded process and starting a thread in the >>> child process does not work. >>> >>> Tested on -current and releng_6 and both fails with more or >>> less the same error message: >>> >>> Fatal error 'mutex is on list' at line 540 in file >>> /usr/src/lib/libpthread/thread/thr_mutex.c (errno = 0) >> >> fork()ing from a threaded process and making calls to functions >> other than those defined as async-signal-safe is not allowed >> by POSIX. libpthread intentionally doesn't support this. > > As you may probably be aware, I don't hold great respect for > POSIX when it comes to writing useful APIs :-) > > First of all, there is a difference between POSIX explicitly > disallowing something and POSIX not guaranteeing that something > works. > > I belive we are in the second range here. You should read section 2.4.3 of "Signal Concepts" in the POSIX spec. > Both Solaris and Linux support the usage which FreeBSD fails and > as far as I can tell from our source-code, we also go a long way > to make it work, just not long enough. > > As far as I can tell, all that's need to make it work correctly is > the attached patch (which I've sent to Jason Evans since it's malloc related.) No, that's not nearly enough. This has been discussed in -threads before. Forking from a multi-threaded program is just like an asynchronous signal in an unthreaded program. You have no idea what state any of the libraries or application data is in. Other threads may have taken either library or application locks and left things in an inconsistent state. The only sure way to safely fork() from a threaded process is to suspend threads while assuring they are not in critical regions, then fork() and reinitialize any necessary library global data in the child, and resume the suspended threads in the parent after the fork(). It may seem simple to #define NOTYET in src/lib/libpthread/thread/thr_kern.c but that isn't going to guarantee that libc or other library data is left in a consistent state and that their locks are reinitialized. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:44:59 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E82D516A4E7; Thu, 3 Aug 2006 18:44:59 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id 814AF43D46; Thu, 3 Aug 2006 18:44:59 +0000 (GMT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id BE8DC170C5; Thu, 3 Aug 2006 18:44:57 +0000 (UTC) To: Daniel Eischen From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 03 Aug 2006 14:34:04 -0400." Date: Thu, 03 Aug 2006 18:44:56 +0000 Message-ID: <49575.1154630696@critter.freebsd.dk> Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:45:00 -0000 In message , Daniel Eischen wr ites: >No, that's not nearly enough. This has been discussed in >-threads before. > >Forking from a multi-threaded program is just like an >asynchronous signal in an unthreaded program. You have >no idea what state any of the libraries or application data >is in. ... Unless of course, the programmer too great care to make sure he did, and therefore assumes that fork() will actually be safe. In my case, I know the exact state of the entire process and I know 100% certain that there are no locks held which will affect the forked copy. ... except that holding all malloc's locks screws me over :-( I will agree that there is no "perfect" solution, but that is not what I'm after, I'm after "works in controlled circumstances. I would argue that an implementation that does: hold any library locks we want to handle fork if (parent) release those locks again return else unlock all locks (since they cannot possibly make sense in the child in a locked state) return That would go a long way towards a "works if you're careful" implementation. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 18:50:23 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3BF6516A4DF for ; Thu, 3 Aug 2006 18:50:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id ED77E43D6E for ; Thu, 3 Aug 2006 18:50:18 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73IoIh7014274 for ; Thu, 3 Aug 2006 18:50:18 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73IoIBr014269; Thu, 3 Aug 2006 18:50:18 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 18:50:18 GMT Message-Id: <200608031850.k73IoIBr014269@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: "Poul-Henning Kamp" Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Poul-Henning Kamp List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 18:50:23 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: "Poul-Henning Kamp" To: Daniel Eischen Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 03 Aug 2006 18:44:56 +0000 In message , Daniel Eischen wr ites: >No, that's not nearly enough. This has been discussed in >-threads before. > >Forking from a multi-threaded program is just like an >asynchronous signal in an unthreaded program. You have >no idea what state any of the libraries or application data >is in. ... Unless of course, the programmer too great care to make sure he did, and therefore assumes that fork() will actually be safe. In my case, I know the exact state of the entire process and I know 100% certain that there are no locks held which will affect the forked copy. ... except that holding all malloc's locks screws me over :-( I will agree that there is no "perfect" solution, but that is not what I'm after, I'm after "works in controlled circumstances. I would argue that an implementation that does: hold any library locks we want to handle fork if (parent) release those locks again return else unlock all locks (since they cannot possibly make sense in the child in a locked state) return That would go a long way towards a "works if you're careful" implementation. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:04:50 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E1A1C16A4DE; Thu, 3 Aug 2006 20:04:50 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7CEDF43D73; Thu, 3 Aug 2006 20:04:50 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.7/8.13.7/NETPLEX) with ESMTP id k73K4nxu003459; Thu, 3 Aug 2006 16:04:49 -0400 (EDT) Date: Thu, 3 Aug 2006 16:04:49 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Poul-Henning Kamp In-Reply-To: <49575.1154630696@critter.freebsd.dk> Message-ID: References: <49575.1154630696@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:04:51 -0000 On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > In message , Daniel Eischen wr > ites: > >> No, that's not nearly enough. This has been discussed in >> -threads before. >> >> Forking from a multi-threaded program is just like an >> asynchronous signal in an unthreaded program. You have >> no idea what state any of the libraries or application data >> is in. > > ... Unless of course, the programmer too great care to make > sure he did, and therefore assumes that fork() will actually > be safe. > > In my case, I know the exact state of the entire process > and I know 100% certain that there are no locks held which > will affect the forked copy. > > ... except that holding all malloc's locks screws me over :-( > > I will agree that there is no "perfect" solution, but that is > not what I'm after, I'm after "works in controlled circumstances. > > I would argue that an implementation that does: > > hold any library locks we want to handle > fork > if (parent) > release those locks again > return > else > unlock all locks (since they cannot possibly > make sense in the child in a locked state) > return There's no easy way to hold all library locks. They are littered in libc and libpthread and the application doesn't have access to them. You would have to teach libc to record these locks and export a function to lib to lock and unlock these them. > That would go a long way towards a "works if you're careful" > implementation. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:08:25 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1648216A4DA; Thu, 3 Aug 2006 20:08:25 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id BBE8443D82; Thu, 3 Aug 2006 20:08:24 +0000 (GMT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 55932170C5; Thu, 3 Aug 2006 20:08:21 +0000 (UTC) To: Daniel Eischen From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 03 Aug 2006 16:04:49 -0400." Date: Thu, 03 Aug 2006 20:08:20 +0000 Message-ID: <50596.1154635700@critter.freebsd.dk> Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:08:25 -0000 In message , Daniel Eischen wr ites: >There's no easy way to hold all library locks. They are >littered in libc and libpthread and the application doesn't >have access to them. You would have to teach libc to >record these locks and export a function to lib >to lock and unlock these them. I would be perfectly happy if libpthread would just at the very least release the locks it specifically grabs for the fork. There's a big difference between giving it a sensible shot and downright sabotaging it the way we do currently. Anyway, apart from the view from the theoretical high ground and the fact that POSIX doesn't actually say anything helpful here, are there any objections to the fix I proposed ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:10:23 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 18CE316A4DE for ; Thu, 3 Aug 2006 20:10:23 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id BBA2343D53 for ; Thu, 3 Aug 2006 20:10:20 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73KAK4M021184 for ; Thu, 3 Aug 2006 20:10:20 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73KAKHf021183; Thu, 3 Aug 2006 20:10:20 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 20:10:20 GMT Message-Id: <200608032010.k73KAKHf021183@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Daniel Eischen Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:10:23 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: Daniel Eischen To: Poul-Henning Kamp Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 3 Aug 2006 16:04:49 -0400 (EDT) On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > In message , Daniel Eischen wr > ites: > >> No, that's not nearly enough. This has been discussed in >> -threads before. >> >> Forking from a multi-threaded program is just like an >> asynchronous signal in an unthreaded program. You have >> no idea what state any of the libraries or application data >> is in. > > ... Unless of course, the programmer too great care to make > sure he did, and therefore assumes that fork() will actually > be safe. > > In my case, I know the exact state of the entire process > and I know 100% certain that there are no locks held which > will affect the forked copy. > > ... except that holding all malloc's locks screws me over :-( > > I will agree that there is no "perfect" solution, but that is > not what I'm after, I'm after "works in controlled circumstances. > > I would argue that an implementation that does: > > hold any library locks we want to handle > fork > if (parent) > release those locks again > return > else > unlock all locks (since they cannot possibly > make sense in the child in a locked state) > return There's no easy way to hold all library locks. They are littered in libc and libpthread and the application doesn't have access to them. You would have to teach libc to record these locks and export a function to lib to lock and unlock these them. > That would go a long way towards a "works if you're careful" > implementation. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:10:29 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6504716A4DE for ; Thu, 3 Aug 2006 20:10:29 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C226C43D55 for ; Thu, 3 Aug 2006 20:10:28 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73KASwM021231 for ; Thu, 3 Aug 2006 20:10:28 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73KASoE021230; Thu, 3 Aug 2006 20:10:28 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 20:10:28 GMT Message-Id: <200608032010.k73KASoE021230@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: "Poul-Henning Kamp" Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Poul-Henning Kamp List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:10:29 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: "Poul-Henning Kamp" To: Daniel Eischen Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 03 Aug 2006 20:08:20 +0000 In message , Daniel Eischen wr ites: >There's no easy way to hold all library locks. They are >littered in libc and libpthread and the application doesn't >have access to them. You would have to teach libc to >record these locks and export a function to lib >to lock and unlock these them. I would be perfectly happy if libpthread would just at the very least release the locks it specifically grabs for the fork. There's a big difference between giving it a sensible shot and downright sabotaging it the way we do currently. Anyway, apart from the view from the theoretical high ground and the fact that POSIX doesn't actually say anything helpful here, are there any objections to the fix I proposed ? -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:19:45 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 71EA716A51E; Thu, 3 Aug 2006 20:19:45 +0000 (UTC) (envelope-from deischen@freebsd.org) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 56D5D43D82; Thu, 3 Aug 2006 20:19:31 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) by mail.ntplx.net (8.13.7/8.13.7/NETPLEX) with ESMTP id k73KJTJN016165; Thu, 3 Aug 2006 16:19:30 -0400 (EDT) Date: Thu, 3 Aug 2006 16:19:29 -0400 (EDT) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Poul-Henning Kamp In-Reply-To: <50596.1154635700@critter.freebsd.dk> Message-ID: References: <50596.1154635700@critter.freebsd.dk> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII; format=flowed X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:19:45 -0000 On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > In message , Daniel Eischen wr > ites: > > >> There's no easy way to hold all library locks. They are >> littered in libc and libpthread and the application doesn't >> have access to them. You would have to teach libc to >> record these locks and export a function to lib >> to lock and unlock these them. > > I would be perfectly happy if libpthread would just at the very > least release the locks it specifically grabs for the fork. > > There's a big difference between giving it a sensible shot and > downright sabotaging it the way we do currently. Actually, I would prefer to emit an error message of the form: "fork() from a threaded process is not defined by POSIX" and purposefully segfault ;-) > Anyway, apart from the view from the theoretical high ground and > the fact that POSIX doesn't actually say anything helpful here, are > there any objections to the fix I proposed ? For that one specific change, no objection. I have an objection to enabling the NOTYET in thr_kern.c without having an overall solution for libc as well. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:20:38 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D52216A4FE for ; Thu, 3 Aug 2006 20:20:38 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id EB2A343DC7 for ; Thu, 3 Aug 2006 20:20:19 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73KKJn3021814 for ; Thu, 3 Aug 2006 20:20:19 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73KKJiq021813; Thu, 3 Aug 2006 20:20:19 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 20:20:19 GMT Message-Id: <200608032020.k73KKJiq021813@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Daniel Eischen Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:20:38 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: Daniel Eischen To: Poul-Henning Kamp Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 3 Aug 2006 16:19:29 -0400 (EDT) On Thu, 3 Aug 2006, Poul-Henning Kamp wrote: > In message , Daniel Eischen wr > ites: > > >> There's no easy way to hold all library locks. They are >> littered in libc and libpthread and the application doesn't >> have access to them. You would have to teach libc to >> record these locks and export a function to lib >> to lock and unlock these them. > > I would be perfectly happy if libpthread would just at the very > least release the locks it specifically grabs for the fork. > > There's a big difference between giving it a sensible shot and > downright sabotaging it the way we do currently. Actually, I would prefer to emit an error message of the form: "fork() from a threaded process is not defined by POSIX" and purposefully segfault ;-) > Anyway, apart from the view from the theoretical high ground and > the fact that POSIX doesn't actually say anything helpful here, are > there any objections to the fix I proposed ? For that one specific change, no objection. I have an objection to enabling the NOTYET in thr_kern.c without having an overall solution for libc as well. -- DE From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:21:10 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1A93116A4DA; Thu, 3 Aug 2006 20:21:10 +0000 (UTC) (envelope-from phk@phk.freebsd.dk) Received: from phk.freebsd.dk (phk.freebsd.dk [130.225.244.222]) by mx1.FreeBSD.org (Postfix) with ESMTP id C036C43D7D; Thu, 3 Aug 2006 20:21:09 +0000 (GMT) (envelope-from phk@phk.freebsd.dk) Received: from critter.freebsd.dk (critter.freebsd.dk [192.168.48.2]) by phk.freebsd.dk (Postfix) with ESMTP id 6D1AF170C5; Thu, 3 Aug 2006 20:21:08 +0000 (UTC) To: Daniel Eischen From: "Poul-Henning Kamp" In-Reply-To: Your message of "Thu, 03 Aug 2006 16:19:29 -0400." Date: Thu, 03 Aug 2006 20:21:07 +0000 Message-ID: <50664.1154636467@critter.freebsd.dk> Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:21:10 -0000 In message , Daniel Eischen wr ites: >Actually, I would prefer to emit an error message of the >form: > > "fork() from a threaded process is not defined by POSIX" > >and purposefully segfault ;-) Are you working for us or the competition ? :-) >> Anyway, apart from the view from the theoretical high ground and >> the fact that POSIX doesn't actually say anything helpful here, are >> there any objections to the fix I proposed ? > >For that one specific change, no objection. I have an >objection to enabling the NOTYET in thr_kern.c without >having an overall solution for libc as well. I have no plans of anything like that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Thu Aug 3 20:30:18 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8045F16A4DD for ; Thu, 3 Aug 2006 20:30:18 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id D8F9943D5A for ; Thu, 3 Aug 2006 20:30:17 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k73KUH1M022489 for ; Thu, 3 Aug 2006 20:30:17 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k73KUHOk022488; Thu, 3 Aug 2006 20:30:17 GMT (envelope-from gnats) Date: Thu, 3 Aug 2006 20:30:17 GMT Message-Id: <200608032030.k73KUHOk022488@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: "Poul-Henning Kamp" Cc: Subject: Re: threads/101323: fork(2) in threaded programs broken. X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: Poul-Henning Kamp List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 03 Aug 2006 20:30:18 -0000 The following reply was made to PR threads/101323; it has been noted by GNATS. From: "Poul-Henning Kamp" To: Daniel Eischen Cc: FreeBSD-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/101323: fork(2) in threaded programs broken. Date: Thu, 03 Aug 2006 20:21:07 +0000 In message , Daniel Eischen wr ites: >Actually, I would prefer to emit an error message of the >form: > > "fork() from a threaded process is not defined by POSIX" > >and purposefully segfault ;-) Are you working for us or the competition ? :-) >> Anyway, apart from the view from the theoretical high ground and >> the fact that POSIX doesn't actually say anything helpful here, are >> there any objections to the fix I proposed ? > >For that one specific change, no objection. I have an >objection to enabling the NOTYET in thr_kern.c without >having an overall solution for libc as well. I have no plans of anything like that. -- Poul-Henning Kamp | UNIX since Zilog Zeus 3.20 phk@FreeBSD.ORG | TCP/IP since RFC 956 FreeBSD committer | BSD since 4.3-tahoe Never attribute to malice what can adequately be explained by incompetence. From owner-freebsd-threads@FreeBSD.ORG Fri Aug 4 11:50:11 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 321F516A4DA for ; Fri, 4 Aug 2006 11:50:11 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6261F43D53 for ; Fri, 4 Aug 2006 11:50:10 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k74BoAgX001802 for ; Fri, 4 Aug 2006 11:50:10 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k74BoAgX001798; Fri, 4 Aug 2006 11:50:10 GMT (envelope-from gnats) Resent-Date: Fri, 4 Aug 2006 11:50:10 GMT Resent-Message-Id: <200608041150.k74BoAgX001798@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Gergely Kovacs Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B528C16A4E9 for ; Fri, 4 Aug 2006 11:44:18 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 65D2843D45 for ; Fri, 4 Aug 2006 11:44:18 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id k74BiHeU078514 for ; Fri, 4 Aug 2006 11:44:17 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id k74BiHbG078513; Fri, 4 Aug 2006 11:44:17 GMT (envelope-from nobody) Message-Id: <200608041144.k74BiHbG078513@www.freebsd.org> Date: Fri, 4 Aug 2006 11:44:17 GMT From: Gergely Kovacs To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Cc: Subject: threads/101355: threaded application spents too much time in _umtx_op with libthr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2006 11:50:11 -0000 >Number: 101355 >Category: threads >Synopsis: threaded application spents too much time in _umtx_op with libthr >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Fri Aug 04 11:50:09 GMT 2006 >Closed-Date: >Last-Modified: >Originator: Gergely Kovacs >Release: 6.1-STABLE >Organization: developer >Environment: FreeBSD fmx18 6.1-STABLE FreeBSD 6.1-STABLE #0: Mon Jul 17 13:39:39 CEST 2006 pisti@fmx18:/tmp/usr/src/sys/DL140 i386 Kernel settings: machine i386 cpu I686_CPU ident DL140 options SMP options SCHED_4BSD # 4BSD scheduler options PREEMPTION # Enable kernel thread preemption options INET # InterNETworking options FFS # Berkeley Fast Filesystem options SOFTUPDATES # Enable FFS soft updates support options UFS_DIRHASH # Improve performance on big directories options MD_ROOT # MD is a potential root device options NFSCLIENT # Network Filesystem Client options CD9660 # ISO 9660 Filesystem options PROCFS # Process filesystem (requires PSEUDOFS) options PSEUDOFS # Pseudo-filesystem framework options GEOM_GPT # GUID Partition Tables. options COMPAT_43 # Compatible with BSD 4.3 [KEEP THIS!] options COMPAT_FREEBSD4 # Compatible with FreeBSD4 options COMPAT_FREEBSD5 # Compatible with FreeBSD5 options SCSI_DELAY=5000 # Delay (in ms) before probing SCSI options KTRACE # ktrace(1) support options SYSVSHM # SYSV-style shared memory options SYSVMSG # SYSV-style message queues options SYSVSEM # SYSV-style semaphores options _KPOSIX_PRIORITY_SCHEDULING # POSIX P1003_1B real-time extensions options KBD_INSTALL_CDEV # install a CDEV entry in /dev options ADAPTIVE_GIANT # Giant mutex is adaptive. device apic # I/O APIC device pci device fdc device ata device atadisk # ATA disk drives device atapicd # ATAPI CDROM drives options ATA_STATIC_ID # Static device numbering device atkbdc # AT keyboard controller device atkbd # AT keyboard device psm # PS/2 mouse device vga # VGA video card driver device splash # Splash screen and screen saver support device sc device agp # support several AGP chipsets device pmtimer device sio # 8250, 16[45]50 based serial ports device miibus # MII bus support device bge # Broadcom BCM570xx Gigabit Ethernet device loop # Network loopback device random # Entropy device device ether # Ethernet support device pty # Pseudo-ttys (telnet etc) device bpf # Berkeley packet filter options IPFIREWALL options IPFIREWALL_VERBOSE >Description: A multi-threaded (libthr; threads are detached) mail defense application (AV) which uses some pthread functions (pthread_mutex_lock, pthread_create) gets extremely slow beside huge (3<) load (around 50-100 thread). It works correct when the load is less. The hardware is HP DL140, Dual Xeon 3.2. On a much slower machine with the same os, kernel settings and almost the same mail traffic could not repeat the problem. According to ktrace and gprof the most time's spent in _umtx_op syscall: ktrace output (kdump -E): 71391 vbmsrv 0.000532 CALL select(0x6,0xbf9fef00,0xbf9fee80,0xbf9fee00,0xbf9fddf8) 71391 vbmsrv 0.000535 RET _umtx_op 0 71391 vbmsrv 0.000542 CALL _umtx_op(0x28180700,0x1,0x1885a,0,0) 71391 vbmsrv 0.000552 RET _umtx_op 0 71391 vbmsrv 0.000559 RET _umtx_op 0 71391 vbmsrv 0.000560 CALL _umtx_op(0x28180700,0,0x1885a,0,0) 71391 vbmsrv 0.000570 CALL _umtx_op(0x28180700,0x1,0x1886d,0,0) 71391 vbmsrv 0.000588 RET _umtx_op 0 71391 vbmsrv 0.000589 RET _umtx_op 0 .. (only the _umtx_op function is called and returned) 71391 vbmsrv 0.002064 RET _umtx_op 0 71391 vbmsrv 0.002076 CALL _umtx_op(0x28180700,0,0x18828,0,0) 71391 vbmsrv 0.002081 CALL _umtx_op(0x28180700,0x1,0x18830,0,0) 71391 vbmsrv 0.002093 RET _umtx_op 0 71391 vbmsrv 0.002094 RET _umtx_op 0 71391 vbmsrv 0.002103 CALL _umtx_op(0x28180700,0,0x18830,0,0) 71391 vbmsrv 0.002104 CALL _umtx_op(0x28180700,0x1,0x18866,0,0) 71391 vbmsrv 0.002393 RET _umtx_op 0 71391 vbmsrv 0.002400 RET _umtx_op 0 71391 vbmsrv 0.002403 CALL _umtx_op(0x28180700,0,0x18866,0,0) 71391 vbmsrv 0.002416 CALL _umtx_op(0x28180700,0x1,0x18828,0,0) 71391 vbmsrv 0.002421 RET select 0 71391 vbmsrv 0.002430 CALL select(0x6,0xbf9fef00,0xbf9fee80,0xbf9fee00,0xbf9fddf8) 71391 vbmsrv 0.002436 RET _umtx_op 0 71391 vbmsrv 0.002452 RET _umtx_op 0 71391 vbmsrv 0.002456 CALL _umtx_op(0x28180700,0,0x18828,0,0) 71391 vbmsrv 0.002460 CALL _umtx_op(0x28180700,0x1,0x18866,0,0) 71391 vbmsrv 0.002481 RET _umtx_op 0 71391 vbmsrv 0.002494 CALL _umtx_op(0x28180700,0,0x18866,0,0) 71391 vbmsrv 0.002495 RET _umtx_op 0 71391 vbmsrv 0.002505 CALL _umtx_op(0x28180700,0x1,0x18828,0,0) 71391 vbmsrv 0.002520 RET _umtx_op 0 71391 vbmsrv Events dropped. 71391 vbmsrv 0.002536 CALL _umtx_op(0x28180700,0x1,0x18828,0,0) 71391 vbmsrv 0.002553 RET _umtx_op 0 71391 vbmsrv 0.002562 CALL _umtx_op(0x28180700,0,0x18828,0,0) gprof output: called/total parents index %time self descendents called+self name index called/total children [1] 83.3 94.41 0.00 _umtx_op [1] ----------------------------------------------- 0.00 0.00 5557/23145457 realloc [259] 2.42 3.53 11547805/23145457 malloc [6] 2.43 3.54 11592095/23145457 free [5] [2] 10.5 4.85 7.07 23145457 pubrealloc [2] 3.50 0.04 10968594/21914112 imalloc [9] 3.49 0.04 10945518/21914112 ifree [8] 0.01 0.00 1663/106492 memcpy [42] ----------------------------------------------- [3] 6.2 6.99 0.08 21914112+53128 [3] 3.64 0.01 10952801+12078 ifree [8] 3.31 0.00 10977846+12521 imalloc [9] 0.03 0.07 36593 malloc_pages [103] ----------------------------------------------- [4] 5.7 0.00 6.50 [4] >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Fri Aug 4 14:06:29 2006 Return-Path: X-Original-To: freebsd-threads@FreeBSD.org Delivered-To: freebsd-threads@FreeBSD.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8194B16A4E0 for ; Fri, 4 Aug 2006 14:06:29 +0000 (UTC) (envelope-from tataz@tataz.chchile.org) Received: from smtp5-g19.free.fr (smtp5-g19.free.fr [212.27.42.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6205943D6A for ; Fri, 4 Aug 2006 14:06:28 +0000 (GMT) (envelope-from tataz@tataz.chchile.org) Received: from tatooine.tataz.chchile.org (tataz.chchile.org [82.233.239.98]) by smtp5-g19.free.fr (Postfix) with ESMTP id 5FE7426563 for ; Fri, 4 Aug 2006 16:06:27 +0200 (CEST) Received: from obiwan.tataz.chchile.org (unknown [192.168.1.25]) by tatooine.tataz.chchile.org (Postfix) with ESMTP id C262C9CA77 for ; Fri, 4 Aug 2006 14:06:57 +0000 (UTC) Received: by obiwan.tataz.chchile.org (Postfix, from userid 1000) id B9063405B; Fri, 4 Aug 2006 16:06:57 +0200 (CEST) Date: Fri, 4 Aug 2006 16:06:57 +0200 From: Jeremie Le Hen To: freebsd-threads@FreeBSD.org Message-ID: <20060804140657.GK4498@obiwan.tataz.chchile.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.11 Cc: Subject: system scope vs. process scope X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2006 14:06:29 -0000 Hi, occasionally, I saw environnement variables LIBPTHREAD_SYSTEM_SCOPE and LIBPTHREAD_PROCESS_SCOPE mentionned hither and thither. I grep(1)'ed through the source tree for some documentation, but I wasn't able to find any. Read the source Luke... It seems that the PTHREAD_SCOPE_SYSTEM thread attribute is the default. Intuitivelely, I would say that setting the PTHREAD_SCOPE_PROCESS attribute creates a new process with its address space being shared with the other userland threads (a la Linux) whereas the default behaviour is to create a new kernel thread. Am I right ? What are the pros and cons of either methods ? In libthr, it seems those environnement variables / attributes leads to set a flag THR_SYSTEM_SCOPE which doesn't seem to be used in the library nor in the kernel. OTOH, in libpthread (libkse), it seems to be widely used. I fumbled through the code, but I am not sure to understand what these flags really do. Thank you. -- Jeremie Le Hen < jeremie at le-hen dot org >< ttz at chchile dot org > From owner-freebsd-threads@FreeBSD.ORG Fri Aug 4 15:35:58 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9FD3016A54F for ; Fri, 4 Aug 2006 15:35:58 +0000 (UTC) (envelope-from john@baldwin.cx) Received: from server.baldwin.cx (66-23-211-162.clients.speedfactory.net [66.23.211.162]) by mx1.FreeBSD.org (Postfix) with ESMTP id DF7BD43D45 for ; Fri, 4 Aug 2006 15:35:57 +0000 (GMT) (envelope-from john@baldwin.cx) Received: from localhost.corp.yahoo.com (john@localhost [127.0.0.1]) (authenticated bits=0) by server.baldwin.cx (8.13.6/8.13.6) with ESMTP id k74FZnG4003112; Fri, 4 Aug 2006 11:35:56 -0400 (EDT) (envelope-from john@baldwin.cx) From: John Baldwin To: freebsd-threads@freebsd.org Date: Fri, 4 Aug 2006 11:34:43 -0400 User-Agent: KMail/1.9.1 References: <20060804140657.GK4498@obiwan.tataz.chchile.org> In-Reply-To: <20060804140657.GK4498@obiwan.tataz.chchile.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200608041134.43979.john@baldwin.cx> X-Greylist: Sender succeeded SMTP AUTH authentication, not delayed by milter-greylist-2.0.2 (server.baldwin.cx [127.0.0.1]); Fri, 04 Aug 2006 11:35:57 -0400 (EDT) X-Virus-Scanned: ClamAV version 0.87.1, clamav-milter version 0.87 on server.baldwin.cx X-Virus-Status: Clean X-Spam-Status: No, score=-4.4 required=4.2 tests=ALL_TRUSTED,AWL,BAYES_00 autolearn=ham version=3.1.0 X-Spam-Checker-Version: SpamAssassin 3.1.0 (2005-09-13) on server.baldwin.cx Cc: Subject: Re: system scope vs. process scope X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 04 Aug 2006 15:35:58 -0000 On Friday 04 August 2006 10:06, Jeremie Le Hen wrote: > Hi, > > occasionally, I saw environnement variables LIBPTHREAD_SYSTEM_SCOPE and > LIBPTHREAD_PROCESS_SCOPE mentionned hither and thither. I grep(1)'ed > through the source tree for some documentation, but I wasn't able to > find any. Read the source Luke... > > It seems that the PTHREAD_SCOPE_SYSTEM thread attribute is the default. > Intuitivelely, I would say that setting the PTHREAD_SCOPE_PROCESS > attribute creates a new process with its address space being shared with > the other userland threads (a la Linux) whereas the default behaviour is > to create a new kernel thread. > > Am I right ? What are the pros and cons of either methods ? That's not what it means. :) It has to do with scheduling. A PTHREAD_SCOPE_SYSTEM will compete for the CPU with other "system" threads (aka kernel threads). That is, each userland thread has a direct kernel thread that is visible to the system (kernel). A PTHREAD_SCOPE_PROCESS thread competes for the CPU just within the current process. That is, each group of PTHREAD_SCOPE_PROCESS thread's all share (in theory) a single kernel thread that competes for CPU with other system threads. An example might explain the theory more completely. Suppose you have a system with 2 processes. One process is single-threaded and is CPU bound. The other process has 2 threads both of which are also CPU bound. If the threads in the second process are PTHREAD_SCOPE_SYSTEM, then each thread will get 33% of the CPU. If the threads in the second process are PTHREAD_SCOPE_PROCESS, then the each process will get 50% of the CPU. The second process will then use its own algorithm to split it's 50% of the CPU up between it's two threads. If it divides it evenly, then each of its' threads will end up with 25% of the CPU whereas the thread for the first process has 50% of the CPU. The idea for this is that if you have a system with several single-threaded processes and one process with 1000 threads, you don't want that process to starve out all the other processes. In practice things get much hairier, but suffice it to say that libpthread manages the scheduling of PTHREAD_SCOPE_PROCESS threads in userland, whereas libthr would have to depend on the kernel managing that. (To some extent libpthread needs some help from the kernel to provide this as well.) -- John Baldwin From owner-freebsd-threads@FreeBSD.ORG Sat Aug 5 00:00:43 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 3CAFA16A4DA; Sat, 5 Aug 2006 00:00:43 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-threads@freebsd.org Date: Sat, 5 Aug 2006 08:00:37 +0800 User-Agent: KMail/1.8.2 References: <200608041144.k74BiHbG078513@www.freebsd.org> In-Reply-To: <200608041144.k74BiHbG078513@www.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset="gb2312" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200608050800.38185.davidxu@freebsd.org> Cc: freebsd-gnats-submit@freebsd.org, Gergely Kovacs Subject: Re: threads/101355: threaded application spents too much time in _umtx_op with libthr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Aug 2006 00:00:43 -0000 On Friday 04 August 2006 19:44, Gergely Kovacs wrote: > >Number: 101355 > >Category: threads > >Synopsis: threaded application spents too much time in _umtx_op with > > libthr Confidential: no > >Severity: serious > >Priority: medium > >Responsible: freebsd-threads > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: sw-bug > >Submitter-Id: current-users > >Arrival-Date: Fri Aug 04 11:50:09 GMT 2006 > >Closed-Date: > >Last-Modified: > >Originator: Gergely Kovacs > >Release: 6.1-STABLE > >Organization: If your application is malloc hunger, then there is performance problem on 6.x since malloc is protected by single lock, you may try -CURRENT, the malloc was rewritten by Jason Evans on -CURRENT. David Xu From owner-freebsd-threads@FreeBSD.ORG Sat Aug 5 00:06:30 2006 Return-Path: X-Original-To: freebsd-threads@freebsd.org Delivered-To: freebsd-threads@freebsd.org Received: from localhost.my.domain (localhost [127.0.0.1]) by hub.freebsd.org (Postfix) with ESMTP id 2033016A4DE; Sat, 5 Aug 2006 00:06:30 +0000 (UTC) (envelope-from davidxu@freebsd.org) From: David Xu To: freebsd-threads@freebsd.org Date: Sat, 5 Aug 2006 08:06:24 +0800 User-Agent: KMail/1.8.2 References: <20060804140657.GK4498@obiwan.tataz.chchile.org> <200608041134.43979.john@baldwin.cx> In-Reply-To: <200608041134.43979.john@baldwin.cx> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200608050806.24502.davidxu@freebsd.org> Cc: John Baldwin Subject: Re: system scope vs. process scope X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Aug 2006 00:06:30 -0000 On Friday 04 August 2006 23:34, John Baldwin wrote: > Suppose you have a system with 2 processes. One process is single-threaded > and is CPU bound. The other process has 2 threads both of which are also > CPU bound. If the threads in the second process are PTHREAD_SCOPE_SYSTEM, > then each thread will get 33% of the CPU. If the threads in the second > process are PTHREAD_SCOPE_PROCESS, then the each process will get 50% of > the CPU. The second process will then use its own algorithm to split it's > 50% of the CPU up between it's two threads. If it divides it evenly, then > each of its' threads will end up with 25% of the CPU whereas the thread for > the first process has 50% of the CPU. The idea for this is that if you > have a system with several single-threaded processes and one process with > 1000 threads, you don't want that process to starve out all the other > processes. > I can make my application fork 800 processes and starve out all the other user. > In practice things get much hairier, but suffice it to say that libpthread > manages the scheduling of PTHREAD_SCOPE_PROCESS threads in userland, > whereas libthr would have to depend on the kernel managing that. (To some > extent libpthread needs some help from the kernel to provide this as well.) The complete solution is FSS (fair sharing schedule) found in Solaris. David Xu From owner-freebsd-threads@FreeBSD.ORG Sat Aug 5 00:10:22 2006 Return-Path: X-Original-To: freebsd-threads@hub.freebsd.org Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 63C3216A4DE for ; Sat, 5 Aug 2006 00:10:22 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2454C43D45 for ; Sat, 5 Aug 2006 00:10:22 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.4/8.13.4) with ESMTP id k750AMPT071831 for ; Sat, 5 Aug 2006 00:10:22 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.4/8.13.4/Submit) id k750AMOw071830; Sat, 5 Aug 2006 00:10:22 GMT (envelope-from gnats) Date: Sat, 5 Aug 2006 00:10:22 GMT Message-Id: <200608050010.k750AMOw071830@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: David Xu Cc: Subject: Re: threads/101355: threaded application spents too much time in _umtx_op with libthr X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 05 Aug 2006 00:10:22 -0000 The following reply was made to PR threads/101355; it has been noted by GNATS. From: David Xu To: freebsd-threads@freebsd.org Cc: Gergely Kovacs , freebsd-gnats-submit@freebsd.org Subject: Re: threads/101355: threaded application spents too much time in _umtx_op with libthr Date: Sat, 5 Aug 2006 08:00:37 +0800 On Friday 04 August 2006 19:44, Gergely Kovacs wrote: > >Number: 101355 > >Category: threads > >Synopsis: threaded application spents too much time in _umtx_op with > > libthr Confidential: no > >Severity: serious > >Priority: medium > >Responsible: freebsd-threads > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: sw-bug > >Submitter-Id: current-users > >Arrival-Date: Fri Aug 04 11:50:09 GMT 2006 > >Closed-Date: > >Last-Modified: > >Originator: Gergely Kovacs > >Release: 6.1-STABLE > >Organization: If your application is malloc hunger, then there is performance problem on 6.x since malloc is protected by single lock, you may try -CURRENT, the malloc was rewritten by Jason Evans on -CURRENT. David Xu