From owner-freebsd-threads@FreeBSD.ORG Sun Jan 23 15:16:29 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3D51916A4CE for ; Sun, 23 Jan 2005 15:16:29 +0000 (GMT) Received: from fledge.watson.org (fledge.watson.org [204.156.12.50]) by mx1.FreeBSD.org (Postfix) with ESMTP id C960743D41 for ; Sun, 23 Jan 2005 15:16:28 +0000 (GMT) (envelope-from robert@fledge.watson.org) Received: from fledge.watson.org (localhost [127.0.0.1]) by fledge.watson.org (8.13.1/8.13.1) with ESMTP id j0NFG9Yj048457; Sun, 23 Jan 2005 10:16:09 -0500 (EST) (envelope-from robert@fledge.watson.org) Received: from localhost (robert@localhost)j0NFG9xj048454; Sun, 23 Jan 2005 15:16:09 GMT (envelope-from robert@fledge.watson.org) Date: Sun, 23 Jan 2005 15:16:09 +0000 (GMT) From: Robert Watson X-Sender: robert@fledge.watson.org To: Petri Helenius In-Reply-To: <41F29E62.1000207@he.iki.fi> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: new mutex and thread stuff X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 23 Jan 2005 15:16:29 -0000 On Sat, 22 Jan 2005, Petri Helenius wrote: > Do I assume correctly that because the new thread stuff (including the > higher performance mutexes) will not be MFC'd to RELENG_5 because they > break the ABI? In general, the documented ABI boundary for applications is the libc interface, and not the kernel system call interface, so subject to some limitations during the upgrade process, there's a lot of scope for changes to the internals of thread implementations without breaking the application ABI. A nice example of this is that using libmap.conf, you can change between libthr, libpthread, and libc_r without breaking anything in the appplication. So subject to careful management of the various ABIs and interfaces, merging new thread work shouldn't be a prohibited for ABI reasons. Robert N M Watson From owner-freebsd-threads@FreeBSD.ORG Mon Jan 24 11:02:17 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8B26E16A4E7 for ; Mon, 24 Jan 2005 11:02:17 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6A8B443D1F for ; Mon, 24 Jan 2005 11:02:17 +0000 (GMT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0OB2Hxj019050 for ; Mon, 24 Jan 2005 11:02:17 GMT (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0OB2Gp8019044 for freebsd-threads@freebsd.org; Mon, 24 Jan 2005 11:02:16 GMT (envelope-from owner-bugmaster@freebsd.org) Date: Mon, 24 Jan 2005 11:02:16 GMT Message-Id: <200501241102.j0OB2Gp8019044@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 11:02:17 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2004/04/22] threads/65883threads libkse's sigwait does not work after fork o [2005/01/04] threads/75795threads applications linked with -lc_r can't clos 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2000/08/26] kern/20861 threads libc_r does not honor socket timeouts o [2001/01/20] threads/24472threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] threads/24632threads libc_r delicate deviation from libc in ha o [2001/01/25] kern/24641 threads pthread_rwlock_rdlock can deadlock o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/02/01] threads/34536threads accept() blocks other threads o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/06/27] threads/39922threads [PATCH?] Threaded applications executed w o [2002/08/04] kern/41331 threads Pthread library open sets O_NONBLOCK flag o [2003/03/02] threads/48856threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] threads/49087threads Signals lost in programs linked with libc o [2003/05/08] threads/51949threads thread in accept cannot be cancelled s [2004/03/15] kern/64313 threads FreeBSD (OpenBSD) pthread implicit set/un o [2004/08/26] threads/70975threads unexpected and unreliable behaviour when o [2004/09/14] threads/71725threads Mysql Crashes frequently giving Sock Erro o [2004/10/05] threads/72353threads Assertion fails in /usr/src/lib/libpthrea o [2004/10/07] threads/72429threads threads blocked in stdio (fgets, etc) are o [2004/10/21] threads/72953threads fork() unblocks blocked signals w/o PTHRE o [2004/11/25] threads/74370threads Cannot get lwp 0 registers in gdb o [2004/12/08] threads/74856threads dig/host broken w/ libthr o [2004/12/19] threads/75273threads FBSD 5.3 libpthread (KSE) bug o [2004/12/21] threads/75374threads pthread_kill() ignores SA_SIGINFO flag 23 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/05/26] kern/18824 threads gethostbyname is not thread safe o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2000/10/21] kern/22190 threads A threaded read(2) from a socketpair(2) f o [2001/09/09] threads/30464threads pthread mutex attributes -- pshared o [2002/05/02] threads/37676threads libc_r: msgsnd(), msgrcv(), pread(), pwri s [2002/07/16] threads/40671threads pthread_cancel doesn't remove thread from o [2004/07/13] threads/69020threads pthreads library leaks _gc_mutex o [2004/09/21] threads/71966threads Mlnet Core Dumped : Fatal error '_pq_inse o [2004/11/21] threads/74180threads KSE problem. Applications those riched ma o [2005/01/20] threads/76513threads libpthread is not working 10 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Jan 24 18:49:30 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E774C16A4CE; Mon, 24 Jan 2005 18:49:30 +0000 (GMT) Received: from creme-brulee.marcuscom.com (creme-brulee.marcuscom.com [24.172.16.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5D6ED43D39; Mon, 24 Jan 2005 18:49:30 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from shumai.marcuscom.com (shumai.marcuscom.com [192.168.1.4]) j0OInmAH043108; Mon, 24 Jan 2005 13:49:48 -0500 (EST) (envelope-from marcus@FreeBSD.org) From: Joe Marcus Clarke To: Daniel Eischen In-Reply-To: References: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-C2j5odxSikAZQJOPtbrW" Organization: FreeBSD, Inc. Date: Mon, 24 Jan 2005 13:49:08 -0500 Message-Id: <1106592548.28710.7.camel@shumai.marcuscom.com> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port cc: threads@FreeBSD.org Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 18:49:31 -0000 --=-C2j5odxSikAZQJOPtbrW Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Sat, 2005-01-22 at 13:03 -0500, Daniel Eischen wrote: > On Fri, 21 Jan 2005, Joe Marcus Clarke wrote: >=20 > > In a follow-up to the previous discussion on increasing our default > > thread stacksize, I thought I'd look at how some other BSD > > implementations do it. Mezz mentioned that he thought NetBSD had a 2 M= B > > default stacksize, so I took a look. > > > > What I found was that NetBSD doesn't have a static default stacksize. > > Instead, they use the stacksize rlimit and a PTHREAD_STACKSIZE > > environment variable to get both the initial stacksize as well as each > > thread's default stacksize. I thought this would be a really good way > > of doing things, so I ported their work to FreeBSD. >=20 > Please no. I don't want to have to set any more environment > variables or login defaults to get libpthread to work with > certain ports. No need. The default stacksize rlimit is more than enough (64 MB) to satisfy every one of the affected ports thus far. The environment variable just gives us more runtime freedom. > Even if it were added, you'd have to do it > to all thread libraries in all branches in order for ports > to remove stacksize related patches. I have no problem porting this to each thread library in each branch. Besides, you'd have to touch the same to increase the static thread stacksizes. >=20 > I thought I had increased default stack sizes, but I see that > I haven't yet. I think this method (the rlimit method) addresses one of your earlier criticisms that a static stacksize would just have to be bumped again and again. With dynamic stacksizes, we don't have to worry about this ever again. Besides, any user that lowers their rlimit stacksize below 1 MB shouldn't be expected much to work anyway. Joe >=20 --=20 Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome --=-C2j5odxSikAZQJOPtbrW Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQBB9UMkb2iPiv4Uz4cRApkZAJ9FQ8qq4nHklEf1Gf2rxmHr+y/iUgCfYur6 vXGXRj4POM93dFDZaN+tstA= =J/+N -----END PGP SIGNATURE----- --=-C2j5odxSikAZQJOPtbrW-- From owner-freebsd-threads@FreeBSD.ORG Mon Jan 24 19:31:37 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8955B16A4CE; Mon, 24 Jan 2005 19:31:37 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id C256F43D41; Mon, 24 Jan 2005 19:31:36 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0OJVZ1M015773; Mon, 24 Jan 2005 14:31:35 -0500 (EST) Date: Mon, 24 Jan 2005 14:31:35 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Joe Marcus Clarke In-Reply-To: <1106592548.28710.7.camel@shumai.marcuscom.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: threads@freebsd.org Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 19:31:37 -0000 On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: > On Sat, 2005-01-22 at 13:03 -0500, Daniel Eischen wrote: > > On Fri, 21 Jan 2005, Joe Marcus Clarke wrote: > > > > > In a follow-up to the previous discussion on increasing our default > > > thread stacksize, I thought I'd look at how some other BSD > > > implementations do it. Mezz mentioned that he thought NetBSD had a 2 MB > > > default stacksize, so I took a look. > > > > > > What I found was that NetBSD doesn't have a static default stacksize. > > > Instead, they use the stacksize rlimit and a PTHREAD_STACKSIZE > > > environment variable to get both the initial stacksize as well as each > > > thread's default stacksize. I thought this would be a really good way > > > of doing things, so I ported their work to FreeBSD. > > > > Please no. I don't want to have to set any more environment > > variables or login defaults to get libpthread to work with > > certain ports. > > No need. The default stacksize rlimit is more than enough (64 MB) to > satisfy every one of the affected ports thus far. The environment And 64MB is way too big for a default stack size... -- DE From owner-freebsd-threads@FreeBSD.ORG Mon Jan 24 19:37:36 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E5C2716A4CE; Mon, 24 Jan 2005 19:37:36 +0000 (GMT) Received: from creme-brulee.marcuscom.com (creme-brulee.marcuscom.com [24.172.16.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5A72043D1F; Mon, 24 Jan 2005 19:37:36 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from shumai.marcuscom.com (shumai.marcuscom.com [192.168.1.4]) j0OJbsrA043865; Mon, 24 Jan 2005 14:37:54 -0500 (EST) (envelope-from marcus@FreeBSD.org) From: Joe Marcus Clarke To: Daniel Eischen In-Reply-To: References: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-drvOjHciAWI/GxRE++cY" Organization: FreeBSD, Inc. Date: Mon, 24 Jan 2005 14:37:14 -0500 Message-Id: <1106595434.28710.29.camel@shumai.marcuscom.com> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port cc: threads@FreeBSD.org Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 19:37:37 -0000 --=-drvOjHciAWI/GxRE++cY Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Mon, 2005-01-24 at 14:31 -0500, Daniel Eischen wrote: > On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: >=20 > > On Sat, 2005-01-22 at 13:03 -0500, Daniel Eischen wrote: > > > On Fri, 21 Jan 2005, Joe Marcus Clarke wrote: > > > > > > > In a follow-up to the previous discussion on increasing our default > > > > thread stacksize, I thought I'd look at how some other BSD > > > > implementations do it. Mezz mentioned that he thought NetBSD had a= 2 MB > > > > default stacksize, so I took a look. > > > > > > > > What I found was that NetBSD doesn't have a static default stacksiz= e. > > > > Instead, they use the stacksize rlimit and a PTHREAD_STACKSIZE > > > > environment variable to get both the initial stacksize as well as e= ach > > > > thread's default stacksize. I thought this would be a really good = way > > > > of doing things, so I ported their work to FreeBSD. > > > > > > Please no. I don't want to have to set any more environment > > > variables or login defaults to get libpthread to work with > > > certain ports. > > > > No need. The default stacksize rlimit is more than enough (64 MB) to > > satisfy every one of the affected ports thus far. The environment >=20 > And 64MB is way too big for a default stack size... Okay, so lobby that it gets reduced in login.conf. Why should threads be given less stack than processes, especially the initial thread? Joe >=20 --=20 Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome --=-drvOjHciAWI/GxRE++cY Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQBB9U5qb2iPiv4Uz4cRAqe6AJ4g18KlhmfnJOdop3euBIy66cKttwCfbglW HbbLLDIZJdmKE9WP+Ix6ICw= =lmiP -----END PGP SIGNATURE----- --=-drvOjHciAWI/GxRE++cY-- From owner-freebsd-threads@FreeBSD.ORG Mon Jan 24 22:25:39 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D0E3D16A4CE for ; Mon, 24 Jan 2005 22:25:39 +0000 (GMT) Received: from panther.cs.ucla.edu (Panther.CS.UCLA.EDU [131.179.128.25]) by mx1.FreeBSD.org (Postfix) with ESMTP id A822343D1D for ; Mon, 24 Jan 2005 22:25:39 +0000 (GMT) (envelope-from yanyu@CS.UCLA.EDU) Received: from localhost (yanyu@localhost)j0OMPdJ08053 for ; Mon, 24 Jan 2005 14:25:39 -0800 (PST) Date: Mon, 24 Jan 2005 14:25:39 -0800 (PST) From: Yan Yu To: freebsd-threads@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Subject: seg fault on kse_release () X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 22:25:39 -0000 Hi, all, I have a newbie Q: I am trying to use creating large number of threads and allocting memory to stress the system. My user program causes SEG fault in the kernel code, kse_release () in kern_kse.c. the stack when the SEG fault happens are: #0 0x08064e54 in kse_release () #1 0x080531c4 in kse_sched_single () #2 0x00000000 in ?? () My simple program is: I have a simple function to create threads: #define NUM_THREADS 5000 #define THREADS_IN_ONE_PROCESS 5 #define BSIZE 500000 static int cc; void CreateThread(int n) { assert( n <= NUM_THREADS ); pthread_t threads[NUM_THREADS]; int rc, t; for(t=0;t < n;t++){ printf("#%d: Creating thread %d\n", cc, t); cc++; rc = pthread_create(&threads[t], NULL, PrintHello, (void *)t); if (rc){ printf("ERROR; return code from pthread_create() is %d\n", rc); } } unsigned long id; char * p = (char *) calloc(BSIZE, sizeof(char) ); if ( p == NULL ) { fprintf(stderr, "calloc error\n"); } while (1) { while (BSIZE <= (id = rand() / (RAND_MAX/BSIZE))); p[id] ++; } } void *PrintHello(void *threadid) { printf("\n%d: Hello World!\n", threadid); CreateThread(THREADS_IN_ONE_PROCESS); pthread_exit(NULL); } int main (int argc, char *argv[]) { CreateThread(THREADS_IN_ONE_PROCESS); } The SEG fault happens after creating nearly 5000 threads. and I use the default pthread.h coming w/ freeBSD 5.3 #define PTHREAD_KEYS_MAX 256 #define PTHREAD_STACK_MIN (1 << 22) #define PTHREAD_THREADS_MAX ULONG_MAX Any idea on what might happen? Many Thanks! yan From owner-freebsd-threads@FreeBSD.ORG Mon Jan 24 23:15:59 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 69A9416A4CE; Mon, 24 Jan 2005 23:15:59 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4270043D1F; Mon, 24 Jan 2005 23:15:59 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id 1DF1D7A403; Mon, 24 Jan 2005 15:15:59 -0800 (PST) Message-ID: <41F581AE.2050603@elischer.org> Date: Mon, 24 Jan 2005 15:15:58 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Joe Marcus Clarke References: <1106595434.28710.29.camel@shumai.marcuscom.com> In-Reply-To: <1106595434.28710.29.camel@shumai.marcuscom.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: Daniel Eischen cc: threads@freebsd.org Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 24 Jan 2005 23:15:59 -0000 Joe Marcus Clarke wrote: >On Mon, 2005-01-24 at 14:31 -0500, Daniel Eischen wrote: > > >>On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: >> >> >> >>>On Sat, 2005-01-22 at 13:03 -0500, Daniel Eischen wrote: >>> >>> >>>>On Fri, 21 Jan 2005, Joe Marcus Clarke wrote: >>>> >>>> >>>> >>>>>In a follow-up to the previous discussion on increasing our default >>>>>thread stacksize, I thought I'd look at how some other BSD >>>>>implementations do it. Mezz mentioned that he thought NetBSD had a 2 MB >>>>>default stacksize, so I took a look. >>>>> >>>>>What I found was that NetBSD doesn't have a static default stacksize. >>>>>Instead, they use the stacksize rlimit and a PTHREAD_STACKSIZE >>>>>environment variable to get both the initial stacksize as well as each >>>>>thread's default stacksize. I thought this would be a really good way >>>>>of doing things, so I ported their work to FreeBSD. >>>>> >>>>> >>>>Please no. I don't want to have to set any more environment >>>>variables or login defaults to get libpthread to work with >>>>certain ports. >>>> >>>> >>>No need. The default stacksize rlimit is more than enough (64 MB) to >>>satisfy every one of the affected ports thus far. The environment >>> >>> >>And 64MB is way too big for a default stack size... >> >> > >Okay, so lobby that it gets reduced in login.conf. Why should threads >be given less stack than processes, especially the initial thread? > because there may be 50 of them? (or maybe even 500?) Threaded programs are supposed to be aware that stack is a limited resource. > >Joe > > > From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 00:08:26 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1139916A4CE; Tue, 25 Jan 2005 00:08:26 +0000 (GMT) Received: from creme-brulee.marcuscom.com (creme-brulee.marcuscom.com [24.172.16.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7C59B43D54; Tue, 25 Jan 2005 00:08:25 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from shumai.marcuscom.com (shumai.marcuscom.com [192.168.1.4]) j0P08gED052965; Mon, 24 Jan 2005 19:08:42 -0500 (EST) (envelope-from marcus@FreeBSD.org) From: Joe Marcus Clarke To: Julian Elischer In-Reply-To: <41F581AE.2050603@elischer.org> References: <1106595434.28710.29.camel@shumai.marcuscom.com> <41F581AE.2050603@elischer.org> Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-ozTg/t36weCHACpytyss" Organization: FreeBSD, Inc. Date: Mon, 24 Jan 2005 19:08:00 -0500 Message-Id: <1106611680.28710.48.camel@shumai.marcuscom.com> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port cc: Daniel Eischen cc: threads@FreeBSD.org Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 00:08:26 -0000 --=-ozTg/t36weCHACpytyss Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Mon, 2005-01-24 at 15:15 -0800, Julian Elischer wrote: >=20 > Joe Marcus Clarke wrote: >=20 > >On Mon, 2005-01-24 at 14:31 -0500, Daniel Eischen wrote: > > =20 > > > >>On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: > >> > >> =20 > >> > >>>On Sat, 2005-01-22 at 13:03 -0500, Daniel Eischen wrote: > >>> =20 > >>> > >>>>On Fri, 21 Jan 2005, Joe Marcus Clarke wrote: > >>>> > >>>> =20 > >>>> > >>>>>In a follow-up to the previous discussion on increasing our default > >>>>>thread stacksize, I thought I'd look at how some other BSD > >>>>>implementations do it. Mezz mentioned that he thought NetBSD had a = 2 MB > >>>>>default stacksize, so I took a look. > >>>>> > >>>>>What I found was that NetBSD doesn't have a static default stacksize= . > >>>>>Instead, they use the stacksize rlimit and a PTHREAD_STACKSIZE > >>>>>environment variable to get both the initial stacksize as well as ea= ch > >>>>>thread's default stacksize. I thought this would be a really good w= ay > >>>>>of doing things, so I ported their work to FreeBSD. > >>>>> =20 > >>>>> > >>>>Please no. I don't want to have to set any more environment > >>>>variables or login defaults to get libpthread to work with > >>>>certain ports. > >>>> =20 > >>>> > >>>No need. The default stacksize rlimit is more than enough (64 MB) to > >>>satisfy every one of the affected ports thus far. The environment > >>> =20 > >>> > >>And 64MB is way too big for a default stack size... > >> =20 > >> > > > >Okay, so lobby that it gets reduced in login.conf. Why should threads > >be given less stack than processes, especially the initial thread? > > >=20 > because there may be 50 of them? (or maybe even 500?) >=20 > Threaded programs are supposed to be aware that stack is a limited resour= ce. I thought about that, but I also thought that KSEs were treated similarly to processes so it wouldn't matter if each one had a full allocation of stacksize? Joe >=20 >=20 > > > >Joe > > > > =20 > > >=20 >=20 --=20 Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome --=-ozTg/t36weCHACpytyss Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQBB9Y3gb2iPiv4Uz4cRAsWQAKCpnxe0tuzAdwxtt5fCw+WawV5QlgCfZfJL 6SYfNDUD+3vCTuAYgHofmNY= =CQcJ -----END PGP SIGNATURE----- --=-ozTg/t36weCHACpytyss-- From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 00:24:24 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1F98116A4CE; Tue, 25 Jan 2005 00:24:24 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id A7F4D43D2F; Tue, 25 Jan 2005 00:24:23 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0P0OCgm029982; Mon, 24 Jan 2005 19:24:12 -0500 (EST) Date: Mon, 24 Jan 2005 19:24:04 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Joe Marcus Clarke In-Reply-To: <1106611680.28710.48.camel@shumai.marcuscom.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: threads@freebsd.org cc: Julian Elischer Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 00:24:24 -0000 On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: > On Mon, 2005-01-24 at 15:15 -0800, Julian Elischer wrote: > > > > >> > > > > > >Okay, so lobby that it gets reduced in login.conf. Why should threads > > >be given less stack than processes, especially the initial thread? > > > > > > > because there may be 50 of them? (or maybe even 500?) > > > > Threaded programs are supposed to be aware that stack is a limited resource. > > I thought about that, but I also thought that KSEs were treated > similarly to processes so it wouldn't matter if each one had a full > allocation of stacksize? KSE != thread A (userland) KSE stack is very small and is just enough to schedule threads. A thread stack is allocated (by default) off the one (and only one) process' stack. Allocating lots of threads with large stacks depletes the process stack. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 00:35:05 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 33C2B16A4CE; Tue, 25 Jan 2005 00:35:05 +0000 (GMT) Received: from creme-brulee.marcuscom.com (creme-brulee.marcuscom.com [24.172.16.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id A00F943D53; Tue, 25 Jan 2005 00:35:04 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from shumai.marcuscom.com (shumai.marcuscom.com [192.168.1.4]) j0P0ZKZW053446; Mon, 24 Jan 2005 19:35:20 -0500 (EST) (envelope-from marcus@FreeBSD.org) From: Joe Marcus Clarke To: Daniel Eischen In-Reply-To: References: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-3UCwqG9roVlPB2r+giyU" Organization: FreeBSD, Inc. Date: Mon, 24 Jan 2005 19:34:39 -0500 Message-Id: <1106613279.28710.63.camel@shumai.marcuscom.com> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port cc: threads@FreeBSD.org cc: Julian Elischer Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 00:35:05 -0000 --=-3UCwqG9roVlPB2r+giyU Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Mon, 2005-01-24 at 19:24 -0500, Daniel Eischen wrote: > On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: >=20 > > On Mon, 2005-01-24 at 15:15 -0800, Julian Elischer wrote: > > > > > > >> > > > > > > > >Okay, so lobby that it gets reduced in login.conf. Why should threa= ds > > > >be given less stack than processes, especially the initial thread? > > > > > > > > > > because there may be 50 of them? (or maybe even 500?) > > > > > > Threaded programs are supposed to be aware that stack is a limited re= source. > > > > I thought about that, but I also thought that KSEs were treated > > similarly to processes so it wouldn't matter if each one had a full > > allocation of stacksize? >=20 > KSE !=3D thread >=20 > A (userland) KSE stack is very small and is just enough to > schedule threads. A thread stack is allocated (by default) > off the one (and only one) process' stack. Allocating lots > of threads with large stacks depletes the process stack. Ah, okay, I suspected that was the case for libc_r, but I wasn't sure if the same thing held for all threading libraries. What about increasing the default stack sizes as you've said you wanted to do, plus leaving in the environment variable to aid in transition should the stack size have to be bumped again in the future? This would also give us an easy way to test for stack overflows without instructing users to rebuild their threading library. Also, what were your planned stacksize increments? I was hoping for something along the lines of: INITIAL (32-bit): 2 MB INITIAL (64-bit) 4 MB DEFAULT (32-bit): 1 MB DEFAULT (64-bit): 2 MB Joe >=20 --=20 Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome --=-3UCwqG9roVlPB2r+giyU Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQBB9ZQfb2iPiv4Uz4cRAjzxAKCBMXGXM7D5rUwDEnAumpxkZxOD3ACfVced AtGeXhJc6I1biv5TcZBINIE= =UKcx -----END PGP SIGNATURE----- --=-3UCwqG9roVlPB2r+giyU-- From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 00:39:46 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8077116A4CE; Tue, 25 Jan 2005 00:39:46 +0000 (GMT) Received: from mail.vicor-nb.com (bigwoop.vicor-nb.com [208.206.78.2]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5185743D39; Tue, 25 Jan 2005 00:39:46 +0000 (GMT) (envelope-from julian@elischer.org) Received: from elischer.org (julian.vicor-nb.com [208.206.78.97]) by mail.vicor-nb.com (Postfix) with ESMTP id 3C96F7A403; Mon, 24 Jan 2005 16:39:46 -0800 (PST) Message-ID: <41F59552.1040302@elischer.org> Date: Mon, 24 Jan 2005 16:39:46 -0800 From: Julian Elischer User-Agent: Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.3.1) Gecko/20030516 X-Accept-Language: en, hu MIME-Version: 1.0 To: Joe Marcus Clarke References: <1106595434.28710.29.camel@shumai.marcuscom.com> <41F581AE.2050603@elischer.org> <1106611680.28710.48.camel@shumai.marcuscom.com> In-Reply-To: <1106611680.28710.48.camel@shumai.marcuscom.com> Content-Type: text/plain; charset=us-ascii; format=flowed Content-Transfer-Encoding: 7bit cc: Daniel Eischen cc: threads@freebsd.org Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 00:39:46 -0000 Joe Marcus Clarke wrote: >On Mon, 2005-01-24 at 15:15 -0800, Julian Elischer wrote: > > >>Joe Marcus Clarke wrote: >> >> >> >>>On Mon, 2005-01-24 at 14:31 -0500, Daniel Eischen wrote: >>> >>> >>> >>> >>>>On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: >>>> >>>> >>>> >>>> >>>> >>>>>On Sat, 2005-01-22 at 13:03 -0500, Daniel Eischen wrote: >>>>> >>>>> >>>>> >>>>> >>>>>>On Fri, 21 Jan 2005, Joe Marcus Clarke wrote: >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>>>In a follow-up to the previous discussion on increasing our default >>>>>>>thread stacksize, I thought I'd look at how some other BSD >>>>>>>implementations do it. Mezz mentioned that he thought NetBSD had a 2 MB >>>>>>>default stacksize, so I took a look. >>>>>>> >>>>>>>What I found was that NetBSD doesn't have a static default stacksize. >>>>>>>Instead, they use the stacksize rlimit and a PTHREAD_STACKSIZE >>>>>>>environment variable to get both the initial stacksize as well as each >>>>>>>thread's default stacksize. I thought this would be a really good way >>>>>>>of doing things, so I ported their work to FreeBSD. >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>Please no. I don't want to have to set any more environment >>>>>>variables or login defaults to get libpthread to work with >>>>>>certain ports. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>No need. The default stacksize rlimit is more than enough (64 MB) to >>>>>satisfy every one of the affected ports thus far. The environment >>>>> >>>>> >>>>> >>>>> >>>>And 64MB is way too big for a default stack size... >>>> >>>> >>>> >>>> >>>Okay, so lobby that it gets reduced in login.conf. Why should threads >>>be given less stack than processes, especially the initial thread? >>> >>> >>> >>because there may be 50 of them? (or maybe even 500?) >> >>Threaded programs are supposed to be aware that stack is a limited resource. >> >> > >I thought about that, but I also thought that KSEs were treated >similarly to processes so it wouldn't matter if each one had a full >allocation of stacksize? > we're talking about he stacksize in userland.. and every thread has to have a separae one, and they have to all fit in the same address space.. 64 MB is what, 16 threads per GB of addresss space? if you take into account that there a re grat chunks of address space not available, then you really would be limiting yourself to about 32 threads.. It doesn't take a lot of imagination to see people using more than 32 threads. > >Joe > > > >> >> >>>Joe >>> >>> >>> >>> >>> >> >> From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 00:41:21 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 750AC16A4CE; Tue, 25 Jan 2005 00:41:21 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0714343D46; Tue, 25 Jan 2005 00:41:21 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0P0fHVF018741; Mon, 24 Jan 2005 19:41:17 -0500 (EST) Date: Mon, 24 Jan 2005 19:41:17 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Joe Marcus Clarke In-Reply-To: <1106613279.28710.63.camel@shumai.marcuscom.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: threads@freebsd.org cc: Julian Elischer Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 00:41:21 -0000 On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: > Ah, okay, I suspected that was the case for libc_r, but I wasn't sure if > the same thing held for all threading libraries. > > What about increasing the default stack sizes as you've said you wanted > to do, plus leaving in the environment variable to aid in transition > should the stack size have to be bumped again in the future? This would I don't want an environment variable :-) > also give us an easy way to test for stack overflows without instructing > users to rebuild their threading library. > > Also, what were your planned stacksize increments? I was hoping for > something along the lines of: > > INITIAL (32-bit): 2 MB > INITIAL (64-bit) 4 MB I think I was going to make the initial bigger than that (I forgot what I chose). > DEFAULT (32-bit): 1 MB > DEFAULT (64-bit): 2 MB Yes, I think that's what I was planning for other-than-initial threads. -- DE From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 00:44:42 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9248F16A4CE; Tue, 25 Jan 2005 00:44:42 +0000 (GMT) Received: from creme-brulee.marcuscom.com (creme-brulee.marcuscom.com [24.172.16.118]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2711043D39; Tue, 25 Jan 2005 00:44:42 +0000 (GMT) (envelope-from marcus@FreeBSD.org) Received: from shumai.marcuscom.com (shumai.marcuscom.com [192.168.1.4]) j0P0iwBR053817; Mon, 24 Jan 2005 19:44:58 -0500 (EST) (envelope-from marcus@FreeBSD.org) From: Joe Marcus Clarke To: Daniel Eischen In-Reply-To: References: Content-Type: multipart/signed; micalg=pgp-sha1; protocol="application/pgp-signature"; boundary="=-068Svqo9jmWrP866ZrEj" Organization: FreeBSD, Inc. Date: Mon, 24 Jan 2005 19:44:17 -0500 Message-Id: <1106613857.28710.66.camel@shumai.marcuscom.com> Mime-Version: 1.0 X-Mailer: Evolution 2.0.3 FreeBSD GNOME Team Port cc: threads@FreeBSD.org cc: Julian Elischer Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 00:44:42 -0000 --=-068Svqo9jmWrP866ZrEj Content-Type: text/plain Content-Transfer-Encoding: quoted-printable On Mon, 2005-01-24 at 19:41 -0500, Daniel Eischen wrote: > On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: >=20 > > Ah, okay, I suspected that was the case for libc_r, but I wasn't sure i= f > > the same thing held for all threading libraries. > > > > What about increasing the default stack sizes as you've said you wanted > > to do, plus leaving in the environment variable to aid in transition > > should the stack size have to be bumped again in the future? This woul= d >=20 > I don't want an environment variable :-) Why? I've listed two good reasons for having some way of dynamically tuning thread stacks. What are the downsides? >=20 > > also give us an easy way to test for stack overflows without instructin= g > > users to rebuild their threading library. > > > > Also, what were your planned stacksize increments? I was hoping for > > something along the lines of: > > > > INITIAL (32-bit): 2 MB > > INITIAL (64-bit) 4 MB >=20 > I think I was going to make the initial bigger than that (I forgot > what I chose). >=20 > > DEFAULT (32-bit): 1 MB > > DEFAULT (64-bit): 2 MB >=20 > Yes, I think that's what I was planning for other-than-initial threads. When do you plan to commit the changes? Joe >=20 --=20 Joe Marcus Clarke FreeBSD GNOME Team :: gnome@FreeBSD.org FreeNode / #freebsd-gnome http://www.FreeBSD.org/gnome --=-068Svqo9jmWrP866ZrEj Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.0 (FreeBSD) iD8DBQBB9ZZhb2iPiv4Uz4cRAi4cAJwI1gmOiGjmhD3XCNyJCzY1WM6eCgCglHQ0 cVY2fGkPGX8w09vnY7Dqegc= =RE6w -----END PGP SIGNATURE----- --=-068Svqo9jmWrP866ZrEj-- From owner-freebsd-threads@FreeBSD.ORG Tue Jan 25 01:36:28 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5055016A4CE; Tue, 25 Jan 2005 01:36:28 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id E55EF43D1F; Tue, 25 Jan 2005 01:36:27 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0P1aP7h028055; Mon, 24 Jan 2005 20:36:25 -0500 (EST) Date: Mon, 24 Jan 2005 20:36:25 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Joe Marcus Clarke In-Reply-To: <1106613857.28710.66.camel@shumai.marcuscom.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: threads@freebsd.org cc: Julian Elischer Subject: Re: [PATCH] Dynamic thread stack size X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 25 Jan 2005 01:36:28 -0000 On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: > On Mon, 2005-01-24 at 19:41 -0500, Daniel Eischen wrote: > > On Mon, 24 Jan 2005, Joe Marcus Clarke wrote: > > > > > Ah, okay, I suspected that was the case for libc_r, but I wasn't sure if > > > the same thing held for all threading libraries. > > > > > > What about increasing the default stack sizes as you've said you wanted > > > to do, plus leaving in the environment variable to aid in transition > > > should the stack size have to be bumped again in the future? This would > > > > I don't want an environment variable :-) > > Why? I've listed two good reasons for having some way of dynamically > tuning thread stacks. What are the downsides? Because I don't want anyone to have to rely on environment variables to get things to work. There's already a POSIX way to set stacks which should be used if you want to use something other than default. That's what should be used, not an environment variable. > > > > > > INITIAL (32-bit): 2 MB > > > INITIAL (64-bit) 4 MB > > > > I think I was going to make the initial bigger than that (I forgot > > what I chose). > > > > > DEFAULT (32-bit): 1 MB > > > DEFAULT (64-bit): 2 MB > > > > Yes, I think that's what I was planning for other-than-initial threads. > > When do you plan to commit the changes? As soon as I can. -- DE From owner-freebsd-threads@FreeBSD.ORG Wed Jan 26 00:50:08 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 013A616A4CF for ; Wed, 26 Jan 2005 00:50:08 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id AFF5743D58 for ; Wed, 26 Jan 2005 00:50:07 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0Q0o7M3037193 for ; Wed, 26 Jan 2005 00:50:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0Q0o708037187; Wed, 26 Jan 2005 00:50:07 GMT (envelope-from gnats) Resent-Date: Wed, 26 Jan 2005 00:50:07 GMT Resent-Message-Id: <200501260050.j0Q0o708037187@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Serguei Leontiev Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DBAD816A4CE for ; Wed, 26 Jan 2005 00:48:25 +0000 (GMT) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id A760743D3F for ; Wed, 26 Jan 2005 00:48:25 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j0Q0mP8V083525 for ; Wed, 26 Jan 2005 00:48:25 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id j0Q0mPpd083524; Wed, 26 Jan 2005 00:48:25 GMT (envelope-from nobody) Message-Id: <200501260048.j0Q0mPpd083524@www.freebsd.org> Date: Wed, 26 Jan 2005 00:48:25 GMT From: Serguei Leontiev To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Subject: threads/76690: fork hang in child for (-lc_r & -lthr) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 00:50:08 -0000 >Number: 76690 >Category: threads >Synopsis: fork hang in child for (-lc_r & -lthr) >Confidential: no >Severity: critical >Priority: medium >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Jan 26 00:50:07 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Serguei Leontiev >Release: 5.2.1 >Organization: Crypto-Pro >Environment: FreeBSD build-fbsd 5.2.1-RELEASE FreeBSD 5.2.1-RELEASE #0: Mon Feb 23 20:45:55 GMT 2004 root@wv1u.btc.adaptec.com:/usr/obj/usr/src/sys/GENERIC i386 >Description: For multithreaded application malloc/free in other thread cause fork() hang in child process. Child process does not anything to work or threading operations. This bug first detected for system() library function in multithreaded application. This bug affected "-lc_r" and "-lthr" library. Library "-lkse" seems OK. Sorry for my bests English. >How-To-Repeat: #include #include #include #include #include #include #include const int INFL = 1000000000; const int PTHR = 4; const int NATR = 2; #ifndef SEMI_OK // if SEMI_OK not defined - >95% hang on my system const int FRKL = 100; #else // if SEMI_OK defined - 50% hang on my system const int FRKL = 3; #endif void *malloc_free(void *pvcnt) { volatile sig_atomic_t *pscnt = (volatile sig_atomic_t *)pvcnt; int n = *pscnt; int i; void *m; for(i = 0; i < n; i++){ m = malloc(10); *pscnt = i; free(m); } return NULL; } int main (void) { pthread_attr_t attrs[NATR]; pthread_t thread_id; volatile sig_atomic_t cnt[PTHR]; int i; pid_t pid, savedpid; int pstat; pthread_attr_init(&attrs[0]); pthread_attr_setdetachstate(&attrs[0], PTHREAD_CREATE_DETACHED); pthread_attr_setscope(&attrs[0], PTHREAD_SCOPE_PROCESS); pthread_attr_init(&attrs[1]); pthread_attr_setdetachstate(&attrs[1], PTHREAD_CREATE_DETACHED); pthread_attr_setscope(&attrs[1], PTHREAD_SCOPE_SYSTEM); for(i = 0; i < PTHR; i++) { cnt[i] = INFL; if(pthread_create(&thread_id, &attrs[i%NATR], &malloc_free, (void *)&cnt[i])){ perror("pthread_create:"); return 1; } } fprintf(stderr, "Threads created.\n"); for (i = 0; i < FRKL; i++) { fprintf(stderr, "forking\n"); switch(pid = fork()) { case -1: /* error */ perror("fork fail:"); return 2; case 0: /* child */ // Child don't anything to work - simple exit // When child hang, this hang in fork code _exit(0); default: /* parent */ savedpid = pid; do{ pid = waitpid(savedpid, &pstat, 0); }while(pid == -1 && errno == EINTR); break; } } fprintf(stderr, "Threads OK:"); for(i = 0; i < PTHR; i++){ fprintf(stderr, " %d", (int)cnt[i]); } fprintf(stderr, "\n"); _exit(0); return 0; } >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Wed Jan 26 01:20:20 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 711F716A4CF for ; Wed, 26 Jan 2005 01:20:20 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 26C6743D41 for ; Wed, 26 Jan 2005 01:20:20 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0Q1KKXS042015 for ; Wed, 26 Jan 2005 01:20:20 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0Q1KJFb042014; Wed, 26 Jan 2005 01:20:19 GMT (envelope-from gnats) Resent-Date: Wed, 26 Jan 2005 01:20:19 GMT Resent-Message-Id: <200501260120.j0Q1KJFb042014@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Serguei Leontiev Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7B47516A4CE for ; Wed, 26 Jan 2005 01:11:41 +0000 (GMT) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BAD443D39 for ; Wed, 26 Jan 2005 01:11:41 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j0Q1Bf2A086740 for ; Wed, 26 Jan 2005 01:11:41 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id j0Q1Bf8G086739; Wed, 26 Jan 2005 01:11:41 GMT (envelope-from nobody) Message-Id: <200501260111.j0Q1Bf8G086739@www.freebsd.org> Date: Wed, 26 Jan 2005 01:11:41 GMT From: Serguei Leontiev To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Subject: threads/76694: fork cause hang in dup()/close() function in child (-lc_r) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 01:20:20 -0000 >Number: 76694 >Category: threads >Synopsis: fork cause hang in dup()/close() function in child (-lc_r) >Confidential: no >Severity: serious >Priority: medium >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Wed Jan 26 01:20:19 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Serguei Leontiev >Release: 5.2.1 >Organization: Crypto-Pro >Environment: FreeBSD build-fbsd 5.2.1-RELEASE FreeBSD 5.2.1-RELEASE #0: Mon Feb 23 20:45:55 GMT 2004 root@wv1u.btc.adaptec.com:/usr/obj/usr/src/sys/GENERIC i386 >Description: For multithreaded application dup()/close() in other thread cause hang dup()/close() in child process after fork. Child do not use thread related operations. This bug first detected for accept() library function after daemon(). May be daemon() not thread-safe, but dup()/close() - MUST thread-safe by POSIX, and MUST compatible with fork() anywhere. This bug affected "-lc_r" library. Library "-lthr" & "-lkse" seems OK. Sorry for my bests English. >How-To-Repeat: #include #include #include #include #include #include #include const int INFL = 1000000000; const int PTHR = 4; const int NATR = 2; #ifndef SEMI_OK // if SEMI_OK not defined - >95% hang on my system const int FRKL = 100; const int CHLD = 100; #else // if SEMI_OK defined - 50% hang on my system const int FRKL = 10; const int CHLD = 10; #endif void *test_write(void *pvcnt) { volatile sig_atomic_t *pscnt = (volatile sig_atomic_t *)pvcnt; int n = *pscnt; int i; int fd; for(i = 0; i < n; i++){ if(0 > (fd = dup(STDIN_FILENO))){ perror("dup:"); continue; } *pscnt = i; if(0 > close(fd)){ perror("close:"); } } return NULL; } int main (void) { pthread_attr_t attrs[NATR]; pthread_t thread_id; volatile sig_atomic_t cnt[PTHR]; int i; int cntr; pid_t pid, savedpid; int pstat; pthread_attr_init(&attrs[0]); pthread_attr_setdetachstate(&attrs[0], PTHREAD_CREATE_DETACHED); pthread_attr_setscope(&attrs[0], PTHREAD_SCOPE_PROCESS); pthread_attr_init(&attrs[1]); pthread_attr_setdetachstate(&attrs[1], PTHREAD_CREATE_DETACHED); pthread_attr_setscope(&attrs[1], PTHREAD_SCOPE_SYSTEM); for(i = 0; i < PTHR; i++) { cnt[i] = INFL; if(pthread_create(&thread_id, &attrs[i%NATR], &test_write, (void *)&cnt[i])){ perror("pthread_create:"); return 1; } } fprintf(stderr, "Threads created.\n"); for (i = 0; i < FRKL; i++) { fprintf(stderr, "forking\n"); switch(pid = fork()) { case -1: /* error */ perror("fork fail:"); return 2; case 0: /* child */ // Child don't use thread related operations // Only dup() & close() cntr = CHLD; test_write(&cntr); _exit(0); default: /* parent */ savedpid = pid; do{ pid = waitpid(savedpid, &pstat, 0); }while(pid == -1 && errno == EINTR); break; } } fprintf(stderr, "Threads OK:"); for(i = 0; i < PTHR; i++){ fprintf(stderr, " %d", (int)cnt[i]); } fprintf(stderr, "\n"); _exit(0); return 0; } >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Wed Jan 26 12:20:25 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 043FA16A4CE for ; Wed, 26 Jan 2005 12:20:25 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id CD86143D41 for ; Wed, 26 Jan 2005 12:20:24 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0QCKOS0059485 for ; Wed, 26 Jan 2005 12:20:24 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0QCKOBc059484; Wed, 26 Jan 2005 12:20:24 GMT (envelope-from gnats) Date: Wed, 26 Jan 2005 12:20:24 GMT Message-Id: <200501261220.j0QCKOBc059484@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: =?koi8-r?B?7MXPztTYxdcg88XSx8XKIOXGyc3P18ne?= Subject: Re: threads/76690: fork hang in child for (-lc_r & -lthr) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: =?koi8-r?B?7MXPztTYxdcg88XSx8XKIOXGyc3P18ne?= List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 12:20:25 -0000 The following reply was made to PR threads/76690; it has been noted by GNATS. From: =?koi8-r?B?7MXPztTYxdcg88XSx8XKIOXGyc3P18ne?= To: , =?koi8-r?B?7MXPztTYxdcg88XSx8XKIOXGyc3P18ne?= Cc: Subject: Re: threads/76690: fork hang in child for (-lc_r & -lthr) Date: Wed, 26 Jan 2005 15:18:38 +0300 Hi, Sorry, program in section "How-To-Repeat" hang only on "-lc_r" library. Hang for library -lthr cause by malloc()/free() in child, not fork() = internally. "Correct" version "How-To-Repeat" program: # This is a shell archive. Save it in a file, remove anything before # this line, and then unpack it by entering "sh file". Note, it may # create directories; files and directories will be owned by you and # have default permissions. # # This archive contains: # # fork-malloc-free.c # echo x - fork-malloc-free.c sed 's/^X//' >fork-malloc-free.c << 'END-of-fork-malloc-free.c' X#include X#include X#include X#include X#include X#include X#include X Xconst int INFL =3D 1000000000; Xconst int PTHR =3D 4; Xconst int NATR =3D 2; X X#ifndef SEMI_OK // if SEMI_OK not defined - >95% hang on = my syst em Xconst int FRKL =3D 100; Xconst int CHLD =3D 100; X#else // if SEMI_OK defined - 50% hang on my system Xconst int FRKL =3D 3; Xconst int CHLD =3D 3; X#endif X Xvoid *malloc_free(void *pvcnt) X{ X volatile sig_atomic_t *pscnt =3D (volatile sig_atomic_t *)pvcnt; X int n =3D *pscnt; X int i; X void *m; X X for(i =3D 0; i < n; i++){ X m =3D malloc(10); X *pscnt =3D i; X free(m); X } X return NULL; X} X Xint main (void) X{ X pthread_attr_t attrs[NATR]; X pthread_t thread_id; X volatile sig_atomic_t cnt[PTHR]; X int i; X int cntr; X pid_t pid, savedpid; X int pstat; X X pthread_attr_init(&attrs[0]); X pthread_attr_setdetachstate(&attrs[0], PTHREAD_CREATE_DETACHED); X pthread_attr_setscope(&attrs[0], PTHREAD_SCOPE_PROCESS); X pthread_attr_init(&attrs[1]); X pthread_attr_setdetachstate(&attrs[1], PTHREAD_CREATE_DETACHED); X pthread_attr_setscope(&attrs[1], PTHREAD_SCOPE_SYSTEM); X for(i =3D 0; i < PTHR; i++) { X cnt[i] =3D INFL; X if(pthread_create(&thread_id, &attrs[i%NATR], X &malloc_free, (void *)&cnt[i])){ X perror("pthread_create:"); X return 1; X } X } X fprintf(stderr, "Threads created.\n"); X X for (i =3D 0; i < FRKL; i++) { X fprintf(stderr, "forking\n"); X switch(pid =3D fork()) { X case -1: /* error */ X perror("fork fail:"); X return 2; X case 0: /* child */ X // -lc_r: X // Child don't anything to work - simple exit X // When child hang, this hang in fork code X // X // -lthr: X // Child hang by malloc()/free(); X cntr =3D CHLD; X malloc_free(&cntr); X _exit(0); X default: /* parent */ X savedpid =3D pid; X do{ X pid =3D waitpid(savedpid, &pstat, 0); X }while(pid =3D=3D -1 && errno =3D=3D EINTR); X break; X } X } X fprintf(stderr, "Threads OK:"); X for(i =3D 0; i < PTHR; i++){ X fprintf(stderr, " %d", (int)cnt[i]); X } X fprintf(stderr, "\n"); X _exit(0); X return 0; X} END-of-fork-malloc-free.c exit -- Sorry for my bests English. Serguei E. Leontiev w:+7(095)289-4367 USSR, Moscow, 127018, Obraztsova = 38 Crypto-Pro=9A=9A=9A=9A=9A=9A=9A=9A=9A w:+7(095)933-1168 m:+7(916)686-1081 SMS: =9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A = p:+7(095)231-3838 for abonent +7(916)686-1081 =9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A = h:+7(095)318-1146 USSR, Moscow, 113303, Kakhovka 6-40 =9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A = w:+7(095)939-2382 USSR, Moscow, Universitetskij 13 =9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A=9A From owner-freebsd-threads@FreeBSD.ORG Wed Jan 26 21:08:57 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B767816A4CE for ; Wed, 26 Jan 2005 21:08:57 +0000 (GMT) Received: from bute.st-andrews.ac.uk (bute.st-and.ac.uk [138.251.12.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6FEAF43D39 for ; Wed, 26 Jan 2005 21:08:56 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by bute.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id VAA26836 for ; Wed, 26 Jan 2005 21:05:57 GMT From: "Niall Douglas" To: freebsd-threads@freebsd.org Date: Wed, 26 Jan 2005 21:08:42 -0000 MIME-Version: 1.0 Message-ID: <41F806DA.5314.3ABF7764@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT Subject: Is select() a thread cancellation point? X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Jan 2005 21:08:57 -0000 According to: http://lists.freebsd.org/pipermail/freebsd-threads/2004- October/002572.html ... it is. But v5.3 FreeBSD doesn't cancel during select(), nor during recv(). OTOH I have other documentation which suggests that neither select() nor recv() are cancellation points. Oh and on Linux, both are cancellation points. In my mind it's probably more useful if they are cancellation points as you can always disable cancellation around them if necessary whereas the opposite is not true. Cheers, Niall From owner-freebsd-threads@FreeBSD.ORG Thu Jan 27 00:40:25 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 126A116A4CE for ; Thu, 27 Jan 2005 00:40:25 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C476E43D39 for ; Thu, 27 Jan 2005 00:40:24 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0R0eOiT048185 for ; Thu, 27 Jan 2005 00:40:24 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0R0eOFh048184; Thu, 27 Jan 2005 00:40:24 GMT (envelope-from gnats) Date: Thu, 27 Jan 2005 00:40:24 GMT Message-Id: <200501270040.j0R0eOFh048184@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Julian Elischer Subject: Re: threads/76694: fork cause hang in dup()/close() function inchild (-lc_r) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Julian Elischer List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 00:40:25 -0000 The following reply was made to PR threads/76694; it has been noted by GNATS. From: Julian Elischer To: Serguei Leontiev Cc: freebsd-gnats-submit@freebsd.org Subject: Re: threads/76694: fork cause hang in dup()/close() function in child (-lc_r) Date: Wed, 26 Jan 2005 16:30:29 -0800 this may seem harsh, but 5.2.1 was still an experimental branch.. you should upgrade to 5.3 where lkse is the defualt (called libpthread). libc_r will not be "actively" maintained after we settle on a kernel supported threading package. Serguei Leontiev wrote: >>Number: 76694 >>Category: threads >>Synopsis: fork cause hang in dup()/close() function in child (-lc_r) >>Confidential: no >>Severity: serious >>Priority: medium >>Responsible: freebsd-threads >>State: open >>Quarter: >>Keywords: >>Date-Required: >>Class: sw-bug >>Submitter-Id: current-users >>Arrival-Date: Wed Jan 26 01:20:19 GMT 2005 >>Closed-Date: >>Last-Modified: >>Originator: Serguei Leontiev >>Release: 5.2.1 >>Organization: >> >> >Crypto-Pro > > >>Environment: >> >> >FreeBSD build-fbsd 5.2.1-RELEASE FreeBSD 5.2.1-RELEASE #0: Mon Feb 23 20:45:55 GMT 2004 root@wv1u.btc.adaptec.com:/usr/obj/usr/src/sys/GENERIC i386 > > > >>Description: >> >> >For multithreaded application dup()/close() in other thread cause hang dup()/close() in child process after fork. Child do not use thread related operations. > >This bug first detected for accept() library function after daemon(). May be daemon() not thread-safe, but dup()/close() - MUST thread-safe by POSIX, and MUST compatible with fork() anywhere. > >This bug affected "-lc_r" library. Library "-lthr" & "-lkse" seems OK. > >Sorry for my bests English. > > > >>How-To-Repeat: >> >> > #include >#include >#include >#include >#include >#include >#include > >const int INFL = 1000000000; >const int PTHR = 4; >const int NATR = 2; > >#ifndef SEMI_OK // if SEMI_OK not defined - >95% hang on my system >const int FRKL = 100; >const int CHLD = 100; >#else // if SEMI_OK defined - 50% hang on my system >const int FRKL = 10; >const int CHLD = 10; >#endif > >void *test_write(void *pvcnt) >{ > volatile sig_atomic_t *pscnt = (volatile sig_atomic_t *)pvcnt; > int n = *pscnt; > int i; > int fd; > > for(i = 0; i < n; i++){ > if(0 > (fd = dup(STDIN_FILENO))){ > perror("dup:"); > continue; > } > *pscnt = i; > if(0 > close(fd)){ > perror("close:"); > } > } > return NULL; >} > >int main (void) >{ > pthread_attr_t attrs[NATR]; > pthread_t thread_id; > volatile sig_atomic_t cnt[PTHR]; > int i; > int cntr; > pid_t pid, savedpid; > int pstat; > > pthread_attr_init(&attrs[0]); > pthread_attr_setdetachstate(&attrs[0], PTHREAD_CREATE_DETACHED); > pthread_attr_setscope(&attrs[0], PTHREAD_SCOPE_PROCESS); > pthread_attr_init(&attrs[1]); > pthread_attr_setdetachstate(&attrs[1], PTHREAD_CREATE_DETACHED); > pthread_attr_setscope(&attrs[1], PTHREAD_SCOPE_SYSTEM); > for(i = 0; i < PTHR; i++) { > cnt[i] = INFL; > if(pthread_create(&thread_id, &attrs[i%NATR], > &test_write, (void *)&cnt[i])){ > perror("pthread_create:"); > return 1; > } > } > fprintf(stderr, "Threads created.\n"); > > for (i = 0; i < FRKL; i++) { > fprintf(stderr, "forking\n"); > switch(pid = fork()) { > case -1: /* error */ > perror("fork fail:"); > return 2; > case 0: /* child */ > // Child don't use thread related operations > // Only dup() & close() > cntr = CHLD; > test_write(&cntr); > _exit(0); > default: /* parent */ > savedpid = pid; > do{ > pid = waitpid(savedpid, &pstat, 0); > }while(pid == -1 && errno == EINTR); > break; > } > } > fprintf(stderr, "Threads OK:"); > for(i = 0; i < PTHR; i++){ > fprintf(stderr, " %d", (int)cnt[i]); > } > fprintf(stderr, "\n"); > _exit(0); > return 0; >} > > > >>Fix: >> >> > > > >>Release-Note: >>Audit-Trail: >>Unformatted: >> >> >_______________________________________________ >freebsd-threads@freebsd.org mailing list >http://lists.freebsd.org/mailman/listinfo/freebsd-threads >To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > > From owner-freebsd-threads@FreeBSD.ORG Thu Jan 27 22:33:58 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EBBAB16A4CE for ; Thu, 27 Jan 2005 22:33:58 +0000 (GMT) Received: from hadar.amcc.com (hadar.amcc.com [192.195.69.168]) by mx1.FreeBSD.org (Postfix) with ESMTP id 993F043D2D for ; Thu, 27 Jan 2005 22:33:58 +0000 (GMT) (envelope-from mmontaseri@amcc.com) Received: from [10.66.12.100] (medi.amcc.com [10.66.12.100]) by hadar.amcc.com (Netscape Messaging Server 4.15) with ESMTP id IAZYOQ03.HWA for ; Thu, 27 Jan 2005 14:34:02 -0800 Message-ID: <41F96C25.9070906@amcc.com> Date: Thu, 27 Jan 2005 14:33:09 -0800 From: "Medi Montaseri" User-Agent: Mozilla Thunderbird 1.0 (X11/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: freebsd-threads@freebsd.org Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit Subject: pthread_t in 5.3 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 27 Jan 2005 22:33:59 -0000 I am confused about the pthread_t type in FreeBSD 5.3, can you help.. Based on /usr/include/pthread.h, typedef struct pthread *pthread_t; and when I look for the declaration of "struct pthread" , all I find is a forward declaration with a comment that says, /* * Forward structure definitions. * * These are mostly opaque to the user. */ struct pthread; That is 'struct pthread' is an opaque type.... Then in my application, when I try to find my thread_id, I say cout << "my tid is " << pthread_self() << endl; and I get a hex value. Which is syntactically correct, but semantically in-correct. I'm not interested in the pointer, I'm interested in the numerical thread ID... Now at this point, you'll think all you have to do is to de-reference the pointer. But since 'struct pthrad' is opaque, gdb and myself are clueless to proceed from here. Can someone shed some light on this please... Thanks From owner-freebsd-threads@FreeBSD.ORG Fri Jan 28 00:22:38 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC9C616A4CE for ; Fri, 28 Jan 2005 00:22:38 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6DCCB43D46 for ; Fri, 28 Jan 2005 00:22:38 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0S0MV6d013574; Thu, 27 Jan 2005 19:22:31 -0500 (EST) Date: Thu, 27 Jan 2005 19:22:31 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Medi Montaseri In-Reply-To: <41F96C25.9070906@amcc.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: pthread_t in 5.3 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 00:22:38 -0000 On Thu, 27 Jan 2005, Medi Montaseri wrote: > I am confused about the pthread_t type in FreeBSD 5.3, can you help.. > > Based on /usr/include/pthread.h, > typedef struct pthread *pthread_t; > and when I look for the declaration of "struct pthread" , all I find is > a forward declaration with a comment that says, > > /* > * Forward structure definitions. > * > * These are mostly opaque to the user. > */ > struct pthread; > > That is 'struct pthread' is an opaque type.... > Then in my application, when I try to find my thread_id, I say > > cout << "my tid is " << pthread_self() << endl; > and I get a hex value. Which is syntactically correct, but semantically > in-correct. Sorry, what pthread_t is, is not for you to know. It is up to the implementation to define it anyway that it wants. And is also why there is a pthread_equal() function. > I'm not interested in the pointer, I'm interested in the numerical > thread ID... There is no such thing as defined by POSIX. > Now at this point, you'll think all you have to do is to de-reference > the pointer. > But since 'struct pthrad' is opaque, gdb and myself are clueless to > proceed from here. > Can someone shed some light on this please... I think you are trying to do something that is non-standard/portable. -- DE From owner-freebsd-threads@FreeBSD.ORG Fri Jan 28 00:31:35 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DBB3916A4CE; Fri, 28 Jan 2005 00:31:35 +0000 (GMT) Received: from hadar.amcc.com (hadar.amcc.com [192.195.69.168]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E4B243D48; Fri, 28 Jan 2005 00:31:35 +0000 (GMT) (envelope-from mmontaseri@amcc.com) Received: from [10.66.12.100] (medi.amcc.com [10.66.12.100]) by hadar.amcc.com (Netscape Messaging Server 4.15) with ESMTP id IB044R01.YVF; Thu, 27 Jan 2005 16:31:39 -0800 Message-ID: <41F987B7.7080308@amcc.com> Date: Thu, 27 Jan 2005 16:30:47 -0800 From: "Medi Montaseri" User-Agent: Mozilla Thunderbird 1.0 (X11/20041206) X-Accept-Language: en-us, en MIME-Version: 1.0 To: Daniel Eischen References: In-Reply-To: Content-Type: text/plain; charset=ISO-8859-1; format=flowed Content-Transfer-Encoding: 7bit cc: freebsd-threads@freebsd.org Subject: Re: pthread_t in 5.3 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 00:31:36 -0000 Daniel Eischen wrote: >On Thu, 27 Jan 2005, Medi Montaseri wrote: > > > >>I am confused about the pthread_t type in FreeBSD 5.3, can you help.. >> >>Based on /usr/include/pthread.h, >>typedef struct pthread *pthread_t; >>and when I look for the declaration of "struct pthread" , all I find is >>a forward declaration with a comment that says, >> >>/* >> * Forward structure definitions. >> * >> * These are mostly opaque to the user. >> */ >>struct pthread; >> >>That is 'struct pthread' is an opaque type.... >>Then in my application, when I try to find my thread_id, I say >> >>cout << "my tid is " << pthread_self() << endl; >>and I get a hex value. Which is syntactically correct, but semantically >>in-correct. >> >> > >Sorry, what pthread_t is, is not for you to know. It is up to the >implementation to define it anyway that it wants. And is also why >there is a pthread_equal() function. > > > >>I'm not interested in the pointer, I'm interested in the numerical >>thread ID... >> >> > >There is no such thing as defined by POSIX. > > > >>Now at this point, you'll think all you have to do is to de-reference >>the pointer. >>But since 'struct pthrad' is opaque, gdb and myself are clueless to >>proceed from here. >>Can someone shed some light on this please... >> >> > >I think you are trying to do something that is non-standard/portable. > >-- >DE > > > But on linux, a thread_t is typedef unsigned long int pthread_t; I guess the idea was if a file descriptor is a int and a socket is an int and a process id is an int then a thread also be an int and people can have an array of those IDs. From owner-freebsd-threads@FreeBSD.ORG Fri Jan 28 00:47:08 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 89F8F16A4CE for ; Fri, 28 Jan 2005 00:47:08 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0320943D4C for ; Fri, 28 Jan 2005 00:47:04 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0S0l1Ji009471; Thu, 27 Jan 2005 19:47:01 -0500 (EST) Date: Thu, 27 Jan 2005 19:47:01 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Medi Montaseri In-Reply-To: <41F987B7.7080308@amcc.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: pthread_t in 5.3 X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 00:47:08 -0000 On Thu, 27 Jan 2005, Medi Montaseri wrote: > Daniel Eischen wrote: > > >Sorry, what pthread_t is, is not for you to know. It is up to the > >implementation to define it anyway that it wants. And is also why > >there is a pthread_equal() function. > > > >>I'm not interested in the pointer, I'm interested in the numerical > >>thread ID... > > > >There is no such thing as defined by POSIX. > > > >>Now at this point, you'll think all you have to do is to de-reference > >>the pointer. > >>But since 'struct pthrad' is opaque, gdb and myself are clueless to > >>proceed from here. > >>Can someone shed some light on this please... > >> > >> > > > >I think you are trying to do something that is non-standard/portable. > > But on linux, a thread_t is > typedef unsigned long int pthread_t; > I guess the idea was if a file descriptor is a int and a socket is an > int and a process id is an int > then a thread also be an int and people can have an array of those IDs. It doesn't matter what Linux does; it matters what the POSIX standard says. Nothing precludes pthread_t from being an int or long, but you shouldn't rely on it being that way. Even though our pthread_t is an opaque id (pointer), nothing is stopping you from storing them in an array. An array of: pthread_t tids[NUM_THREADS]; works regardless of the whether you are on Linux or FreeBSD. -- DE From owner-freebsd-threads@FreeBSD.ORG Fri Jan 28 22:10:07 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C127416A4CE for ; Fri, 28 Jan 2005 22:10:07 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id A16DE43D49 for ; Fri, 28 Jan 2005 22:10:07 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0SMA7Pw026381 for ; Fri, 28 Jan 2005 22:10:07 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0SMA7np026380; Fri, 28 Jan 2005 22:10:07 GMT (envelope-from gnats) Date: Fri, 28 Jan 2005 22:10:07 GMT Message-Id: <200501282210.j0SMA7np026380@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: "Serguei E. Leontiev" Subject: Re: threads/76694: fork cause hang in dup()/close() function in child (-lc_r) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: "Serguei E. Leontiev" List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 28 Jan 2005 22:10:07 -0000 The following reply was made to PR threads/76694; it has been noted by GNATS. From: "Serguei E. Leontiev" To: , Cc: "'Serguei E. Leontiev'" Subject: Re: threads/76694: fork cause hang in dup()/close() function in child (-lc_r) Date: Sat, 29 Jan 2005 01:01:41 +0300 Hi, > this may seem harsh, but 5.2.1 was still an experimental branch.. I do not have FreeBSD 4.11 for testing, but both problem (threads/76690, threads/76694) should affect it. IMHO by source code view. -- Sorry for my bests English. Serguei E. Leontiev w:+7(095)939-2382 USSR, Moscow, Universitetskij 13 Sternberg Astronom. w:+7(095)933-1168 USSR, Moscow, 127018, Obraztsova 38 Institute, MSU w:+7(095)289-4367 USSR, Moscow, 127018, Obraztsova 38 h:+7(095)318-1146 USSR, Moscow, 113303, Kakhovka 6-40 m:+7(916)686-1081 SMS: Pager:+7(095)231-3838 for abonent +7(916)686-1081 From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 14:31:54 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7FCBD16A4CE for ; Sat, 29 Jan 2005 14:31:54 +0000 (GMT) Received: from bute.st-andrews.ac.uk (bute.st-and.ac.uk [138.251.12.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id 48D3343D48 for ; Sat, 29 Jan 2005 14:31:53 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by bute.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id OAA20208 for ; Sat, 29 Jan 2005 14:28:51 GMT From: "Niall Douglas" To: freebsd-threads@freebsd.org Date: Sat, 29 Jan 2005 14:31:12 -0000 MIME-Version: 1.0 Message-ID: <41FB9E30.1726.48C69D9F@localhost> Priority: normal X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT Subject: Trying again: select() should be a cancellation point X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 14:31:54 -0000 As no one replied last time, here's this email again. I've found that I cannot work around select() not being a cancellation point on FreeBSD in my code - I had to #ifdef __FreeBSD__ in a hack which manually calls pthread_testcancel() every second. This is *nasty*! If there's any alternative, I'd very much like to hear it. Preferably I'd like to see select() made a cancellation point or a new form of select() eg; select_tc() added. Cheers, Niall > According to: > > http://lists.freebsd.org/pipermail/freebsd-threads/2004- > October/002572.html > > ... it is. But v5.3 FreeBSD doesn't cancel during select(), nor > during recv(). > > OTOH I have other documentation which suggests that neither select() nor > recv() are cancellation points. > > Oh and on Linux, both are cancellation points. In my mind it's > probably more useful if they are cancellation points as you can > always disable cancellation around them if necessary whereas the > opposite is not true. > > Cheers, > Niall > From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 15:14:39 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E65FF16A4CE for ; Sat, 29 Jan 2005 15:14:39 +0000 (GMT) Received: from bute.st-andrews.ac.uk (bute.st-and.ac.uk [138.251.12.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id D393B43D49 for ; Sat, 29 Jan 2005 15:14:38 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by bute.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id PAA21300 for ; Sat, 29 Jan 2005 15:11:37 GMT From: "Niall Douglas" To: freebsd-threads@freebsd.org Date: Sat, 29 Jan 2005 15:13:58 -0000 MIME-Version: 1.0 Message-ID: <41FBA836.25106.48EDC638@localhost> Priority: normal In-reply-to: <41FB9E30.1726.48C69D9F@localhost> X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT Subject: Re: Trying again: select() should be a cancellation point X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 15:14:40 -0000 On 29 Jan 2005 at 14:31, Niall Douglas wrote: > As no one replied last time, here's this email again. > > I've found that I cannot work around select() not being a > cancellation point on FreeBSD in my code - I had to #ifdef > __FreeBSD__ in a hack which manually calls pthread_testcancel() every > second. This is *nasty*! > > If there's any alternative, I'd very much like to hear it. Preferably > I'd like to see select() made a cancellation point or a new form of > select() eg; select_tc() added. Isn't it typical that five minutes after you send an email you realise you were wrong? :( It seems that select() indeed IS a cancellation point. What was happening was that my code does the following: 1. Thread waits on non-blocking read pipe using select() 2. Thread gets cancelled. This invokes thread cancellation handler. 3. Thread cancellation handler sends a "I am disconnecting" message to write pipe. Note that this is a blocking pipe not the same as the read pipe. The message is tiny, around 20 bytes. Now on all systems, the write pipe is always kept fully empty and because they have a 4Kb buffer, 20 bytes should NEVER stall. But on FreeBSD it does stall where it does not on Linux nor WinNT. And that's why I had thought select() was not a cancellation point as the thread seemed to never die. Note that sockets work fine here, as do files ie; you CAN write data to these during a thread cancellation handler. Unfortunately it seems the same is not true for pipes. So what's going on then? Thoughts? You guys will probably want a short example ... Cheers, Niall > > According to: > > > > http://lists.freebsd.org/pipermail/freebsd-threads/2004- > > October/002572.html > > > > ... it is. But v5.3 FreeBSD doesn't cancel during select(), nor > > during recv(). > > > > OTOH I have other documentation which suggests that neither select() > > nor recv() are cancellation points. > > > > Oh and on Linux, both are cancellation points. In my mind it's > > probably more useful if they are cancellation points as you can > > always disable cancellation around them if necessary whereas the > > opposite is not true. > > > > Cheers, > > Niall > > > > > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to > "freebsd-threads-unsubscribe@freebsd.org" > > From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 17:20:04 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 77F1F16A4CF for ; Sat, 29 Jan 2005 17:20:04 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 33AE643D5D for ; Sat, 29 Jan 2005 17:20:04 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0THK48I087025 for ; Sat, 29 Jan 2005 17:20:04 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0THK4CF087024; Sat, 29 Jan 2005 17:20:04 GMT (envelope-from gnats) Resent-Date: Sat, 29 Jan 2005 17:20:04 GMT Resent-Message-Id: <200501291720.j0THK4CF087024@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-threads@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Niall Douglas Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6696616A4CE for ; Sat, 29 Jan 2005 17:19:27 +0000 (GMT) Received: from www.freebsd.org (www.freebsd.org [216.136.204.117]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3B22C43D58 for ; Sat, 29 Jan 2005 17:19:27 +0000 (GMT) (envelope-from nobody@FreeBSD.org) Received: from www.freebsd.org (localhost [127.0.0.1]) by www.freebsd.org (8.13.1/8.13.1) with ESMTP id j0THJRnG033946 for ; Sat, 29 Jan 2005 17:19:27 GMT (envelope-from nobody@www.freebsd.org) Received: (from nobody@localhost) by www.freebsd.org (8.13.1/8.13.1/Submit) id j0THJRKr033945; Sat, 29 Jan 2005 17:19:27 GMT (envelope-from nobody) Message-Id: <200501291719.j0THJRKr033945@www.freebsd.org> Date: Sat, 29 Jan 2005 17:19:27 GMT From: Niall Douglas To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-2.3 Subject: threads/76821: Add access to gdb unique thread id X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 17:20:04 -0000 >Number: 76821 >Category: threads >Synopsis: Add access to gdb unique thread id >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-threads >State: open >Quarter: >Keywords: >Date-Required: >Class: change-request >Submitter-Id: current-users >Arrival-Date: Sat Jan 29 17:20:03 GMT 2005 >Closed-Date: >Last-Modified: >Originator: Niall Douglas >Release: 5.3 >Organization: >Environment: >Description: It would be really handy if a unique integer could be obtained from struct pthread_t so that one can print debug information to stdout like so: Thread 12345678 gets 32 bytes from pipe .. and then in GDB one can match up thread id 12345678 with what gdb knows. Then you can get some idea of which thread is which, something that is currently rather hard :( Some function like pthread_getid_np() would be ideal. Cheers, Niall >How-To-Repeat: >Fix: >Release-Note: >Audit-Trail: >Unformatted: From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 18:23:49 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B5B516A4CE for ; Sat, 29 Jan 2005 18:23:49 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id E778E43D31 for ; Sat, 29 Jan 2005 18:23:48 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0TINluK021948; Sat, 29 Jan 2005 13:23:47 -0500 (EST) Date: Sat, 29 Jan 2005 13:23:47 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Niall Douglas In-Reply-To: <41FB9E30.1726.48C69D9F@localhost> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-threads@freebsd.org Subject: Re: Trying again: select() should be a cancellation point X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 18:23:49 -0000 On Sat, 29 Jan 2005, Niall Douglas wrote: > As no one replied last time, here's this email again. > > I've found that I cannot work around select() not being a > cancellation point on FreeBSD in my code - I had to #ifdef > __FreeBSD__ in a hack which manually calls pthread_testcancel() every > second. This is *nasty*! > > If there's any alternative, I'd very much like to hear it. Preferably > I'd like to see select() made a cancellation point or a new form of > select() eg; select_tc() added. select() is a cancellation point. You're going to have to show us that it isn't ;-) #include #include #include #include #include static volatile int done = 0; static void cleanup(void *arg) { printf("Thread cleanup handler called.\n"); } void * selector(void *arg) { fd_set fds; int ret; printf("Thread started\n"); while (!done) { FD_ZERO(&fds); FD_SET(2, &fds); /* stderr */ pthread_cleanup_push(cleanup, NULL); ret = select(3, &fds, NULL, NULL, NULL); printf("select returned %d, errno %d\n", ret, errno); pthread_cleanup_pop(0); } return (NULL); } int main(void) { pthread_t t; pthread_create(&t, NULL, selector, NULL); sleep(2); pthread_cancel(t); sleep(1); pthread_join(t, NULL); return (0); } -- DE From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 18:28:45 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B482016A4CE; Sat, 29 Jan 2005 18:28:45 +0000 (GMT) Received: from mail.ntplx.net (mail.ntplx.net [204.213.176.10]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5E11843D2F; Sat, 29 Jan 2005 18:28:45 +0000 (GMT) (envelope-from deischen@freebsd.org) Received: from sea.ntplx.net (sea.ntplx.net [204.213.176.11]) j0TISiHm025442; Sat, 29 Jan 2005 13:28:44 -0500 (EST) Date: Sat, 29 Jan 2005 13:28:44 -0500 (EST) From: Daniel Eischen X-X-Sender: eischen@sea.ntplx.net To: Niall Douglas In-Reply-To: <200501291719.j0THJRKr033945@www.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII X-Virus-Scanned: by AMaViS and Clam AntiVirus (mail.ntplx.net) cc: freebsd-gnats-submit@freebsd.org cc: freebsd-threads@freebsd.org Subject: Re: threads/76821: Add access to gdb unique thread id X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 18:28:45 -0000 On Sat, 29 Jan 2005, Niall Douglas wrote: > > >Number: 76821 > >Category: threads > >Synopsis: Add access to gdb unique thread id > >Confidential: no > >Severity: non-critical > >Priority: low > >Responsible: freebsd-threads > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: change-request > >Submitter-Id: current-users > >Arrival-Date: Sat Jan 29 17:20:03 GMT 2005 > >Closed-Date: > >Last-Modified: > >Originator: Niall Douglas > >Release: 5.3 > >Organization: > >Environment: > >Description: > It would be really handy if a unique integer could be obtained from struct pthread_t so that one can print debug information to stdout like so: You can use pthread_getspecific() or TLS. -- DE From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 18:30:15 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3C18416A4CF for ; Sat, 29 Jan 2005 18:30:15 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 250DF43D48 for ; Sat, 29 Jan 2005 18:30:15 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0TIUF6k094736 for ; Sat, 29 Jan 2005 18:30:15 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0TIUE31094732; Sat, 29 Jan 2005 18:30:14 GMT (envelope-from gnats) Date: Sat, 29 Jan 2005 18:30:14 GMT Message-Id: <200501291830.j0TIUE31094732@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: Daniel Eischen Subject: Re: threads/76821: Add access to gdb unique thread id X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Daniel Eischen List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 18:30:15 -0000 The following reply was made to PR threads/76821; it has been noted by GNATS. From: Daniel Eischen To: Niall Douglas Cc: freebsd-gnats-submit@freebsd.org, Subject: Re: threads/76821: Add access to gdb unique thread id Date: Sat, 29 Jan 2005 13:28:44 -0500 (EST) On Sat, 29 Jan 2005, Niall Douglas wrote: > > >Number: 76821 > >Category: threads > >Synopsis: Add access to gdb unique thread id > >Confidential: no > >Severity: non-critical > >Priority: low > >Responsible: freebsd-threads > >State: open > >Quarter: > >Keywords: > >Date-Required: > >Class: change-request > >Submitter-Id: current-users > >Arrival-Date: Sat Jan 29 17:20:03 GMT 2005 > >Closed-Date: > >Last-Modified: > >Originator: Niall Douglas > >Release: 5.3 > >Organization: > >Environment: > >Description: > It would be really handy if a unique integer could be obtained from struct pthread_t so that one can print debug information to stdout like so: You can use pthread_getspecific() or TLS. -- DE From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 18:47:54 2005 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 058C316A4CE; Sat, 29 Jan 2005 18:47:54 +0000 (GMT) Received: from bute.st-andrews.ac.uk (bute.st-and.ac.uk [138.251.12.1]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0F5C43D39; Sat, 29 Jan 2005 18:47:52 +0000 (GMT) (envelope-from s_sourceforge@nedprod.com) Received: from kate (res04-ned6.res.st-and.ac.uk [138.251.234.67]) by bute.st-andrews.ac.uk (8.9.1a/8.9.1) with SMTP id SAA26947; Sat, 29 Jan 2005 18:44:50 GMT From: "Niall Douglas" To: Daniel Eischen Date: Sat, 29 Jan 2005 18:47:11 -0000 MIME-Version: 1.0 Message-ID: <41FBDA2F.9191.49B0FC18@localhost> Priority: normal In-reply-to: References: <200501291719.j0THJRKr033945@www.freebsd.org> X-mailer: Pegasus Mail for Windows (4.21c) Content-type: text/plain; charset=US-ASCII Content-transfer-encoding: 7BIT Content-description: Mail message body Content-Transfer-Encoding: 7BIT cc: freebsd-gnats-submit@freebsd.org cc: freebsd-threads@freebsd.org Subject: Re: threads/76821: Add access to gdb unique thread id X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 18:47:54 -0000 On 29 Jan 2005 at 13:28, Daniel Eischen wrote: > > It would be really handy if a unique integer could be obtained from > > struct > pthread_t so that one can print debug information to stdout like so: > > You can use pthread_getspecific() or TLS. Doesn't help from the point of view of seeing which threads in GDB match which debug output statements. On Linux this is easy as pthread_t is a uint printed by gdb. How about getting gdb to print the pthread_t pointer address instead? I don't care what the number is, so long as I can compare the thread listing in gdb to the debug output and it makes sense! Cheers, Niall From owner-freebsd-threads@FreeBSD.ORG Sat Jan 29 18:50:17 2005 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 33D8016A4CF for ; Sat, 29 Jan 2005 18:50:17 +0000 (GMT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F13543D31 for ; Sat, 29 Jan 2005 18:50:17 +0000 (GMT) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (gnats@localhost [127.0.0.1]) by freefall.freebsd.org (8.13.1/8.13.1) with ESMTP id j0TIoGVS096051 for ; Sat, 29 Jan 2005 18:50:16 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.13.1/8.13.1/Submit) id j0TIoGrt096050; Sat, 29 Jan 2005 18:50:16 GMT (envelope-from gnats) Date: Sat, 29 Jan 2005 18:50:16 GMT Message-Id: <200501291850.j0TIoGrt096050@freefall.freebsd.org> To: freebsd-threads@FreeBSD.org From: "Niall Douglas" Subject: Re: threads/76821: Add access to gdb unique thread id X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: Niall Douglas List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 29 Jan 2005 18:50:17 -0000 The following reply was made to PR threads/76821; it has been noted by GNATS. From: "Niall Douglas" To: Daniel Eischen Cc: freebsd-gnats-submit@freebsd.org, freebsd-threads@freebsd.org Subject: Re: threads/76821: Add access to gdb unique thread id Date: Sat, 29 Jan 2005 18:47:11 -0000 On 29 Jan 2005 at 13:28, Daniel Eischen wrote: > > It would be really handy if a unique integer could be obtained from > > struct > pthread_t so that one can print debug information to stdout like so: > > You can use pthread_getspecific() or TLS. Doesn't help from the point of view of seeing which threads in GDB match which debug output statements. On Linux this is easy as pthread_t is a uint printed by gdb. How about getting gdb to print the pthread_t pointer address instead? I don't care what the number is, so long as I can compare the thread listing in gdb to the debug output and it makes sense! Cheers, Niall