From owner-freebsd-threads@FreeBSD.ORG Mon Jul 28 11:01:47 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EFE1937B404 for ; Mon, 28 Jul 2003 11:01:46 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 74A0743F3F for ; Mon, 28 Jul 2003 11:01:46 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6SI1kUp082276 for ; Mon, 28 Jul 2003 11:01:46 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6SI1j14082271 for freebsd-threads@freebsd.org; Mon, 28 Jul 2003 11:01:45 -0700 (PDT) Date: Mon, 28 Jul 2003 11:01:45 -0700 (PDT) Message-Id: <200307281801.h6SI1j14082271@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2003 18:01:47 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2002/01/16] kern/33951 threads pthread_cancel is ignored 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/01/19] bin/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] bin/24632 threads libc_r delicate deviation from libc in ha o [2001/04/02] bin/26307 threads libc_r aborts when using the KDE media pl o [2001/10/31] bin/31661 threads pthread_kill signal handler doesn't get s o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/03/07] bin/35622 threads sigaltstack is missing in libc_r o [2002/06/27] bin/39922 threads [PATCH?] Threaded applications executed w o [2003/03/02] bin/48856 threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] bin/49087 threads Signals lost in programs linked with libc o [2003/05/07] bin/51949 threads thread in accept cannot be cancelled o [2002/02/01] i386/34536 threads accept() blocks other threads o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/10/10] kern/43887 threads abnormal CPU useage when use pthread_mute o [2003/05/30] kern/52817 threads top(1) shows garbage for threaded process o [2000/08/26] misc/20861 threads libc_r does not honor socket timeouts o [2001/01/25] misc/24641 threads pthread_rwlock_rdlock can deadlock o [2002/08/04] misc/41331 threads Pthread library open sets O_NONBLOCK flag 18 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/09/09] bin/30464 threads pthread mutex attributes -- pshared o [2002/05/02] bin/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwri o [2000/05/25] misc/18824 threads gethostbyname is not thread safe o [2000/10/21] misc/22190 threads A threaded read(2) from a socketpair(2) f o [2002/07/16] misc/40671 threads pthread_cancel doesn't remove thread from 5 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Jul 28 11:03:33 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4497437B401 for ; Mon, 28 Jul 2003 11:03:33 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8850143FE1 for ; Mon, 28 Jul 2003 11:03:23 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6SI3NUp084414 for ; Mon, 28 Jul 2003 11:03:23 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6SI3N3T084408 for threads@freebsd.org; Mon, 28 Jul 2003 11:03:23 -0700 (PDT) Date: Mon, 28 Jul 2003 11:03:23 -0700 (PDT) Message-Id: <200307281803.h6SI3N3T084408@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Jul 2003 18:03:33 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin 1 problem total. Non-critical problems From owner-freebsd-threads@FreeBSD.ORG Tue Jul 29 13:03:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 17F1C37B401; Tue, 29 Jul 2003 13:03:19 -0700 (PDT) Received: from pool-151-200-10-97.res.east.verizon.net (pool-138-88-11-203.res.east.verizon.net [138.88.11.203]) by mx1.FreeBSD.org (Postfix) with ESMTP id 97FE043FA3; Tue, 29 Jul 2003 13:03:17 -0700 (PDT) (envelope-from mtm@identd.net) Received: from kokeb.ambesa.net (shjjhq370o05zz3h@localhost [127.0.0.1]) id h6TK3GN4058322; Tue, 29 Jul 2003 16:03:16 -0400 (EDT) (envelope-from mtm@identd.net) Received: (from mtm@localhost) by kokeb.ambesa.net (8.12.9/8.12.9/Submit) id h6TK3Gaa058321; Tue, 29 Jul 2003 16:03:16 -0400 (EDT) (envelope-from mtm@identd.net) X-Authentication-Warning: kokeb.ambesa.net: mtm set sender to mtm@identd.net using -f Date: Tue, 29 Jul 2003 16:03:16 -0400 From: Mike Makonnen To: "Jacques A. Vidrine" Message-ID: <20030729200315.GA53927@kokeb.ambesa.net> References: <20030618210812.GB21622@rot13.obsecurity.org> <20030619055933.FNBD4805.out003.verizon.net@kokeb.ambesa.net> <20030623150627.GD82167@madman.celabo.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030623150627.GD82167@madman.celabo.org> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD/5.1-CURRENT (i386) cc: threads@FreeBSD.org cc: kris@obsecurity.org Subject: Re: making nsswitch(3) thread-safe (was Re: Removal of bogus gethostbyaddr_r()) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 29 Jul 2003 20:03:19 -0000 Ok Jacques, I have abandoned the method I was thinking of doing it in. Changes to third-party nss modules is definitely a show-stopper IMO. So, this email is essentially to let you know I have decided to use the modifications from PR: bin/29581 (as posted by sobomax a while back) as a starting point since it generally got a good response. For those who don't remember, it's the one that uses the POSIX thread-local storage primitives. BTW, in case it's not clear what I'm talking about: I won't be touching the nsswitch stuff, I'll just be making the the gethostbyXXX functions thread-safe. The essence of the patch won't change, I'll just be modifying the backend to make some things less static, and also hiding the whole thing behind __isthreaded. I should be done in a few days and will post it here after some testing. On Mon, Jun 23, 2003 at 10:06:27AM -0500, Jacques A. Vidrine wrote: > [Sorry for the late reply. As you may or may not have noticed :-) > my FreeBSD hiatus continues :-( I'll be back, just not sure when > yet.] > > On Thu, Jun 19, 2003 at 01:59:32AM -0400, Mike Makonnen wrote: > > > > [ Jacques, I'm CC'ing you since this affects nss ] > > Thanks, Mike! -- for CC'ing me and for looking at this in the first > place. > > > I just took a look into what it would take to make the gethostby*_r functions > > thread safe, and it isn't pretty. The "thread-unsafeness" goes all the way > > down into the individual methods for the various nsswitch(3) databases (dns, > > files, etc). So, the most expedient and least invasive way to go about it would > > be to put a mutex in the gethostby* functions that is dependent on __isthreaded > > being true. This, offcourse, means that the entire call-chain (from top to > > bottom) is locked for the duration of a call to one of these functions, which > > can be a considerable amount of time if it has to timeout. During this time no > > other thread in the application can resolve a host because it will be blocked on > > said mutex. > > You couldn't take this approach even if you wanted. No part of libc > should hold a lock while making calls through nsdispatch. NSS modules > are free to call back into libc. Imagine holding a lock in > gethostbyname_r, calling through nsdispatch and getting to, say, the > LDAP backend. The LDAP backend might itself invoke gethostbyname_r > or some other nsdispatch consumer resulting in recursive locking or > lock order reversals. > > > To my mind the way to fix this properly is to make the nsdispatch(3) > > call-chain in libc thread-safe from the ground-up. This is a huge task in and of > > itself, but is complicated even further by the fact that whatever we do can't > > change the semantics of the existing user-visible functions. > > Yes, this is the approach that must be taken. The nsdispatch engine > is itself thread-safe. The getpw*_r, getgr*_r functions are > thread-safe. I believe that with the exception of the `dns' backend, > the existing backends are thread-safe. > > > I think there are three points to keep in mind: > > > > 1. The nsswitch(3) database functions should be made thread-safe, but at the > > same time not change the thread-unsafe semantics that third-party apps > > might expect (through nsdispatch(3)). > > 2. The thread-unsafeness starts at the bottom (low-level functions) of the > > call-chain, in the nsswitch(3) database functions. > > 3. Because so many databases (hosts, passwd, group, etc) will be affected by > > this it would be too huge a task to do it all at once. The safest approach > > is probably to introduce the changes separately a few at a time. > > What do you mean by the `nsswitch(3) database functions' ? Do you > mean the backends? If so, yes, the backends must ultimately be > thread-safe, and it is best to manage them one database at a time > i.e., all built-in backends --- files, nis, dns --- for a database > must be done at once. > > > We can achieve this by > > Introducing a variable that gets passed down the call-chain telling the > > nsswitch(3) database functions whether the "destination address" (return > > value) will by allocated by the caller or not. If this variable is false then > > the routines can use a static buffer, otherwise, they will assume the caller has > > allocated the necessary space. > > > > Most of the functions will need a lot more work than this to make them > > fully thread-safe. The advantage of doing it this way is that it maintains > > the status-quo while allowing us to make the individual subsystems > > thread-safe separately and as time and resources permit. > > > > The following patch shows what I have in mind. It won't > > compile just yet, but it might make what I'm saying a little clearer :-) > > This initial phase is really just a mechanical change of argument lists. It > > doesn't introduce any thread-safeness on its own. It just makes it easier to > > introduce such changes safely on a subsytem by subsystem basis. > > > > Basically, the _nsdispatch() functionality is moved into nsdispatch_common(), > > which takes a va_list as its last argument instead of a variable number of > > arguments. It also takes one additional argument(int is_r) so callers can tell > > it what kind of semantics they want. Applications calling nsdispatch() get a > > wrapper that passes a false (0) value in the is_r argument. Callers from within > > libc will continue calling_nsdispatch(), which also grows an additional argument > > (int is_r) that callers can pass down to the nsswitch(3) database functions > > through nsdispatch_common(). > > > > What do you think? Should I continue with this or do you think it's bogus? > > > > The important point to keep in mind is that we have to keep the dual semantics > > (thread-safe and unsafe) all the way down the call-chain because only the > > top-level and low-level functions know the actual memory type and size that > > needs to be allocated for the return value to the library users. So, we can't > > make the low-level nsswitch(2) database functions unconditionally thread-safe > > and restrict the dual semantics to the top-level functions. > > I think this is bogus :-) I must say I don't really understand what > you are trying to accomplish ... maybe we should get on IRC and chat. > Also, please look at getpwent.c to see how thread-safe versus > non-thread-safe entry points to getpw* are handled if you have not > already. I can't see any need to add this `is_r' argument. I don't > see the purpose of it. > > The hardest part of making the thread-safe gethost*_r functions, IMHO > (after having done some work in this area) is untangling the > KAME-contributed code. There is a lot of incest between the > implementations of the various high-level resolver routines > (gethostbyname, getaddrinfo, getipnodebyname, etc.) that complicates > things. But the approach is conceptually simple: > > 1. Write gethostbyname_r (thread-safe), which invokes > nsdispatch("gethostbyname_r", ...). > > 2. Make each back-end thread-safe. Starting with a global lock is > reasonable. I don't think we can ever do anything efficient > with the BIND 4 resolver, anyway. > > Only I think you want to do the whole family of functions in one blow, > not one-at-a-time. (But maybe I'm wrong here, 'cause that has > certainly kept me from committing any partial work in this area.) > > Note also that the libc<-->NSS modules interface is constrained. > Nominally we use the GNU libc interface, which has different entry > points for e.g. gethostbyname and gethostbyname_r, with a set number > and type of parameters. We don't want to change these interface lest > we make porting NSS modules to FreeBSD harder. We can add some > interfaces, though, where needed (i.e. getaddrinfo). > > You can contact me in a number of ways to discuss: > > - EFnet nickname `nectar' (or whatever nick shows up for > `nectar@free.bsd') > - AIM `nectar5' > - ICQ `17783107' > - Yahoo! `nectar' > - MSN `nectar523@hotmail.com' > - Voice 612-743-3364 > > It would be best if we could `schedule' something. I apologize, but > my freetime is very slim lately. > > Cheers, > -- > Jacques Vidrine . NTT/Verio SME . FreeBSD UNIX . Heimdal > nectar@celabo.org . jvidrine@verio.net . nectar@freebsd.org . nectar@kth.se -- Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc mtm@identd.net | D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 mtm@FreeBSD.Org| FreeBSD - Unleash the Daemon! From owner-freebsd-threads@FreeBSD.ORG Wed Jul 30 14:37:20 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BAC2337B404 for ; Wed, 30 Jul 2003 14:37:19 -0700 (PDT) Received: from gw.celabo.org (gw.celabo.org [208.42.49.153]) by mx1.FreeBSD.org (Postfix) with ESMTP id ACDF243F93 for ; Wed, 30 Jul 2003 14:37:18 -0700 (PDT) (envelope-from nectar@celabo.org) Received: from madman.celabo.org (madman.celabo.org [10.0.1.111]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "madman.celabo.org", Issuer "celabo.org CA" (verified OK)) by gw.celabo.org (Postfix) with ESMTP id 3F31554840; Wed, 30 Jul 2003 16:37:18 -0500 (CDT) Received: by madman.celabo.org (Postfix, from userid 1001) id D11FD6D455; Wed, 30 Jul 2003 16:37:17 -0500 (CDT) Date: Wed, 30 Jul 2003 16:37:17 -0500 From: "Jacques A. Vidrine" To: Mike Makonnen Message-ID: <20030730213717.GA76112@madman.celabo.org> References: <20030618210812.GB21622@rot13.obsecurity.org> <20030619055933.FNBD4805.out003.verizon.net@kokeb.ambesa.net> <20030623150627.GD82167@madman.celabo.org> <20030729200315.GA53927@kokeb.ambesa.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030729200315.GA53927@kokeb.ambesa.net> X-Url: http://www.celabo.org/ User-Agent: Mutt/1.5.4i-ja.1 cc: threads@FreeBSD.org cc: kris@obsecurity.org Subject: Re: making nsswitch(3) thread-safe (was Re: Removal of bogus gethostbyaddr_r()) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 21:37:21 -0000 Hi, Mike! Thanks for the update! If you don't mind, can you `hold this thought' until Monday? By then I should have had time to look at sobomax's patch. I believe by your description that this direction is fine, but I want to double-check some issues. Cheers! -- Jacques Vidrine . NTT/Verio SME . FreeBSD UNIX . Heimdal nectar@celabo.org . jvidrine@verio.net . nectar@freebsd.org . nectar@kth.se From owner-freebsd-threads@FreeBSD.ORG Wed Jul 30 15:13:44 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D159737B401 for ; Wed, 30 Jul 2003 15:13:44 -0700 (PDT) Received: from perrin.int.nxad.com (internal.ext.nxad.com [69.1.70.251]) by mx1.FreeBSD.org (Postfix) with ESMTP id 55F5543FCB for ; Wed, 30 Jul 2003 15:13:44 -0700 (PDT) (envelope-from sean@nxad.com) Received: by perrin.int.nxad.com (Postfix, from userid 1001) id AFFE120F00; Wed, 30 Jul 2003 15:13:43 -0700 (PDT) Date: Wed, 30 Jul 2003 15:13:43 -0700 From: Sean Chittenden To: Wesley Morgan Message-ID: <20030730221343.GF34647@perrin.int.nxad.com> References: <20030728102545.F1269@catalyst.chemikals.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030728102545.F1269@catalyst.chemikals.org> X-PGP-Key: finger seanc@FreeBSD.org X-PGP-Fingerprint: 3849 3760 1AFE 7B17 11A0 83A6 DD99 E31F BC84 B341 X-Web-Homepage: http://sean.chittenden.org/ User-Agent: Mutt/1.5.4i cc: threads@FreeBSD.org Subject: Re: postgresql-devel "psql" and -current X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 22:13:45 -0000 > I've been following the postgresql-devel port on a development box > running -current for some time now. There is some kind of a problem > with the "pager" functionality of psql that causes it to screw up my > terminal to the point of requiring a reset. I haven't got a -stable > box handy to verify this on, but it's a very annoying problem that I > thought might be fixed during the development process, but I'm > beginning to think it is a freebsd-specific problem. Have you seen > anything similar? Whoa, hrm, yeah... I ran into it last night while writing the jail_read_only_transactions patch... and this looks like it's out of my area of expertise. How recent is your -CURRENT? This looks like a libc/libc_r's problem. Let me CC: -threads to see if they have any hints or ideas. Using the most recent snapshot of PostgreSQL, the psql(1) client is crashing when the pager terminates and psql(1) resumes control of the terminal. What's really obnoxious, is when I compile in debugging symbols, I can't get it to core, but I can get it to sit infinitely in poll(2). Without debugging symbols: #0 0x2821eb37 in _pthread_mutex_trylock () from /usr/lib/libc_r.so.5 #1 0x2821ed12 in _pthread_mutex_lock () from /usr/lib/libc_r.so.5 #2 0x2821c838 in pthread_rwlock_rdlock () from /usr/lib/libc_r.so.5 #3 0x281c679c in nsdispatch () from /usr/lib/libc.so.5 #4 0x281a09c5 in getpwuid_r () from /usr/lib/libc.so.5 #5 0x281a0bcb in getpwuid_r () from /usr/lib/libc.so.5 #6 0x281a0acb in getpwuid_r () from /usr/lib/libc.so.5 #7 0x281a0c79 in getpwuid () from /usr/lib/libc.so.5 #8 0x280b78ae in pqGetpwuid () from /home/sean/open_source/postgresql/pgsql/src/test/regress/tmp_check/install/usr/local/pgsql/lib//libpq.so.3 #9 0x280a5c8a in fe_getauthname () from /home/sean/open_source/postgresql/pgsql/src/test/regress/tmp_check/install/usr/local/pgsql/lib//libpq.so.3 #10 0x280a8bdb in pqPacketSend () from /home/sean/open_source/postgresql/pgsql/src/test/regress/tmp_check/install/usr/local/pgsql/lib//libpq.so.3 #11 0x280a5e07 in PQconnectStart () from /home/sean/open_source/postgresql/pgsql/src/test/regress/tmp_check/install/usr/local/pgsql/lib//libpq.so.3 #12 0x280a6408 in PQsetdbLogin () from /home/sean/open_source/postgresql/pgsql/src/test/regress/tmp_check/install/usr/local/pgsql/lib//libpq.so.3 #13 0x0804c8eb in do_connect () #14 0x0804a97c in exec_command () #15 0x0804a66e in HandleSlashCmds () #16 0x0805089b in MainLoop () #17 0x08052030 in main () #18 0x0804a502 in _start () With debugging symbols: 0x2817d91f in poll () from /usr/lib/libc.so.5 (gdb) bt #0 0x2817d91f in poll () from /usr/lib/libc.so.5 #1 0x282279a1 in _thread_kern_sched_state_unlock () from /usr/lib/libc_r.so.5 #2 0x28227395 in _thread_kern_scheduler () from /usr/lib/libc_r.so.5 Any thoughts? I've been away from email for a week or so and am out of the loop atm. I'm working on recompiling my libs with -g, but it'll be another 6hrs before my world will be complete. -sc -- Sean Chittenden From owner-freebsd-threads@FreeBSD.ORG Wed Jul 30 16:34:05 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4FD4737B401 for ; Wed, 30 Jul 2003 16:34:05 -0700 (PDT) Received: from volatile.chemikals.org (cae57-160-011.sc.rr.com [66.57.160.11]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4E64C43F3F for ; Wed, 30 Jul 2003 16:34:02 -0700 (PDT) (envelope-from morganw@chemikals.org) Received: from localhost (morganw@localhost [127.0.0.1]) h6UNY0lq011538; Wed, 30 Jul 2003 19:34:01 -0400 (EDT) (envelope-from morganw@chemikals.org) Date: Wed, 30 Jul 2003 19:34:00 -0400 (EDT) From: Wesley Morgan To: Sean Chittenden In-Reply-To: <20030730221343.GF34647@perrin.int.nxad.com> Message-ID: <20030730192811.U10689@volatile.chemikals.org> References: <20030728102545.F1269@catalyst.chemikals.org> <20030730221343.GF34647@perrin.int.nxad.com> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@FreeBSD.org Subject: Re: postgresql-devel "psql" and -current X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 30 Jul 2003 23:34:05 -0000 On Wed, 30 Jul 2003, Sean Chittenden wrote: > Whoa, hrm, yeah... I ran into it last night while writing the > jail_read_only_transactions patch... and this looks like it's out of > my area of expertise. How recent is your -CURRENT? This looks like a > libc/libc_r's problem. Let me CC: -threads to see if they have any > hints or ideas. > > Using the most recent snapshot of PostgreSQL, the psql(1) client is > crashing when the pager terminates and psql(1) resumes control of the > terminal. What's really obnoxious, is when I compile in debugging > symbols, I can't get it to core, but I can get it to sit infinitely in > poll(2). My -current is very recent, July 22. This problem has existed for quite a while though. I have also seen randomish "corruption" of output from psql that I assumed was just a flaky development problem. I try to use KSE for threads, but there appear to be some problems with it. I can't get apache 2 to run with PHP that uses anything other than libc_r (libthr and libkse crash). I also have a strange problem with an app that runs on a serial interface. If I link in libpq static (along with all the other dependencies) it works fine, but without the static link somehow something gets clobbered and it won't work properly, however I have not devoted much time to debugging this as the interface library is still in the developmental stages. -- Hi! I'm a .signature virus! Copy me into your ~/.signature to help me spread! From owner-freebsd-threads@FreeBSD.ORG Wed Jul 30 22:37:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2C80837B401; Wed, 30 Jul 2003 22:37:00 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7550043F93; Wed, 30 Jul 2003 22:36:59 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h6V5axwO084014; Wed, 30 Jul 2003 22:36:59 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h6V5awHm078977; Wed, 30 Jul 2003 22:36:58 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h6V5aws4078881; Wed, 30 Jul 2003 22:36:58 -0700 (PDT) (envelope-from marcel) Date: Wed, 30 Jul 2003 22:36:58 -0700 From: Marcel Moolenaar To: alpha@FreeBSD.org, threads@FreeBSD.org Message-ID: <20030731053658.GA62373@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: libthr ported to alpha X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 05:37:00 -0000 Gang, In case it went unnoticed: libthr is now officially supported on alpha. JFYI, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Wed Jul 30 23:57:39 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F32F437B401; Wed, 30 Jul 2003 23:57:38 -0700 (PDT) Received: from pool-151-200-10-97.res.east.verizon.net (pool-138-88-5-64.res.east.verizon.net [138.88.5.64]) by mx1.FreeBSD.org (Postfix) with ESMTP id A95B443F75; Wed, 30 Jul 2003 23:57:31 -0700 (PDT) (envelope-from mtm@identd.net) Received: from kokeb.ambesa.net (kkmtq6kqkzqcunht@localhost [127.0.0.1]) id h6V6vGJO021906; Thu, 31 Jul 2003 02:57:16 -0400 (EDT) (envelope-from mtm@identd.net) Received: (from mtm@localhost) by kokeb.ambesa.net (8.12.9/8.12.9/Submit) id h6V6vCWi021905; Thu, 31 Jul 2003 02:57:12 -0400 (EDT) (envelope-from mtm@identd.net) X-Authentication-Warning: kokeb.ambesa.net: mtm set sender to mtm@identd.net using -f Date: Thu, 31 Jul 2003 02:57:12 -0400 From: Mike Makonnen To: Marcel Moolenaar Message-ID: <20030731065711.GA21875@kokeb.ambesa.net> References: <20030731053658.GA62373@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030731053658.GA62373@athlon.pn.xcllnt.net> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD/5.1-CURRENT (i386) cc: threads@freebsd.org cc: alpha@freebsd.org Subject: Re: libthr ported to alpha X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 06:57:39 -0000 On Wed, Jul 30, 2003 at 10:36:58PM -0700, Marcel Moolenaar wrote: > Gang, > > In case it went unnoticed: libthr is now officially supported on > alpha. > That's great news! I completely missed it in the commit mails. Motivates me to continue the cleanup, as well :-) Cheers. -- Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc mtm@identd.net | D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 mtm@FreeBSD.Org| FreeBSD - Unleash the Daemon! From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 01:11:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 041AF37B401; Thu, 31 Jul 2003 01:11:35 -0700 (PDT) Received: from freebie.xs4all.nl (freebie.xs4all.nl [213.84.32.253]) by mx1.FreeBSD.org (Postfix) with ESMTP id A8DB543FA3; Thu, 31 Jul 2003 01:11:33 -0700 (PDT) (envelope-from wkb@freebie.xs4all.nl) Received: from freebie.xs4all.nl (localhost [127.0.0.1]) by freebie.xs4all.nl (8.12.9/8.12.9) with ESMTP id h6V8BWCE075589; Thu, 31 Jul 2003 10:11:32 +0200 (CEST) (envelope-from wkb@freebie.xs4all.nl) Received: (from wkb@localhost) by freebie.xs4all.nl (8.12.9/8.12.9/Submit) id h6V8BWEh075588; Thu, 31 Jul 2003 10:11:32 +0200 (CEST) Date: Thu, 31 Jul 2003 10:11:32 +0200 From: Wilko Bulte To: Marcel Moolenaar Message-ID: <20030731081132.GA75573@freebie.xs4all.nl> References: <20030731053658.GA62373@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030731053658.GA62373@athlon.pn.xcllnt.net> User-Agent: Mutt/1.4.1i X-OS: FreeBSD 4.8-STABLE X-PGP: finger wilko@freebsd.org cc: threads@freebsd.org cc: alpha@freebsd.org Subject: Re: libthr ported to alpha X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 31 Jul 2003 08:11:35 -0000 On Wed, Jul 30, 2003 at 10:36:58PM -0700, Marcel Moolenaar wrote: > Gang, > > In case it went unnoticed: libthr is now officially supported on > alpha. I owe you a Grolsch ;-) -- | / o / /_ _ wilko@FreeBSD.org |/|/ / / /( (_) Bulte From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 19:31:40 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D5D1637B401 for ; Thu, 31 Jul 2003 19:31:40 -0700 (PDT) Received: from ns.aratech.co.kr (ns.aratech.co.kr [61.34.11.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 19C5D43F93 for ; Thu, 31 Jul 2003 19:31:40 -0700 (PDT) (envelope-from tj@atj.dyndns.org) Received: from tj.aratech.co.kr ([61.34.11.212] helo=atj.dyndns.org ident=mail) by ns.aratech.co.kr with esmtp (Exim 3.36 #1 (Debian)) id 19iOhv-0001OH-00 for ; Fri, 01 Aug 2003 10:27:47 +0900 Received: from tj by atj.dyndns.org with local (Exim 4.20) id 19iOTN-00085r-31 for freebsd-threads@freebsd.org; Fri, 01 Aug 2003 10:12:45 +0900 Date: Fri, 1 Aug 2003 10:12:45 +0900 To: freebsd-threads@freebsd.org Message-ID: <20030801011245.GA31048@atj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i From: TeJun Huh Subject: Regarding KSE X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: tejun@aratech.co.kr List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 02:31:41 -0000 Hello, I'm currently evaluating various thread packages for a paper. The focus is on aptness for Internet servers, so performance and scalability are main concerns. I'm trying to evaluate M:N threading (scheduler activaton) and 1:1 threading using KSE and maybe traditional libc_r on FreeBSD 5.1. I've encountered several problems with KSE. I made a small test case program to demonstrate the problem. The program accepts one argument telling it how many threads should be created. The main thread creates given number of threads with user allocated stack (pthread_attr_setstack()). After createing all threads the main thread waits forever. Each created thread reads fd 0 (stdin). After reading some input from stdin, it prints the input and exits. The source of test program is appended at the tail of this mail. First problem is that I can hang whole operating system (even ctrl-alt-sec doesn't work). Creating 160 threads and pressing ctrl-c after all threads are created hang the kernel reliably. Secondly, I cannot get more than 150 threads active at once. I guess this is the limit of KSEs per process. Is there any way to increase this number? I'm looking at ten thousands threads at least. Thanks in advance. /*****************************************************************/ /***************** test program source follows *******************/ /* compile with '$ gcc -D_THREAD_SAFE -o test test.c -lkse' ******/ /*****************************************************************/ #include #include #include #include static void * service_startup(void *arg) { char buf[64]; int ret; printf("service startup %d\n", (int)arg); if ((ret = read(0, buf, sizeof(buf)-1)) < 0) printf("read failed %d\n", errno); else { buf[ret] = '\0'; printf("read: \"%s\"\n", buf); } printf("exiting\n"); return NULL; } #define STACK_SIZE 32768 int main(int argc, char **argv) { static const struct timespec ts1sec = { 1, 0 }; int nr, i; if (argc < 2) { fprintf(stderr, "babo\n"); return 1; } nr = atoi(argv[1]); for (i = 0; i < nr; i++) { void *stack; pthread_t thr; pthread_attr_t attr; if ((stack = malloc(STACK_SIZE)) == NULL) { fprintf(stderr, "malloc failed\n"); return 1; } pthread_attr_init(&attr); pthread_attr_setdetachstate(&attr, 1); pthread_attr_setstack(&attr, stack, STACK_SIZE); if (pthread_create(&thr, &attr, service_startup, (void *)i) < 0) { fprintf(stderr, "pthread_create failed\n"); return 1; } } while (1) nanosleep(&ts1sec, NULL); return 0; } From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 20:16:41 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 34A9437B401 for ; Thu, 31 Jul 2003 20:16:41 -0700 (PDT) Received: from ns.aratech.co.kr (ns.aratech.co.kr [61.34.11.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9A1E243FE0 for ; Thu, 31 Jul 2003 20:16:40 -0700 (PDT) (envelope-from tj@atj.dyndns.org) Received: from tj.aratech.co.kr ([61.34.11.212] helo=atj.dyndns.org ident=mail) by ns.aratech.co.kr with esmtp (Exim 3.36 #1 (Debian)) id 19iQPF-0004FL-00 for ; Fri, 01 Aug 2003 12:16:37 +0900 Received: from tj by atj.dyndns.org with local (Exim 4.20) id 19iQPk-0000Ym-SY for freebsd-threads@freebsd.org; Fri, 01 Aug 2003 12:17:08 +0900 Date: Fri, 1 Aug 2003 12:17:08 +0900 To: freebsd-threads@freebsd.org Message-ID: <20030801031708.GA2128@atj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i From: TeJun Huh Subject: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: tejun@aratech.co.kr List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 03:16:41 -0000 I forgot to mention about -lthr (1:1 KSE threading) related problems. 1. If the program is compiled with '-static -lthr', the program crashes at the first thread creation. Stack trace follows. (gdb) bt full #0 0x0804da8c in _spinlock_pthread (pthread=0x0, lck=0x805c3f8) at umtx.h:59 No locals. #1 0x0804da2a in _spinlock (lck=0x805c3f8) at /usr/src/lib/libthr/thread/thr_spinlock.c:69 No locals. #2 0x08058344 in malloc () No symbol table info available. #3 0x080483f9 in _pthread_create (thread=0x0, attr=0x3a0, start_routine=0, arg=0x0) at /usr/src/lib/libthr/thread/thr_create.c:80 f_gc = 928 ret = 0 gc_thread = (struct pthread *) 0x0 new_thread = (struct pthread *) 0x2c pattr = (struct pthread_attr *) 0x80487a8 flags = 0 stack = (void *) 0x805c3f8 #4 0x0804835b in main (argc=2, argv=0xbfbffb40) at testpthread.c:50 stack = (void *) 0x8062000 thr = (struct pthread *) 0x0 attr = (struct pthread_attr *) 0x806a000 ts1sec = {tv_sec = 1, tv_nsec = 0} nr = 10 i = 0 #5 0x08048145 in _start () 2. Without -static, the program works fine, but pthread_create() fails with EAGAIN at thread count 110. B.T.W. pthread_create() return value handling was wrong in the example program. It always confuses me. :-( How can I increase this limit? Again, I'm looking for something like ten thousands. From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 22:48:09 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 31A9737B401 for ; Thu, 31 Jul 2003 22:48:09 -0700 (PDT) Received: from pool-151-200-10-97.res.east.verizon.net (pool-138-88-5-64.res.east.verizon.net [138.88.5.64]) by mx1.FreeBSD.org (Postfix) with ESMTP id 946F643FB1 for ; Thu, 31 Jul 2003 22:48:07 -0700 (PDT) (envelope-from mtm@identd.net) Received: from kokeb.ambesa.net (fzud4e22zjigjz0g@localhost [127.0.0.1]) id h715ltJO030383; Fri, 1 Aug 2003 01:47:55 -0400 (EDT) (envelope-from mtm@identd.net) Received: (from mtm@localhost) by kokeb.ambesa.net (8.12.9/8.12.9/Submit) id h715lsBI030382; Fri, 1 Aug 2003 01:47:54 -0400 (EDT) (envelope-from mtm@identd.net) X-Authentication-Warning: kokeb.ambesa.net: mtm set sender to mtm@identd.net using -f Date: Fri, 1 Aug 2003 01:47:54 -0400 From: Mike Makonnen To: tejun@aratech.co.kr Message-ID: <20030801054754.GA30332@kokeb.ambesa.net> References: <20030801031708.GA2128@atj.dyndns.org> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030801031708.GA2128@atj.dyndns.org> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD/5.1-CURRENT (i386) cc: freebsd-threads@freebsd.org Subject: Re: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 05:48:09 -0000 On Fri, Aug 01, 2003 at 12:17:08PM +0900, TeJun Huh wrote: > I forgot to mention about -lthr (1:1 KSE threading) related problems. > > 1. If the program is compiled with '-static -lthr', the program > crashes at the first thread creation. Stack trace follows. Are you using 5.1-RELEASE or -CURRENT ? I think this has already been fixed in -CURRENT. > 2. Without -static, the program works fine, but pthread_create() > fails with EAGAIN at thread count 110. B.T.W. pthread_create() return > value handling was wrong in the example program. It always confuses > me. :-( How can I increase this limit? Again, I'm looking for > something like ten thousands. This is problematic because on i386, libthr uses the LDTs, which are limited to 9182. But if I'm not mistaken the first NLDT (17) entries are already in use so in reality you can have only (9182 - NLDT) threads. You can do this by changing MAXTHR in src/lib/libthr/arch/i386/i386/_setcurthread.c. You might also want to bump up UMTX_QUEUES in src/sys/kern/kern_umtx.c if you do that. I am not aware of any limits on alpha or sparc64 (but then I'm not knowledgeable about either :) BTW, libthr uses some of the same code as KSE, but it is not based on KSE. Cheers. -- Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc mtm@identd.net | D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 mtm@FreeBSD.Org| FreeBSD - Unleash the Daemon! From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 23:08:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0517537B401 for ; Thu, 31 Jul 2003 23:08:35 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88B2843FB1 for ; Thu, 31 Jul 2003 23:08:34 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003080106083401500kh90de>; Fri, 1 Aug 2003 06:08:34 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id XAA38655; Thu, 31 Jul 2003 23:08:24 -0700 (PDT) Date: Thu, 31 Jul 2003 23:08:24 -0700 (PDT) From: Julian Elischer To: Mike Makonnen In-Reply-To: <20030801054754.GA30332@kokeb.ambesa.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 06:08:35 -0000 On Fri, 1 Aug 2003, Mike Makonnen wrote: > On Fri, Aug 01, 2003 at 12:17:08PM +0900, TeJun Huh wrote: > > I forgot to mention about -lthr (1:1 KSE threading) related problems. > > > > 1. If the program is compiled with '-static -lthr', the program > > crashes at the first thread creation. Stack trace follows. > > Are you using 5.1-RELEASE or -CURRENT ? I think this has already > been fixed in -CURRENT. > > > 2. Without -static, the program works fine, but pthread_create() > > fails with EAGAIN at thread count 110. B.T.W. pthread_create() return > > value handling was wrong in the example program. It always confuses > > me. :-( How can I increase this limit? Again, I'm looking for > > something like ten thousands. > > This is problematic because on i386, libthr uses the LDTs, which > are limited to 9182. But if I'm not mistaken the first NLDT (17) > entries are already in use so in reality you can have only > (9182 - NLDT) threads. You can do this by changing MAXTHR > in src/lib/libthr/arch/i386/i386/_setcurthread.c. You might also > want to bump up UMTX_QUEUES in src/sys/kern/kern_umtx.c if you > do that. > > I am not aware of any limits on alpha or sparc64 (but then I'm > not knowledgeable about either :) > > BTW, libthr uses some of the same code as KSE, but it is not > based on KSE. As Mike says.. libthr and libkse are not the same, though both are derived from the libc_r code. libthr is 1:1 libkse can be compiled to be 1:1 or M:N In M:N mode the 8192 limit does not apply as we multilex a lot of threads to a single LDT entry. > > Cheers. > -- > Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc > mtm@identd.net | D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 > mtm@FreeBSD.Org| FreeBSD - Unleash the Daemon! > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 23:18:11 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 56FCC37B401 for ; Thu, 31 Jul 2003 23:18:11 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8C3F143F85 for ; Thu, 31 Jul 2003 23:18:10 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h716HXwO099696; Thu, 31 Jul 2003 23:17:33 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h716HXQ3005557; Thu, 31 Jul 2003 23:17:33 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h716HXKf005556; Thu, 31 Jul 2003 23:17:33 -0700 (PDT) (envelope-from marcel) Date: Thu, 31 Jul 2003 23:17:33 -0700 From: Marcel Moolenaar To: Mike Makonnen Message-ID: <20030801061733.GC5221@athlon.pn.xcllnt.net> References: <20030801031708.GA2128@atj.dyndns.org> <20030801054754.GA30332@kokeb.ambesa.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030801054754.GA30332@kokeb.ambesa.net> User-Agent: Mutt/1.5.4i cc: freebsd-threads@freebsd.org Subject: Re: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 06:18:11 -0000 On Fri, Aug 01, 2003 at 01:47:54AM -0400, Mike Makonnen wrote: > > I am not aware of any limits on alpha or sparc64 (but then I'm > not knowledgeable about either :) Correct, there's no hard limit on alpha, ia64 nor sparc64. -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Thu Jul 31 23:56:48 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 26E0F37B401 for ; Thu, 31 Jul 2003 23:56:48 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 758FC43F75 for ; Thu, 31 Jul 2003 23:56:47 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h716ugax004423; Fri, 1 Aug 2003 02:56:42 -0400 (EDT) Date: Fri, 1 Aug 2003 02:56:42 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 06:56:48 -0000 On Thu, 31 Jul 2003, Julian Elischer wrote: > > On Fri, 1 Aug 2003, Mike Makonnen wrote: > > > On Fri, Aug 01, 2003 at 12:17:08PM +0900, TeJun Huh wrote: > > > I forgot to mention about -lthr (1:1 KSE threading) related problems. > > > > > > 1. If the program is compiled with '-static -lthr', the program > > > crashes at the first thread creation. Stack trace follows. > > > > Are you using 5.1-RELEASE or -CURRENT ? I think this has already > > been fixed in -CURRENT. > > > > > 2. Without -static, the program works fine, but pthread_create() > > > fails with EAGAIN at thread count 110. B.T.W. pthread_create() return > > > value handling was wrong in the example program. It always confuses > > > me. :-( How can I increase this limit? Again, I'm looking for > > > something like ten thousands. You can create thousands with libkse. You'll really only be limited by memory -- unless you create a lot of threads as PTHREAD_SCOPE_SYSTEM, in which case you'll likely have similar limitations as libthr. As julian said, libkse can be compiled to be in 1:1 mode, so if you build it that way (the default is M:N, not 1:1), you'll also be limited. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Aug 1 00:18:16 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 73CBA37B401 for ; Fri, 1 Aug 2003 00:18:16 -0700 (PDT) Received: from ns.aratech.co.kr (ns.aratech.co.kr [61.34.11.200]) by mx1.FreeBSD.org (Postfix) with ESMTP id 50A9143F75 for ; Fri, 1 Aug 2003 00:18:15 -0700 (PDT) (envelope-from tj@atj.dyndns.org) Received: from tj.aratech.co.kr ([61.34.11.212] helo=atj.dyndns.org ident=mail) by ns.aratech.co.kr with esmtp (Exim 3.36 #1 (Debian)) id 19iUAz-00008X-00 for ; Fri, 01 Aug 2003 16:18:09 +0900 Received: from tj by atj.dyndns.org with local (Exim 4.20) id 19iUBP-0000hC-TJ for freebsd-threads@freebsd.org; Fri, 01 Aug 2003 16:18:35 +0900 Date: Fri, 1 Aug 2003 16:18:35 +0900 To: freebsd-threads@freebsd.org Message-ID: <20030801071835.GA2653@atj.dyndns.org> References: Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i From: TeJun Huh Subject: Re: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: tejun@aratech.co.kr List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 07:18:16 -0000 On Fri, Aug 01, 2003 at 02:56:42AM -0400, Daniel Eischen wrote: > > You can create thousands with libkse. You'll really > only be limited by memory -- unless you create a lot > of threads as PTHREAD_SCOPE_SYSTEM, in which case you'll > likely have similar limitations as libthr. As julian > said, libkse can be compiled to be in 1:1 mode, so if > you build it that way (the default is M:N, not 1:1), > you'll also be limited. > > Dan Eischen Yes, I tried that. Currently, what I wanna know are 1. How to increase the number of threads upto ~9000 using 1:1 threading. It doesn't matter whether I use libkse or libthr. I'm gonna try recompile libthr with MAXTHR, UMTX_QUEUES adjusted. If that works, this problem is solved. (Although still limited to ~9000) 2. On M:N KSE, how to put more KSEs into one process or KSE group. >From what I understand, a KSE multiplexes userthreads, but when the user thread invokes a blocking systemcall, the KSE blocks handling it and another KSE does an upcall to user level scheduler notifying the blocking and the user level scheduler can utilize the upcalling KSE to run another user thread, thus maintaining concurrency. (Am I getting it right?) So, with large number of threads blocked, large number of KSEs are needed and, from what I've read, the hard limit will be ~9000 because KSEs shouldn't be very different from kernel threads from this perspective. But what I get is something around 150 and wanna know if there is any sysctl or boot kernel parameter or anything that can be tuned to increase this limit. 3. Lastly, about the kernel hang problem. Is it a known bug or fixed in 5.1-CURRENT? If not, I think this is a very serious problem, because any user can hang the whole system very hard. Thanks in advance. From owner-freebsd-threads@FreeBSD.ORG Fri Aug 1 00:55:01 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1FB1237B401; Fri, 1 Aug 2003 00:55:01 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id A966243F93; Fri, 1 Aug 2003 00:55:00 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) Received: from localhost (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h717suUp072767; Fri, 1 Aug 2003 00:54:58 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) From: David Xu To: tejun@aratech.co.kr, TeJun Huh , freebsd-threads@FreeBSD.org Date: Fri, 1 Aug 2003 15:57:59 +0800 User-Agent: KMail/1.5.2 References: <20030801071835.GA2653@atj.dyndns.org> In-Reply-To: <20030801071835.GA2653@atj.dyndns.org> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200308011557.59484.davidxu@FreeBSD.org> Subject: Re: And problems regarding -lthr (1:1 KSE) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: davidxu@FreeBSD.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 01 Aug 2003 07:55:01 -0000 On Friday 01 August 2003 15:18, TeJun Huh wrote: > 2. On M:N KSE, how to put more KSEs into one process or KSE group. > From what I understand, a KSE multiplexes userthreads, but when the > user thread invokes a blocking systemcall, the KSE blocks handling it > and another KSE does an upcall to user level scheduler notifying the > blocking and the user level scheduler can utilize the upcalling KSE to > run another user thread, thus maintaining concurrency. (Am I getting > it right?) > > So, with large number of threads blocked, large number of KSEs are > needed and, from what I've read, the hard limit will be ~9000 because > KSEs shouldn't be very different from kernel threads from this > perspective. But what I get is something around 150 and wanna know if > there is any sysctl or boot kernel parameter or anything that can be > tuned to increase this limit. > The following sysctls may help you: max contextes can be blocked in kernel(not limited by LDT): kern.threads.max_threads_per_proc max system scope thread can be created(limited by LDT): kern.threads.max_groups_per_proc > 3. Lastly, about the kernel hang problem. Is it a known bug or fixed > in 5.1-CURRENT? If not, I think this is a very serious problem, because > any user can hang the whole system very hard. > After 5.1 release, there are much improvements had been commited. > Thanks in advance. > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" David Xu