From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 07:33:40 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F52337B401; Mon, 21 Apr 2003 07:33:40 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id D2EA143FA3; Mon, 21 Apr 2003 07:33:39 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LEXcBg011698; Mon, 21 Apr 2003 10:33:38 -0400 (EDT) Received: from localhost (eischen@localhost)h3LEXcrv011695; Mon, 21 Apr 2003 10:33:38 -0400 (EDT) Date: Mon, 21 Apr 2003 10:33:38 -0400 (EDT) From: Daniel Eischen To: threads@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: davidxu@freebsd.org Subject: Thread-related LOR in exit1() (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 14:33:40 -0000 David, did you see this message sent to -current? Also, I have scope system threads running in a simple test. I'm getting EPROCLIM though when trying to create more than 4 or so of them (KSEs with their own KSEG). Here's the patches to libpthread that I'm currently using along with a test program. http://people.freebsd.org/~deischen/kse/libpthread.diffs http://people.freebsd.org/~deischen/kse/thread_test2.c -- Dan ---------- Forwarded message ---------- Date: Mon, 21 Apr 2003 20:25:41 +1000 From: Tim Robbins To: current@freebsd.org Subject: Thread-related LOR in exit1() While running a trivial test program linked with libkse that created 3 threads: lock order reversal 1st 0xc0f0f428 process lock (process lock) @ /usr/src/sys/kern/kern_exit.c:134 2nd 0xc02f75c0 Giant (Giant) @ /usr/src/sys/kern/kern_exit.c:142 Tim _______________________________________________ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to "freebsd-current-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 07:57:21 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D4B8537B401; Mon, 21 Apr 2003 07:57:21 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0637C43FBF; Mon, 21 Apr 2003 07:57:20 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with SMTP id h3LEvHUp077950; Mon, 21 Apr 2003 07:57:19 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <000b01c30816$e71101a0$0701a8c0@tiger> From: "David Xu" To: "Daniel Eischen" , References: Date: Mon, 21 Apr 2003 23:01:33 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 Subject: Re: Thread-related LOR in exit1() (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 14:57:22 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: Cc: Sent: Monday, April 21, 2003 10:33 PM Subject: Thread-related LOR in exit1() (fwd) > David, did you see this message sent to -current? >=20 I have committed a fix. > Also, I have scope system threads running in a simple test. I'm > getting EPROCLIM though when trying to create more than 4 or so > of them (KSEs with their own KSEG). >=20 Yeah, sysctl -a |grep kern.threads should show the reason, please increase max_groups_per_proc sysctl by = hand, I know it's too small. :-) > Here's the patches to libpthread that I'm currently using along > with a test program. >=20 > http://people.freebsd.org/~deischen/kse/libpthread.diffs > http://people.freebsd.org/~deischen/kse/thread_test2.c >=20 I am going to sleep again, I will test it tomorrow. it seems KMF_NOCOMPLETED should be added in _kse_lock_wait(). > --=20 > Dan >=20 > ---------- Forwarded message ---------- > Date: Mon, 21 Apr 2003 20:25:41 +1000 > From: Tim Robbins > To: current@freebsd.org > Subject: Thread-related LOR in exit1() >=20 > While running a trivial test program linked with libkse that created > 3 threads: >=20 > lock order reversal > 1st 0xc0f0f428 process lock (process lock) @ = /usr/src/sys/kern/kern_exit.c:134 > 2nd 0xc02f75c0 Giant (Giant) @ /usr/src/sys/kern/kern_exit.c:142 >=20 >=20 > Tim > _______________________________________________ > freebsd-current@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-current > To unsubscribe, send any mail to = "freebsd-current-unsubscribe@freebsd.org" >=20 > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to = "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 08:41:41 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 90D4C37B405 for ; Mon, 21 Apr 2003 08:41:41 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id BB5BD43FE0 for ; Mon, 21 Apr 2003 08:41:40 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LFfeBg021551; Mon, 21 Apr 2003 11:41:40 -0400 (EDT) Received: from localhost (eischen@localhost)h3LFfdDV021548; Mon, 21 Apr 2003 11:41:39 -0400 (EDT) Date: Mon, 21 Apr 2003 11:41:39 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <000b01c30816$e71101a0$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Thread-related LOR in exit1() (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 15:41:41 -0000 On Mon, 21 Apr 2003, David Xu wrote: > ----- Original Message ----- > From: "Daniel Eischen" > To: > Cc: > Sent: Monday, April 21, 2003 10:33 PM > Subject: Thread-related LOR in exit1() (fwd) > > > > David, did you see this message sent to -current? > > > I have committed a fix. > > > Also, I have scope system threads running in a simple test. I'm > > getting EPROCLIM though when trying to create more than 4 or so > > of them (KSEs with their own KSEG). > > > > Yeah, sysctl -a |grep kern.threads > should show the reason, please increase max_groups_per_proc sysctl by hand, > I know it's too small. :-) Yeah, it should probably be whatever libthr uses for max threads (KSEs). Perhaps minus the number of virtual CPUs (upcalls) allowed in the main KSEG. > > Here's the patches to libpthread that I'm currently using along > > with a test program. > > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > http://people.freebsd.org/~deischen/kse/thread_test2.c > > > I am going to sleep again, I will test it tomorrow. > it seems KMF_NOCOMPLETED should be added in _kse_lock_wait(). That's all you do is sleep! I'll look at using KMF_NOCOMPLETED in _kse_lock_wait(). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 08:44:02 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E27D37B401 for ; Mon, 21 Apr 2003 08:44:02 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 68C8243F85 for ; Mon, 21 Apr 2003 08:44:01 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LFi0Bg021962 for ; Mon, 21 Apr 2003 11:44:00 -0400 (EDT) Received: (from eischen@localhost) by pcnet1.pcnet.com (8.12.8/8.12.8/Submit) id h3LFi0C3021950 for threads@freebsd.org; Mon, 21 Apr 2003 11:44:00 -0400 (EDT) Date: Mon, 21 Apr 2003 11:44:00 -0400 (EDT) From: Daniel Eischen Message-Id: <200304211544.h3LFi0C3021950@pcnet1.pcnet.com> To: threads@freebsd.org Subject: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 15:44:02 -0000 Since libkse seems to be generally useful, anyone mind if I go back to installing it as libpthread? -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 09:44:57 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 291A137B404 for ; Mon, 21 Apr 2003 09:44:57 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 63D3F43FDF for ; Mon, 21 Apr 2003 09:44:56 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3LGitt70332; Mon, 21 Apr 2003 12:44:55 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Mon, 21 Apr 2003 12:44:55 -0400 (EDT) From: Jeff Roberson To: Daniel Eischen In-Reply-To: <200304211544.h3LFi0C3021950@pcnet1.pcnet.com> Message-ID: <20030421124423.Y76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 16:44:57 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > Since libkse seems to be generally useful, anyone mind if I > go back to installing it as libpthread? > > -- > Dan Eischen There is some question over whether kse or thr will be the default threading implementation for 5.1. I think this should be discussed before we decide on what lib uses libpthread. Cheers, Jeff From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 11:28:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 972EC37B401 for ; Mon, 21 Apr 2003 11:28:00 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id F13B843F3F for ; Mon, 21 Apr 2003 11:27:59 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LIRrBg017388; Mon, 21 Apr 2003 14:27:53 -0400 (EDT) Received: from localhost (eischen@localhost)h3LIRrX6017385; Mon, 21 Apr 2003 14:27:53 -0400 (EDT) Date: Mon, 21 Apr 2003 14:27:52 -0400 (EDT) From: Daniel Eischen To: Jeff Roberson In-Reply-To: <20030421124423.Y76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 18:28:00 -0000 On Mon, 21 Apr 2003, Jeff Roberson wrote: > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > Since libkse seems to be generally useful, anyone mind if I > > go back to installing it as libpthread? > > > > -- > > Dan Eischen > > There is some question over whether kse or thr will be the default > threading implementation for 5.1. I think this should be discussed before > we decide on what lib uses libpthread. Uhh, there really is no *default* unless you are talking about building all ports with one or the other (changing PTHREAD_LIBS in Mk/*). Libthr should not be installed as libpthread; it should be libthr (or libthread) and libpthread should be installed as libpthread. Changing PTHREAD_LIBS to one or the other has no bearing on how they are installed. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 11:29:44 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5C21237B401 for ; Mon, 21 Apr 2003 11:29:44 -0700 (PDT) Received: from stork.mail.pas.earthlink.net (stork.mail.pas.earthlink.net [207.217.120.188]) by mx1.FreeBSD.org (Postfix) with ESMTP id D45C243F3F for ; Mon, 21 Apr 2003 11:29:43 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0061.cvx22-bradley.dialup.earthlink.net ([209.179.198.61] helo=mindspring.com) by stork.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197g2p-0001pd-00; Mon, 21 Apr 2003 11:29:36 -0700 Message-ID: <3EA4382F.6CA7C63A@mindspring.com> Date: Mon, 21 Apr 2003 11:27:59 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson References: <20030421124423.Y76635-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a411ca1d8b0db9acde5e82ce88d3d8bccd350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 18:29:44 -0000 Jeff Roberson wrote: > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > Since libkse seems to be generally useful, anyone mind if I > > go back to installing it as libpthread? > > There is some question over whether kse or thr will be the default > threading implementation for 5.1. I think this should be discussed before > we decide on what lib uses libpthread. Isn't libthr really just libkse with N = M? These should behave exactly the same, right? It should even be possible to drop the UTS out of the picture for everything but signal handling, I think (i.e. it would never get upcalled) in the 1:1 case, if this were done? -- Terry From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 11:38:16 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 77DFD37B401 for ; Mon, 21 Apr 2003 11:38:16 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id C3F3E43FBF for ; Mon, 21 Apr 2003 11:38:15 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LIcBBg019003; Mon, 21 Apr 2003 14:38:12 -0400 (EDT) Received: from localhost (eischen@localhost)h3LIcBPq018989; Mon, 21 Apr 2003 14:38:11 -0400 (EDT) Date: Mon, 21 Apr 2003 14:38:11 -0400 (EDT) From: Daniel Eischen To: Terry Lambert In-Reply-To: <3EA4382F.6CA7C63A@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 18:38:16 -0000 On Mon, 21 Apr 2003, Terry Lambert wrote: > Jeff Roberson wrote: > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > Since libkse seems to be generally useful, anyone mind if I > > > go back to installing it as libpthread? > > > > There is some question over whether kse or thr will be the default > > threading implementation for 5.1. I think this should be discussed before > > we decide on what lib uses libpthread. > > Isn't libthr really just libkse with N = M? Yes, I wasn't going to bring that up, though. > These should behave exactly the same, right? It should even be > possible to drop the UTS out of the picture for everything but > signal handling, I think (i.e. it would never get upcalled) in > the 1:1 case, if this were done? Libpthread can be made to behave the same as libthr just by forcing every thread to be scope system. Currently, the implementation for scope system threads does have a small amount of overhead in that they still get upcalls after the thread blocks in the kernel (in this case the KSE just reenters the kernel with kse_release() and waits for the thread to become unblocked). These KSEs also require a small stack separate from the thread's stack. The code is in place (in the UTS) to not require a separate stack and not get any upcalls for these threads, but we just need a bit more kernel work to optimize this overhead away. I do have scope system threads running in a simple test program. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 13:45:05 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5863C37B401 for ; Mon, 21 Apr 2003 13:45:05 -0700 (PDT) Received: from rwcrmhc51.attbi.com (rwcrmhc51.attbi.com [204.127.198.38]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4BD143F3F for ; Mon, 21 Apr 2003 13:45:04 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc51.attbi.com (rwcrmhc51) with ESMTP id <2003042120450405100ftr2oe>; Mon, 21 Apr 2003 20:45:04 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA86261; Mon, 21 Apr 2003 13:45:03 -0700 (PDT) Date: Mon, 21 Apr 2003 13:45:01 -0700 (PDT) From: Julian Elischer To: Daniel Eischen In-Reply-To: <200304211544.h3LFi0C3021950@pcnet1.pcnet.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 20:45:05 -0000 WE have a small problem.. when we started we had only on pthreads package and libKSE was teh heir.. now we have to live with libthr in the picture. it is possible that we should think of a naming scheme that allows all 3 libraries to have meaningful related names libc_r -> libpthreadM1 libthr -> libpthreadMM libkse -> libpthreadMN or something. On Mon, 21 Apr 2003, Daniel Eischen wrote: > Since libkse seems to be generally useful, anyone mind if I > go back to installing it as libpthread? > > -- > Dan Eischen > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 14:03:38 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E9E937B401 for ; Mon, 21 Apr 2003 14:03:37 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id DE0CC43FAF for ; Mon, 21 Apr 2003 14:03:36 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LL3UBg012225; Mon, 21 Apr 2003 17:03:30 -0400 (EDT) Received: from localhost (eischen@localhost)h3LL3TJf012218; Mon, 21 Apr 2003 17:03:29 -0400 (EDT) Date: Mon, 21 Apr 2003 17:03:29 -0400 (EDT) From: Daniel Eischen To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 21:03:38 -0000 On Mon, 21 Apr 2003, Julian Elischer wrote: > > WE have a small problem.. when we started we had only on pthreads > package and libKSE was teh heir.. No, libkse WAS libpthread. We renamed the library temporarily until it was proven to work reasonably well (I did the commit that renamed it). > now we have to live with libthr in the picture. > it is possible that we should think of a naming scheme that > allows all 3 libraries to have meaningful related names > > libc_r -> libpthreadM1 > libthr -> libpthreadMM > libkse -> libpthreadMN > > or something. The current naming scheme is fine. Libc_r is already a well known name along with its shortcomings. Solaris uses libthread for 1:1 as far as I can tell which would seem to match libthr, and libpthread is the POSIX threads library whose goal is to support POSIX as fully as possible (scope process and scope system). I would object to libpthread{M1,MM,MN}. We already had a name for libpthread. Libthr can live fine as libthr or libthread. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 14:28:39 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7FCE237B401 for ; Mon, 21 Apr 2003 14:28:39 -0700 (PDT) Received: from sccrmhc02.attbi.com (sccrmhc02.attbi.com [204.127.202.62]) by mx1.FreeBSD.org (Postfix) with ESMTP id C82F643FBD for ; Mon, 21 Apr 2003 14:28:38 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc02.attbi.com (sccrmhc02) with ESMTP id <2003042121283700200pkc61e>; Mon, 21 Apr 2003 21:28:38 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA86346; Mon, 21 Apr 2003 14:28:35 -0700 (PDT) Date: Mon, 21 Apr 2003 14:28:32 -0700 (PDT) From: Julian Elischer To: Terry Lambert In-Reply-To: <3EA4382F.6CA7C63A@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Daniel Eischen cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 21:28:39 -0000 On Mon, 21 Apr 2003, Terry Lambert wrote: > Jeff Roberson wrote: > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > Since libkse seems to be generally useful, anyone mind if I > > > go back to installing it as libpthread? > > > > There is some question over whether kse or thr will be the default > > threading implementation for 5.1. I think this should be discussed before > > we decide on what lib uses libpthread. > > Isn't libthr really just libkse with N = M? yes and no.. it could have been but for practical reasons was implemented separatly. > > These should behave exactly the same, right? It should even be > possible to drop the UTS out of the picture for everything but > signal handling, I think (i.e. it would never get upcalled) in > the 1:1 case, if this were done? > > -- Terry > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 14:31:43 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 571D937B401 for ; Mon, 21 Apr 2003 14:31:43 -0700 (PDT) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by mx1.FreeBSD.org (Postfix) with ESMTP id A5D2343F3F for ; Mon, 21 Apr 2003 14:31:42 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc01.attbi.com (sccrmhc01) with ESMTP id <200304212131410010084q43e>; Mon, 21 Apr 2003 21:31:41 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA86354; Mon, 21 Apr 2003 14:31:40 -0700 (PDT) Date: Mon, 21 Apr 2003 14:31:38 -0700 (PDT) From: Julian Elischer To: Daniel Eischen In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 21:31:43 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > On Mon, 21 Apr 2003, Julian Elischer wrote: > > > > > WE have a small problem.. when we started we had only on pthreads > > package and libKSE was teh heir.. > > No, libkse WAS libpthread. We renamed the library > temporarily until it was proven to work reasonably > well (I did the commit that renamed it). > > > now we have to live with libthr in the picture. > > it is possible that we should think of a naming scheme that > > allows all 3 libraries to have meaningful related names > > > > libc_r -> libpthreadM1 > > libthr -> libpthreadMM > > libkse -> libpthreadMN > > > > or something. > > The current naming scheme is fine. Libc_r is already > a well known name along with its shortcomings. Solaris > uses libthread for 1:1 as far as I can tell which would > seem to match libthr, and libpthread is the POSIX threads > library whose goal is to support POSIX as fully as > possible (scope process and scope system). > > I would object to libpthread{M1,MM,MN}. We already > had a name for libpthread. Libthr can live fine as > libthr or libthread. > > -- > Dan Eischen > > fair enough From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 14:55:15 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0E8F537B401 for ; Mon, 21 Apr 2003 14:55:15 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 06F6A43FDD for ; Mon, 21 Apr 2003 14:55:14 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h3LLseUE030825; Tue, 22 Apr 2003 00:54:40 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h3LLsdJK030822; Tue, 22 Apr 2003 00:54:39 +0300 (EEST) Date: Tue, 22 Apr 2003 00:54:37 +0300 (EEST) From: Narvi To: Daniel Eischen In-Reply-To: Message-ID: <20030422004324.I29990-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 21:55:15 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > On Mon, 21 Apr 2003, Julian Elischer wrote: > > > > > WE have a small problem.. when we started we had only on pthreads > > package and libKSE was teh heir.. > > No, libkse WAS libpthread. We renamed the library > temporarily until it was proven to work reasonably > well (I did the commit that renamed it). > > > now we have to live with libthr in the picture. > > it is possible that we should think of a naming scheme that > > allows all 3 libraries to have meaningful related names > > > > libc_r -> libpthreadM1 > > libthr -> libpthreadMM > > libkse -> libpthreadMN > > > > or something. > > The current naming scheme is fine. Libc_r is already > a well known name along with its shortcomings. Solaris > uses libthread for 1:1 as far as I can tell which would > seem to match libthr, and libpthread is the POSIX threads > library whose goal is to support POSIX as fully as > possible (scope process and scope system). > Solaris libthr is not a 1:1 thread library - the difference between libpthread and libthread is that libthread implements UI (sometimes also called Solaris) threads, and not pthreads. > I would object to libpthread{M1,MM,MN}. We already > had a name for libpthread. Libthr can live fine as > libthr or libthread. > No - if both provide the same API with potentially different implementation tradeoffs, then ythe libraries should be installed with different names and a symlink to the prefered one installed. At least for the moment it is not clear that one of teh libraries is a winner and will eliminate the other, so it would be evil to force people to explicitly link against one or the other causing future compatibility problems. Both provide the same pthreads API so there is no reasonable case for demanding that one of them can't have its SONAME be libpthpread.so.1 > -- > Dan Eischen > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 15:24:53 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85D6E37B401 for ; Mon, 21 Apr 2003 15:24:53 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id AEE9143FDD for ; Mon, 21 Apr 2003 15:24:52 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LMOkBg024261; Mon, 21 Apr 2003 18:24:46 -0400 (EDT) Received: from localhost (eischen@localhost)h3LMOjuF024258; Mon, 21 Apr 2003 18:24:45 -0400 (EDT) Date: Mon, 21 Apr 2003 18:24:45 -0400 (EDT) From: Daniel Eischen To: Narvi In-Reply-To: <20030422004324.I29990-100000@haldjas.folklore.ee> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 22:24:53 -0000 On Tue, 22 Apr 2003, Narvi wrote: > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > On Mon, 21 Apr 2003, Julian Elischer wrote: > > > > > > > > WE have a small problem.. when we started we had only on pthreads > > > package and libKSE was teh heir.. > > > > No, libkse WAS libpthread. We renamed the library > > temporarily until it was proven to work reasonably > > well (I did the commit that renamed it). > > > > > now we have to live with libthr in the picture. > > > it is possible that we should think of a naming scheme that > > > allows all 3 libraries to have meaningful related names > > > > > > libc_r -> libpthreadM1 > > > libthr -> libpthreadMM > > > libkse -> libpthreadMN > > > > > > or something. > > > > The current naming scheme is fine. Libc_r is already > > a well known name along with its shortcomings. Solaris > > uses libthread for 1:1 as far as I can tell which would > > seem to match libthr, and libpthread is the POSIX threads > > library whose goal is to support POSIX as fully as > > possible (scope process and scope system). > > > > Solaris libthr is not a 1:1 thread library - the difference between Yes, I believe it now is. > libpthread and libthread is that libthread implements UI (sometimes also > called Solaris) threads, and not pthreads. And libthr threads are sorta like native threads. > > I would object to libpthread{M1,MM,MN}. We already > > had a name for libpthread. Libthr can live fine as > > libthr or libthread. > > > > No - if both provide the same API with potentially different > implementation tradeoffs, then ythe libraries should be installed with > different names and a symlink to the prefered one installed. The default library, whatever that may be, is easily changed by setting PTHREAD_LIBS, which the ports system knows about. > At least for the moment it is not clear that one of teh libraries is a > winner and will eliminate the other, It was never intended for libthr to eliminate libpthread. In fact, it was advertised as an interim solution to buy us time. Trying to work around libthr in the kernel hasn't really bought us time though. > so it would be evil to force people > to explicitly link against one or the other causing future compatibility > problems. Both provide the same pthreads API so there is no reasonable > case for demanding that one of them can't have its SONAME be > libpthpread.so.1 We can, in libpthread, build it so that every thread is 1:1. True, we might not be able to do this without a small amount of overhead right now, but it will be possible in the future. If built that way, we can install it as libthr so that any application relying on libthr.so.1 will still have it there. This has been one of my goals for some time. I want libpthread to get out there for 5.1. I want to see those bug reports roll in, so that by the time 5.2 (-stable) comes around we have a good handle on what the problems are and have addressed them. We have 3 people (David Xu, Julian, and myself) wanting to maintain this and keep it moving forward. I don't want to see it go away. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 16:05:24 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1630A37B401 for ; Mon, 21 Apr 2003 16:05:24 -0700 (PDT) Received: from speicher.org (sirius.speicher.org [209.74.10.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id B6BC043FBD for ; Mon, 21 Apr 2003 16:05:22 -0700 (PDT) (envelope-from geoff@speicher.org) Received: from localhost (geoff@localhost) by speicher.org (8.11.6/8.11.6) with ESMTP id h3LNJdR60326; Mon, 21 Apr 2003 19:19:42 -0400 (EDT) (envelope-from geoff@speicher.org) Date: Mon, 21 Apr 2003 19:19:39 -0400 (EDT) From: "Geoffrey C. Speicher" To: Narvi In-Reply-To: <20030422004324.I29990-100000@haldjas.folklore.ee> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Daniel Eischen cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 23:05:24 -0000 On Tue, 22 Apr 2003, Narvi wrote: > At least for the moment it is not clear that one of teh libraries is a > winner and will eliminate the other, so it would be evil to force people What's not clear about it? libkse is a superset of the functionality of libthr. Seems pretty straightforward to me that the long-term winner is libkse. > to explicitly link against one or the other causing future compatibility > problems. Both provide the same pthreads API so there is no reasonable > case for demanding that one of them can't have its SONAME be > libpthpread.so.1 Same API, but different capabilities. The one called pthread should be the complete one. No compatability problems in that direction. Geoff From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 16:08:21 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D0C437B401 for ; Mon, 21 Apr 2003 16:08:21 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id E4F8443FE3 for ; Mon, 21 Apr 2003 16:08:20 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3LN8HU93074; Mon, 21 Apr 2003 19:08:17 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Mon, 21 Apr 2003 19:08:17 -0400 (EDT) From: Jeff Roberson To: Daniel Eischen In-Reply-To: Message-ID: <20030421190707.B76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 23:08:21 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > On Tue, 22 Apr 2003, Narvi wrote: > > > > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > > > On Mon, 21 Apr 2003, Julian Elischer wrote: > > > > > > > > > > > WE have a small problem.. when we started we had only on pthreads > > > > package and libKSE was teh heir.. > > > > > > No, libkse WAS libpthread. We renamed the library > > > temporarily until it was proven to work reasonably > > > well (I did the commit that renamed it). > > > > > > > now we have to live with libthr in the picture. > > > > it is possible that we should think of a naming scheme that > > > > allows all 3 libraries to have meaningful related names > > > > > > > > libc_r -> libpthreadM1 > > > > libthr -> libpthreadMM > > > > libkse -> libpthreadMN > > > > > > > > or something. > > > > > > The current naming scheme is fine. Libc_r is already > > > a well known name along with its shortcomings. Solaris > > > uses libthread for 1:1 as far as I can tell which would > > > seem to match libthr, and libpthread is the POSIX threads > > > library whose goal is to support POSIX as fully as > > > possible (scope process and scope system). > > > > > > > Solaris libthr is not a 1:1 thread library - the difference between > > Yes, I believe it now is. > > > libpthread and libthread is that libthread implements UI (sometimes also > > called Solaris) threads, and not pthreads. > > And libthr threads are sorta like native threads. thr system calls are native threads. libthr implements pthreads. I don't really care what you call it now. Go ahead and call it libpthreads. > > > > I would object to libpthread{M1,MM,MN}. We already > > > had a name for libpthread. Libthr can live fine as > > > libthr or libthread. > > > > > > > No - if both provide the same API with potentially different > > implementation tradeoffs, then ythe libraries should be installed with > > different names and a symlink to the prefered one installed. > > The default library, whatever that may be, is easily changed > by setting PTHREAD_LIBS, which the ports system knows about. > > > At least for the moment it is not clear that one of teh libraries is a > > winner and will eliminate the other, > > It was never intended for libthr to eliminate libpthread. > In fact, it was advertised as an interim solution to buy > us time. Trying to work around libthr in the kernel hasn't > really bought us time though. > > > so it would be evil to force people > > to explicitly link against one or the other causing future compatibility > > problems. Both provide the same pthreads API so there is no reasonable > > case for demanding that one of them can't have its SONAME be > > libpthpread.so.1 > > We can, in libpthread, build it so that every thread is 1:1. > True, we might not be able to do this without a small > amount of overhead right now, but it will be possible in the > future. If built that way, we can install it as libthr so > that any application relying on libthr.so.1 will still have > it there. This has been one of my goals for some time. > > I want libpthread to get out there for 5.1. I want to see > those bug reports roll in, so that by the time 5.2 (-stable) > comes around we have a good handle on what the problems are > and have addressed them. We have 3 people (David Xu, Julian, > and myself) wanting to maintain this and keep it moving > forward. I don't want to see it go away. > > -- > Dan Eischen > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 16:12:01 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2A10B37B401 for ; Mon, 21 Apr 2003 16:12:01 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4613F43FBD for ; Mon, 21 Apr 2003 16:12:00 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3LNBfZ94876; Mon, 21 Apr 2003 19:11:41 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Mon, 21 Apr 2003 19:11:40 -0400 (EDT) From: Jeff Roberson To: "Geoffrey C. Speicher" In-Reply-To: Message-ID: <20030421190831.S76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Daniel Eischen cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 23:12:01 -0000 On Mon, 21 Apr 2003, Geoffrey C. Speicher wrote: > On Tue, 22 Apr 2003, Narvi wrote: > > > At least for the moment it is not clear that one of teh libraries is a > > winner and will eliminate the other, so it would be evil to force people > > What's not clear about it? libkse is a superset of the functionality > of libthr. Seems pretty straightforward to me that the long-term > winner is libkse. Uh, no, it's not that clear. One is probably 30k lines of code smaller which makes it less likely to have bugs. They also have different performance characteristics. You could construct tests which demonstrated better performance for different workloads on each. I appreciate you trying to be informative on the list but please restrict your comments to things that you are more intimately familiar with. There are enough other people leading casual observers astray. > > > to explicitly link against one or the other causing future compatibility > > problems. Both provide the same pthreads API so there is no reasonable > > case for demanding that one of them can't have its SONAME be > > libpthpread.so.1 > > Same API, but different capabilities. The one called pthread should be > the complete one. No compatability problems in that direction. > > Geoff > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 16:12:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81FF737B401 for ; Mon, 21 Apr 2003 16:12:19 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7B1D243FE3 for ; Mon, 21 Apr 2003 16:12:18 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h3LNCGUE031252; Tue, 22 Apr 2003 02:12:16 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h3LNCGjp031249; Tue, 22 Apr 2003 02:12:16 +0300 (EEST) Date: Tue, 22 Apr 2003 02:12:16 +0300 (EEST) From: Narvi To: "Geoffrey C. Speicher" In-Reply-To: Message-ID: <20030422020939.S29990-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 23:12:19 -0000 On Mon, 21 Apr 2003, Geoffrey C. Speicher wrote: > On Tue, 22 Apr 2003, Narvi wrote: > > > At least for the moment it is not clear that one of teh libraries is a > > winner and will eliminate the other, so it would be evil to force people > > What's not clear about it? libkse is a superset of the functionality > of libthr. Seems pretty straightforward to me that the long-term > winner is libkse. > This assumes that libkse M:N model will provide supperior performance and scalability, and this is not clear. Or does merely the fact that itis M:N somehow make it more winning contender? > > to explicitly link against one or the other causing future compatibility > > problems. Both provide the same pthreads API so there is no reasonable > > case for demanding that one of them can't have its SONAME be > > libpthpread.so.1 > > Same API, but different capabilities. The one called pthread should be > the complete one. No compatability problems in that direction. > > Geoff > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 16:21:58 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D1D2F37B401 for ; Mon, 21 Apr 2003 16:21:58 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4DF6743FEA for ; Mon, 21 Apr 2003 16:21:30 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h3LNLTUE031314; Tue, 22 Apr 2003 02:21:29 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h3LNLT9k031311; Tue, 22 Apr 2003 02:21:29 +0300 (EEST) Date: Tue, 22 Apr 2003 02:21:29 +0300 (EEST) From: Narvi To: Daniel Eischen In-Reply-To: Message-ID: <20030422015621.P29990-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 23:21:59 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > > Solaris libthr is not a 1:1 thread library - the difference between > > Yes, I believe it now is. > If by "now" you mean Solaris 9, then yes, this is so. This is not a fundamental issue, merely how kernel API-s are used. On Solaris 8 you get both the "old" M:N version and the Solaris 9 style 1:1 version in /usr/lib/lwp. There is no way to tell what it will be in Solaris 9+x for some arbitrary positive value of x. > > libpthread and libthread is that libthread implements UI (sometimes also > > called Solaris) threads, and not pthreads. > > And libthr threads are sorta like native threads. > No. The 'native' threads are LWP-s and both libthread and libpthread are implemented in terms of LWPs and related syscalls. [snip] > > so it would be evil to force people > > to explicitly link against one or the other causing future compatibility > > problems. Both provide the same pthreads API so there is no reasonable > > case for demanding that one of them can't have its SONAME be > > libpthpread.so.1 > > We can, in libpthread, build it so that every thread is 1:1. > True, we might not be able to do this without a small > amount of overhead right now, but it will be possible in the > future. If built that way, we can install it as libthr so > that any application relying on libthr.so.1 will still have > it there. This has been one of my goals for some time. > > I want libpthread to get out there for 5.1. I want to see > those bug reports roll in, so that by the time 5.2 (-stable) > comes around we have a good handle on what the problems are > and have addressed them. We have 3 people (David Xu, Julian, > and myself) wanting to maintain this and keep it moving > forward. I don't want to see it go away. > If you are confident that a working libkse is available and that there will not be a support nightmerte afterwards, then naturaly, my objections to the naming scheme evaporate. > -- > Dan Eischen > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 16:55:32 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D925537B401 for ; Mon, 21 Apr 2003 16:55:32 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 30C8C43FDF for ; Mon, 21 Apr 2003 16:55:32 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3LNtVBg007785; Mon, 21 Apr 2003 19:55:31 -0400 (EDT) Received: from localhost (eischen@localhost)h3LNtToK007777; Mon, 21 Apr 2003 19:55:31 -0400 (EDT) Date: Mon, 21 Apr 2003 19:55:29 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Thread-related LOR in exit1() (fwd) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Apr 2003 23:55:33 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > On Mon, 21 Apr 2003, David Xu wrote: > > ----- Original Message ----- > > From: "Daniel Eischen" > > To: > > Cc: > > Sent: Monday, April 21, 2003 10:33 PM > > Subject: Thread-related LOR in exit1() (fwd) > > > > > > > David, did you see this message sent to -current? > > > > > I have committed a fix. > > > > > Also, I have scope system threads running in a simple test. I'm > > > getting EPROCLIM though when trying to create more than 4 or so > > > of them (KSEs with their own KSEG). > > > > > > > Yeah, sysctl -a |grep kern.threads > > should show the reason, please increase max_groups_per_proc sysctl by hand, > > I know it's too small. :-) > > Yeah, it should probably be whatever libthr uses for max > threads (KSEs). Perhaps minus the number of virtual CPUs > (upcalls) allowed in the main KSEG. > > > > Here's the patches to libpthread that I'm currently using along > > > with a test program. > > > > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > > http://people.freebsd.org/~deischen/kse/thread_test2.c > > > > > I am going to sleep again, I will test it tomorrow. > > it seems KMF_NOCOMPLETED should be added in _kse_lock_wait(). > > That's all you do is sleep! I'll look at using KMF_NOCOMPLETED > in _kse_lock_wait(). Yep, your exactly right. We need to prevent upcalls while in _kse_lock_wait() during kse_release(). This seems to fix any locklevel problems I was having. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 17:18:31 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9AADC37B401 for ; Mon, 21 Apr 2003 17:18:31 -0700 (PDT) Received: from speicher.org (sirius.speicher.org [209.74.10.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4D13043FB1 for ; Mon, 21 Apr 2003 17:18:30 -0700 (PDT) (envelope-from geoff@speicher.org) Received: from localhost (geoff@localhost) by speicher.org (8.11.6/8.11.6) with ESMTP id h3M0X4o60473; Mon, 21 Apr 2003 20:33:05 -0400 (EDT) (envelope-from geoff@speicher.org) Date: Mon, 21 Apr 2003 20:33:04 -0400 (EDT) From: "Geoffrey C. Speicher" To: Narvi In-Reply-To: <20030422020939.S29990-100000@haldjas.folklore.ee> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 00:18:31 -0000 On Tue, 22 Apr 2003, Narvi wrote: > On Mon, 21 Apr 2003, Geoffrey C. Speicher wrote: > > > On Tue, 22 Apr 2003, Narvi wrote: > > > > > At least for the moment it is not clear that one of teh libraries is a > > > winner and will eliminate the other, so it would be evil to force people > > > > What's not clear about it? libkse is a superset of the functionality > > of libthr. Seems pretty straightforward to me that the long-term > > winner is libkse. > > > > This assumes that libkse M:N model will provide supperior performance and > scalability, and this is not clear. Or does merely the fact that itis M:N > somehow make it more winning contender? The whole idea behind implementing M:N is performance and scalability under heavy loads. True, whether or not it beats out anything else in the short term may be unclear. But the mere fact that it is M:N indicates to me that it is the target to shoot for. Geoff From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 17:25:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8428737B401 for ; Mon, 21 Apr 2003 17:25:35 -0700 (PDT) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id CD7AE43FA3 for ; Mon, 21 Apr 2003 17:25:34 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0346.cvx21-bradley.dialup.earthlink.net ([209.179.193.91] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197lbD-0000ld-00; Mon, 21 Apr 2003 17:25:28 -0700 Message-ID: <3EA48BA6.E152E654@mindspring.com> Date: Mon, 21 Apr 2003 17:24:06 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Narvi References: <20030422004324.I29990-100000@haldjas.folklore.ee> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a400de5aba062d2b40ac5e1ad87aacbf8d350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: Daniel Eischen cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 00:25:35 -0000 Narvi wrote: > Solaris libthr is not a 1:1 thread library - the difference between > libpthread and libthread is that libthread implements UI (sometimes also > called Solaris) threads, and not pthreads. And back in SunOS 4.1.3_U2 days, and carried forward for binary compatability for a while, there was a "liblwp", which was a user space call conversion threads library, very much like our modern FreeBSD libc_r. Everything old is new again... 8-). > > I would object to libpthread{M1,MM,MN}. We already > > had a name for libpthread. Libthr can live fine as > > libthr or libthread. > > No - if both provide the same API with potentially different > implementation tradeoffs, then ythe libraries should be installed with > different names and a symlink to the prefered one installed. > > At least for the moment it is not clear that one of teh libraries is a > winner and will eliminate the other, so it would be evil to force people > to explicitly link against one or the other causing future compatibility > problems. Both provide the same pthreads API so there is no reasonable > case for demanding that one of them can't have its SONAME be > libpthpread.so.1 FWIW: My post earlier, and Daniel's response, were meant to highlight the fact that there are, in fact, only different tradeoffs at this point because the libkse gets an upcall on block and calls kse_release() in the 1:1 case, but otherwise has identical function as libthr (no other tradeoffs). The upcalls and the kse_release(), as well as UTS entry, could be (and are intended to be) eliminated in the PTHREAD_SCOPE_SYSTEM case. After that, there are *NO* implementation tradeoffs between the two; libkse is a complete superset of libthr. The only real difference, after that, is that the signals in libthr are questionable (i.e. they have been questioned but not answered to everyone's satisfaction), and that it's impossible for libthr to implement the proper POSIX compliant semantics for PTHREAD_SCOPE_PROCESS. It's really important for people to understand both implementations well enough to be able to comment on them, before claiming differences that are really not there. -- Terry From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 17:30:40 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 44F5437B407 for ; Mon, 21 Apr 2003 17:30:40 -0700 (PDT) Received: from speicher.org (sirius.speicher.org [209.74.10.51]) by mx1.FreeBSD.org (Postfix) with ESMTP id D648543FE9 for ; Mon, 21 Apr 2003 17:30:38 -0700 (PDT) (envelope-from geoff@speicher.org) Received: from localhost (geoff@localhost) by speicher.org (8.11.6/8.11.6) with ESMTP id h3M0j6Q60496; Mon, 21 Apr 2003 20:45:06 -0400 (EDT) (envelope-from geoff@speicher.org) Date: Mon, 21 Apr 2003 20:45:06 -0400 (EDT) From: "Geoffrey C. Speicher" To: Jeff Roberson In-Reply-To: <20030421190831.S76635-100000@mail.chesapeake.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Daniel Eischen cc: Julian Elischer Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 00:30:40 -0000 On Mon, 21 Apr 2003, Jeff Roberson wrote: > On Mon, 21 Apr 2003, Geoffrey C. Speicher wrote: > > > On Tue, 22 Apr 2003, Narvi wrote: > > > > > At least for the moment it is not clear that one of teh libraries is a > > > winner and will eliminate the other, so it would be evil to force people > > > > What's not clear about it? libkse is a superset of the functionality > > of libthr. Seems pretty straightforward to me that the long-term > > winner is libkse. > > Uh, no, it's not that clear. One is probably 30k lines of code smaller > which makes it less likely to have bugs. They also have different > performance characteristics. You could construct tests which demonstrated > better performance for different workloads on each. I will not deny that you are correct on all of the above, but bugs will be fixed, and inefficiencies will be corrected. Then we will have two very similar things in terms of stability and performance under 1:1 threading. One will also have support for 1:N and M:N, and the other will not. I do not mean to take anything away from your work, which is valuable for many reasons, but I really do think that the long-term goal should be more cooperation to make libkse perform as well as libthr in the 1:1 case once everything has shaken out rather than maintain them under separate tents. Geoff From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 17:49:26 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 43DEF37B401 for ; Mon, 21 Apr 2003 17:49:26 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6983F43FAF for ; Mon, 21 Apr 2003 17:49:25 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0346.cvx21-bradley.dialup.earthlink.net ([209.179.193.91] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197lyA-0000Kb-00; Mon, 21 Apr 2003 17:49:11 -0700 Message-ID: <3EA49134.71A5BDC8@mindspring.com> Date: Mon, 21 Apr 2003 17:47:48 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Narvi References: <20030422020939.S29990-100000@haldjas.folklore.ee> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4cb31d744f6166ad1e0da695f36c568db350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: "Geoffrey C. Speicher" cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 00:49:26 -0000 Narvi wrote: > On Mon, 21 Apr 2003, Geoffrey C. Speicher wrote: > > What's not clear about it? libkse is a superset of the functionality > > of libthr. Seems pretty straightforward to me that the long-term > > winner is libkse. > > This assumes that libkse M:N model will provide supperior performance and > scalability, and this is not clear. Or does merely the fact that itis M:N > somehow make it more winning contender? See other discussion. Performance should be identical, after the upcall is eliminated in the case where all threads are created PTHREAD_SCOPE_SYSTEM, and libkse communicates this to the kernel to avoid generation of upcalls on blocking operations. Jeff makes a good point about the potential for bugs because of the more sophisticated and larger total number of lines of code, however. Daniel also makes a good point about not finding them unless the code is put in production. To my mind, it comes down to bug avoidance vs. bug detection, if you weigh these arguments against each other. -- Terry From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 17:56:47 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0698837B401; Mon, 21 Apr 2003 17:56:46 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3C76143FA3; Mon, 21 Apr 2003 17:56:46 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3M0ujBg017167; Mon, 21 Apr 2003 20:56:45 -0400 (EDT) Received: from localhost (eischen@localhost)h3M0ujtv017163; Mon, 21 Apr 2003 20:56:45 -0400 (EDT) Date: Mon, 21 Apr 2003 20:56:45 -0400 (EDT) From: Daniel Eischen To: threads@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: davidxu@freebsd.org Subject: X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 00:56:47 -0000 David, I was able to get this panic running my test program (thread_test2.c posted earlier) with 10 scope system threads. Sometimes you can get it with 4 or 5. [ copied by hand ] panic: blockable sleep lock (sleep mutex) Giant@vm_fault.c:210 Debugger(c04018dc) at Debugger+0x45 db>trace panic(c0404cc, c0414ba0, c04010e4, c0414670, d2) at panic+0x9f witness_lock(c0490d20, 8, c0414670, d2, 2) at witness_lock+0x92 _mtx_lock_flags(c0490d20, 0, c0414670, d2) at _mtx_lock_flags+0x7f vm_fault(c082f000, d0d10000, 2, 0, c1c55720) at vm_fault+0x4a trap_pfault(cc9b7bf4, 0, d0d1d48) at trap_pfault+0x111 trap(c02a0018, 10, 10, d0d1d48, cc9b7d48) at trap+0x379 calltrap() at calltrap+0x5 --- trap 0xc, eip = 0xc03b001a, esp = 0xcc96734m ebp = 0xcc9b7cbc --- generic_bcopy(c1c55720, c1f97210) at generic_bcopy+0x1a kse_create(c1c55720, cc9b7d14, 2, 1, 286) at kse_create+0x277 syscall(2f, 2f, 2f, 804d000, 807c000) at syscall+0x1f1 Xint0x80_syscall() at Xint0x80_syscall+0x1d --- syscall(381, FreeBSD ELF32, kse_create), eip = 0x280cc17f, esp = 0xbfbff9cc, ebp = 0xbfbffaf8 -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 17:57:23 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BA6E937B401 for ; Mon, 21 Apr 2003 17:57:23 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 361B743F75 for ; Mon, 21 Apr 2003 17:57:23 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0346.cvx21-bradley.dialup.earthlink.net ([209.179.193.91] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197m60-00020Q-00; Mon, 21 Apr 2003 17:57:16 -0700 Message-ID: <3EA49320.F62E1B1E@mindspring.com> Date: Mon, 21 Apr 2003 17:56:00 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Narvi References: <20030422015621.P29990-100000@haldjas.folklore.ee> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a435653d636b67524dc984efc4edb776a2350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 00:57:24 -0000 Narvi wrote: > If by "now" you mean Solaris 9, then yes, this is so. This is not a > fundamental issue, merely how kernel API-s are used. On Solaris 8 you get > both the "old" M:N version and the Solaris 9 style 1:1 version in > /usr/lib/lwp. There is no way to tell what it will be in Solaris 9+x for > some arbitrary positive value of x. Right. The liblwp in SunOS 4.1.3_U2 (first appearing in SunOS 4.0.2, I believe) is a totally different liblwp, as well (just the same name); LWP used to be a purely user space abstraction. Sun did this same thing in a different order; Linux too: FreeBSD libc libc_r(N:1) libthr(1:1) libkse(N:M) Sun libc liblwp(N:1) liblwp(N:M) liblwp(1:1) Linux libc pthreads(N:1) pthreads(1:1) - (no N:M at all) I've heard it anecdotally claimed that Sun made the change to avoid bugs; I've also heard it anecdotally claimed by Sun engineers that they made the change because they don't have the man power remaining to perform ordinary maintenance on a lot of their existing code base. Perhaps they were just disgruntled, and the first excuse is the correct one... ;^). -- Terry From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 19:09:58 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 836F537B401 for ; Mon, 21 Apr 2003 19:09:58 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 961AA43FB1 for ; Mon, 21 Apr 2003 19:09:39 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id JM648R72; Tue, 22 Apr 2003 09:54:40 +0800 Message-ID: <00be01c30874$768d8300$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" , References: Date: Tue, 22 Apr 2003 10:11:19 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MIMEOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 Subject: Re: kse_create X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 02:09:58 -0000 I think this is the VM subsystem falls on its face, it seems Giant lock is biting them again.=20 I can not find problem in kse_create, it is the most thinked KSE code. David Xu ----- Original Message -----=20 From: "Daniel Eischen" To: Cc: Sent: Tuesday, April 22, 2003 8:56 AM > David, >=20 > I was able to get this panic running my test program=20 > (thread_test2.c posted earlier) with 10 scope system > threads. Sometimes you can get it with 4 or 5. >=20 > [ copied by hand ] >=20 > panic: blockable sleep lock (sleep mutex) Giant@vm_fault.c:210 > Debugger(c04018dc) at Debugger+0x45 >=20 > db>trace > panic(c0404cc, c0414ba0, c04010e4, c0414670, d2) at panic+0x9f > witness_lock(c0490d20, 8, c0414670, d2, 2) at witness_lock+0x92 > _mtx_lock_flags(c0490d20, 0, c0414670, d2) at _mtx_lock_flags+0x7f > vm_fault(c082f000, d0d10000, 2, 0, c1c55720) at vm_fault+0x4a > trap_pfault(cc9b7bf4, 0, d0d1d48) at trap_pfault+0x111 > trap(c02a0018, 10, 10, d0d1d48, cc9b7d48) at trap+0x379 > calltrap() at calltrap+0x5 > --- trap 0xc, eip =3D 0xc03b001a, esp =3D 0xcc96734m ebp =3D = 0xcc9b7cbc --- > generic_bcopy(c1c55720, c1f97210) at generic_bcopy+0x1a > kse_create(c1c55720, cc9b7d14, 2, 1, 286) at kse_create+0x277 > syscall(2f, 2f, 2f, 804d000, 807c000) at syscall+0x1f1 > Xint0x80_syscall() at Xint0x80_syscall+0x1d > --- syscall(381, FreeBSD ELF32, kse_create), eip =3D 0x280cc17f, esp = =3D 0xbfbff9cc, ebp =3D 0xbfbffaf8 >=20 > --=20 > Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 20:01:55 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id AC1BD37B401 for ; Mon, 21 Apr 2003 20:01:55 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1EB3B43FDF for ; Mon, 21 Apr 2003 20:01:53 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h3M31pUE033115; Tue, 22 Apr 2003 06:01:51 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h3M31p2v033112; Tue, 22 Apr 2003 06:01:51 +0300 (EEST) Date: Tue, 22 Apr 2003 06:01:51 +0300 (EEST) From: Narvi To: Terry Lambert In-Reply-To: <3EA49320.F62E1B1E@mindspring.com> Message-ID: <20030422055559.L33034-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 03:01:56 -0000 On Mon, 21 Apr 2003, Terry Lambert wrote: > Narvi wrote: > > If by "now" you mean Solaris 9, then yes, this is so. This is not a > > fundamental issue, merely how kernel API-s are used. On Solaris 8 you get > > both the "old" M:N version and the Solaris 9 style 1:1 version in > > /usr/lib/lwp. There is no way to tell what it will be in Solaris 9+x for > > some arbitrary positive value of x. > > Right. The liblwp in SunOS 4.1.3_U2 (first appearing in SunOS 4.0.2, > I believe) is a totally different liblwp, as well (just the same name); > LWP used to be a purely user space abstraction. > I don't think there is a liblwp in modern solaris, lwp remains only as a name for kernel threads. /usr/lib/lwp is just a directory containing alternative threads libraries on solaris 8 and is a symbolic link to /usr/lib on solaris 9. > > Sun did this same thing in a different order; Linux too: > > FreeBSD libc libc_r(N:1) libthr(1:1) libkse(N:M) > Sun libc liblwp(N:1) liblwp(N:M) liblwp(1:1) > Linux libc pthreads(N:1) pthreads(1:1) - (no N:M at all) > > I've heard it anecdotally claimed that Sun made the change to > avoid bugs; I've also heard it anecdotally claimed by Sun > engineers that they made the change because they don't have > the man power remaining to perform ordinary maintenance on a > lot of their existing code base. Perhaps they were just > disgruntled, and the first excuse is the correct one... ;^). > the claim in the man page(s)/whitepaper(s) is about perfomance though - but as always, these would be claims about a particular implementation and freebsd one might not run into whichever problems caused the speed loss on solaris. > -- Terry > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 20:25:39 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1074637B404 for ; Mon, 21 Apr 2003 20:25:39 -0700 (PDT) Received: from haldjas.folklore.ee (Haldjas.folklore.ee [193.40.6.121]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8455C43FAF for ; Mon, 21 Apr 2003 20:25:37 -0700 (PDT) (envelope-from narvi@haldjas.folklore.ee) Received: from haldjas.folklore.ee (localhost [127.0.0.1]) by haldjas.folklore.ee (8.12.3/8.11.3) with ESMTP id h3M3PaUE033209; Tue, 22 Apr 2003 06:25:36 +0300 (EEST) (envelope-from narvi@haldjas.folklore.ee) Received: from localhost (narvi@localhost)h3M3PZxi033206; Tue, 22 Apr 2003 06:25:36 +0300 (EEST) Date: Tue, 22 Apr 2003 06:25:35 +0300 (EEST) From: Narvi To: Terry Lambert In-Reply-To: <3EA49134.71A5BDC8@mindspring.com> Message-ID: <20030422060235.Y33034-100000@haldjas.folklore.ee> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 03:25:39 -0000 On Mon, 21 Apr 2003, Terry Lambert wrote: > Narvi wrote: > > On Mon, 21 Apr 2003, Geoffrey C. Speicher wrote: > > > What's not clear about it? libkse is a superset of the functionality > > > of libthr. Seems pretty straightforward to me that the long-term > > > winner is libkse. > > > > This assumes that libkse M:N model will provide supperior performance and > > scalability, and this is not clear. Or does merely the fact that itis M:N > > somehow make it more winning contender? > > See other discussion. Performance should be identical, after > the upcall is eliminated in the case where all threads are > created PTHREAD_SCOPE_SYSTEM, and libkse communicates this to > the kernel to avoid generation of upcalls on blocking operations. > > Jeff makes a good point about the potential for bugs because of > the more sophisticated and larger total number of lines of code, > however. > Unless something went inheremtly wrong in case of the solaris threading libraries (or the published information is wrong), there is the potential for the additional code and complexity to add sufficent overhead to be measurable as perfomance degradation. It could be such might happen only with large values of M and N (and thus not surface in the next K years in a serious way in FreeBSD) or it could show up also on relatively small values of M and N or not at all. There are some unknowns in the picture, and say assigning a non-trivial likelyhood to at some point all processorts being 8-way (or even 4-way) EV8 style SMT would change some aspects of it. > Daniel also makes a good point about not finding them unless the > code is put in production. > > To my mind, it comes down to bug avoidance vs. bug detection, if > you weigh these arguments against each other. > I prefer bug avoidance and spending the saved time on perfomance - but this is for the moment a preference of somebody without time to spend on either thread library. > -- Terry > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 21:10:39 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E7C0337B401 for ; Mon, 21 Apr 2003 21:10:39 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 622D943FDF for ; Mon, 21 Apr 2003 21:10:39 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0085.cvx40-bradley.dialup.earthlink.net ([216.244.42.85] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197p72-0001DL-00; Mon, 21 Apr 2003 21:10:33 -0700 Message-ID: <3EA4C06B.F607710C@mindspring.com> Date: Mon, 21 Apr 2003 21:09:15 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Narvi References: <20030422055559.L33034-100000@haldjas.folklore.ee> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4a0d841dab08150fd9c4c09501af161e6350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 04:10:40 -0000 Narvi wrote: > On Mon, 21 Apr 2003, Terry Lambert wrote: > > Narvi wrote: > > > If by "now" you mean Solaris 9, then yes, this is so. This is not a > > > fundamental issue, merely how kernel API-s are used. On Solaris 8 you get > > > both the "old" M:N version and the Solaris 9 style 1:1 version in > > > /usr/lib/lwp. There is no way to tell what it will be in Solaris 9+x for > > > some arbitrary positive value of x. > > > > Right. The liblwp in SunOS 4.1.3_U2 (first appearing in SunOS 4.0.2, > > I believe) is a totally different liblwp, as well (just the same name); > > LWP used to be a purely user space abstraction. > > > > I don't think there is a liblwp in modern solaris, lwp remains only as a > name for kernel threads. /usr/lib/lwp is just a directory containing > alternative threads libraries on solaris 8 and is a symbolic link to > /usr/lib on solaris 9. 4.1.3_U2 and 4.0.2 are circa the late 1980's. > > I've heard it anecdotally claimed that Sun made the change to > > avoid bugs; I've also heard it anecdotally claimed by Sun > > engineers that they made the change because they don't have > > the man power remaining to perform ordinary maintenance on a > > lot of their existing code base. Perhaps they were just > > disgruntled, and the first excuse is the correct one... ;^). > > the claim in the man page(s)/whitepaper(s) is about perfomance though - > but as always, these would be claims about a particular implementation and > freebsd one might not run into whichever problems caused the speed loss on > solaris. It wouldn't. The main issue as far as performance went, and why we (Novell USG) used processes instead of SVR4 threads, and did file descriptor table sharing, and shared client context data in a shared memory segment (8-)) is that SVR4-derived systems without a unified VM and buffer cache do a lot of page thrashing. One of the "innovations" in Solaris 9, if you read the white papers, is that they reintroduced a seperate buffer cache. It makes sense that in doing that, they would reintroduce the same performance problems that came from having a seperate buffer cache in the first place. Probably the correct thing to do instead would have been to introduce a seperate working set quota on file objects, so a single file being randomly accessed couldn't LRU out all the other files in the system. 8-). -- Terry From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 21:28:06 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE55C37B401 for ; Mon, 21 Apr 2003 21:28:06 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 380AB43FB1 for ; Mon, 21 Apr 2003 21:28:06 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0085.cvx40-bradley.dialup.earthlink.net ([216.244.42.85] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197pNy-0003By-00; Mon, 21 Apr 2003 21:28:03 -0700 Message-ID: <3EA4C486.C9AD39C0@mindspring.com> Date: Mon, 21 Apr 2003 21:26:46 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Narvi References: <20030422060235.Y33034-100000@haldjas.folklore.ee> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4af22abddeaf2b94b624d5faadfef232d93caf27dac41a8fd350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 04:28:07 -0000 Narvi wrote: > > Jeff makes a good point about the potential for bugs because of > > the more sophisticated and larger total number of lines of code, > > however. > > Unless something went inheremtly wrong in case of the solaris threading > libraries (or the published information is wrong), there is the potential > for the additional code and complexity to add sufficent overhead to be > measurable as perfomance degradation. It could be such might happen only > with large values of M and N (and thus not surface in the next K years in > a serious way in FreeBSD) or it could show up also on relatively small > values of M and N or not at all. Actually, we're only talking about FreeBSD threading, and we're talking about the degenerate case in N:M where N == M == 1. With one exception, which is just an optimization that hasn't happened yet, the UTS totally drops out, and the subset of the the code that is called drops down to the same as libthr. This is pretty much on purpose. So the complexity argument isn't really valid (after the additional optimization to let the kernel know that all threads are created as PTHREAD_SCOPE_SYSTEM). There's not really any additional overhead. Right now, ther *is* measurable overhead, but only because there's an upcall generated on a block, and the upcall is immediately terminated with a kse_release() call. That's the optimization that hasn't been done. > There are some unknowns in the picture, and say assigning a non-trivial > likelyhood to at some point all processorts being 8-way (or even 4-way) > EV8 style SMT would change some aspects of it. Not really. The system call path would be essentially identical, and so the overhead. If you really want to talk about 8-way or even NUMA, well, the N:M model "rules" in that case, so long as you ensure that your KSEGRP allocation follows the NUMA cluster in terms of grouping mutually accessible shared memory. At this point, though, FreeBSD is pretty shared-memory-multiprocessor centric in its design, and it's not very amenable to clustering (even NUMA clustering) and it's current schedulers aren't really up to node-migration of tasks. On a PPC architecture, too, the coherency model is probably really completely broken; probably so for any architecture that doesn't comply with the Intel MPSpec 1.4. > > Daniel also makes a good point about not finding them unless the > > code is put in production. > > > > To my mind, it comes down to bug avoidance vs. bug detection, if > > you weigh these arguments against each other. > > I prefer bug avoidance and spending the saved time on perfomance - but > this is for the moment a preference of somebody without time to spend on > either thread library. If you can prove something is correct, then you've eliminated the bugs. If you can only prove that you've avoided some bugs, you can't prove correctness from that. It's really a false savings; it's the DJB "qmail" argument all over again. I like to think engineers are capable of dealing with complexity, and that they don't need to be boxed into little sandboxes, like children, to keep them from hurting themselves. 8-) 8-). -- Terry From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 21:50:55 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0D15537B401 for ; Mon, 21 Apr 2003 21:50:55 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5590143FBF for ; Mon, 21 Apr 2003 21:50:54 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3M4oqq66523; Tue, 22 Apr 2003 00:50:52 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Tue, 22 Apr 2003 00:50:52 -0400 (EDT) From: Jeff Roberson To: Terry Lambert In-Reply-To: <3EA4C06B.F607710C@mindspring.com> Message-ID: <20030422004950.R76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 04:50:55 -0000 On Mon, 21 Apr 2003, Terry Lambert wrote: > It wouldn't. The main issue as far as performance went, and why > we (Novell USG) used processes instead of SVR4 threads, and did > file descriptor table sharing, and shared client context data in > a shared memory segment (8-)) is that SVR4-derived systems without > a unified VM and buffer cache do a lot of page thrashing. Please explain how using processes instead of threads improves page thrashing. > > One of the "innovations" in Solaris 9, if you read the white papers, > is that they reintroduced a seperate buffer cache. It makes sense > that in doing that, they would reintroduce the same performance > problems that came from having a seperate buffer cache in the first > place. > > Probably the correct thing to do instead would have been to introduce > a seperate working set quota on file objects, so a single file being > randomly accessed couldn't LRU out all the other files in the system. > 8-). > > -- Terry > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Mon Apr 21 23:52:52 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0298137B401 for ; Mon, 21 Apr 2003 23:52:51 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4136A43FAF for ; Mon, 21 Apr 2003 23:52:51 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0155.cvx21-bradley.dialup.earthlink.net ([209.179.192.155] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 197rdz-0001xb-00; Mon, 21 Apr 2003 23:52:44 -0700 Message-ID: <3EA4E66B.52980656@mindspring.com> Date: Mon, 21 Apr 2003 23:51:23 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson References: <20030422004950.R76635-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a419c7bf81e9da2477fd055b377f65efef350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 06:52:52 -0000 Jeff Roberson wrote: > On Mon, 21 Apr 2003, Terry Lambert wrote: > > It wouldn't. The main issue as far as performance went, and why > > we (Novell USG) used processes instead of SVR4 threads, and did > > file descriptor table sharing, and shared client context data in > > a shared memory segment (8-)) is that SVR4-derived systems without > > a unified VM and buffer cache do a lot of page thrashing. > > Please explain how using processes instead of threads improves page > thrashing. SVR4.0.2 (Dell UNIX) and SVR4.2 (UnixWare 2.x) have a seperate VM and buffer cache. Because of this, you tend to get page thrashing under any overload condition, even for nominally shared code pages, if you are doing a lot of data pages work. The problem is most easily seen in the UnixWare 1.x, prior to the introduction of the "fixed" scheduling class. In order to put memory pressure on the system, in a UI-visible way, run X Windows, and then perform a compilation on a large project. When the ld program is run, it will mmap() all of the .o files, and then randomly access them in quick succession, in order to perform symbol resolution for the large project. When this happens, the UI will "lock up", and you will effectively lose the ability to move the mouse. As you attempt to move the mouse, the mouse will not move, and it will trigger paging in of the X server, and then paging in of the application, both of whose code pages were forced out of core (and will be forced back out of core again, immediately) by the ld's access to data pages. So once every one or two seconds or so, it will move, generate expose events, and lock up again. The net effect is the system appears to lock up, either completely, or for multiple seconds at a time. The UnixWare 2.x/SVR4.2 solution to this problem is to introduce a "fixed" scheduling class, so that a fixed percentage of the CPU time is dedicated to processing the X server code. This doesn't stop it from being paged out, but it does provide a fixed amount of CPU to spend paging it back in, and then doing some processing on top of that (basically, I/O is accounted on a per process basis). This was basically a lazy way of introducing a "precious" working set low watermark for the X server pages. Much better to have established a per-file quota for the .o files themselves, so ld might thrash, but the only program that would get hurt by it would be ld. Now consider the specific case we were dealing with, which is the NetWare for UNIX (formerly Portable NetWare) problem. In this case, if we were to use the SVR4/Solaris threading (this was after the merge of the Solaris and SVR4.2 code bases, as part of a joint project between Sun and USL, in which Sun got VM and FS code, and USL got the threads and some other code, in trade). The implementation paradigm for this code was as "anonymous work to do engines" -- essentially, the server consisted of a number of specific tasks (an intention mode transaction based long manager, a monitoring daemon, some miscellaneous tasks), and a number of identical tasks which implemented "work to do engines" -- all the latter tasks were identical, in that the client context for any client session was known to all of them. As a result, any of these tasks could service any request. Since the NCP packets are, with the sole exception of delayed lock grants, which are reported async via a covert channel, request/response in architecture, the number of concurrent client requests is limited by the number of tasks that are available to service the requests. Our intent was to be able to service a large number of clients. Now consider that, while maximum concurrency was an issue, so was locality of file data sets, and locality of code pages, with the two contending for the limited available divided memory pools that were contended between the VM and buffer cache (effectively, there was a total set size, with a reserve held back for each type of pages, and the remaining pages were contended). Use of the "fixed" scheduling class was not an option. Using threads would not allow prefferential scheduling between the tasks, neither would it have allowed sharing of all client context (though it would have allowed descriptor sharing) without some form of marshalling and locking. This is because a client that did not believe the server was responding "fast enough" would repeat the request. It was necessary to respond to these clients with a "server busy" message. The reason behind this because IPX is a unreliable datagram protocol, like UDP, and does not have a retry mechanism built into it. The upshot of this is that, with threads, the per process working set would be very large, and would be fragmented across the process address space. This increased contention, well above what a process could withstand, without forcing VM pressure on the buffer cache. But the reason for the existance of the software was a *file* server, so this was unacceptable. By seperating the address spaces, this pressure was reduced, and the amount of overall contention was reduced, thus reducing the buffer cache pressure from the processes. In the limit, with all processes fully utilized (i.e. a request backlog at the stream MUX), it equalled out in performance. In the common case, however, not all tasks were utilized all the time, and it was possible to allow them to be paged out. On top of this, there were a number of speed benefits to System V shared memory for the client contexts; if you have read "The Magic Garden Explained", these should be pretty obvious. John Dyson made a number of similar changes to the FreeBSD implementation for Oracle Corporation, when Oracle was using FreeBSD as the basis of its "Network Computer" server. Basically, the pages are VM pages only, with no write-through to the backing store; in the SVR4 case, this would have been buffer cache pages, backed by swap, if this were anonymous memory instead (the kind you get in a threads heap). Thus we come to part 2, which is that we modified the streams MUX to ensure that requests were assigned to engines as they entered the stream mux FD with a write+read request in LIFO, rather than FIFO order. By doing this, were able to ensure that, most likely, the pages which were going to be requested were "in core" in the process making the request (performing default FIFO ordering would have resulted in a guarantee that the pages were not in core). I dubbed this approach "hot engine scheduling". Attempting to use a similar approach in the threads case, besides the completely fragmented process memory that caused a much larger number of pages to need to be resident to do the same work, the MUX assignment of "work to do" would in fact have been effectively "random". Anything less than total utilization of the system was *worse* with random allocation of work units, and *better* with LIFO allocation. And that's why using processes instead of threads resulted in less page thrashing. There were, of course, other reasons for using processes, instead of threads, the primary among which was "better quantum utilization" (SVR4.2/UnixWare 2.0 did not support thread group affinity in the scheduler; as you have discovered, supporting that is NP-hard, unless you get tricky, and make migration explicit and initial selection intentional). -- Terry From owner-freebsd-threads@FreeBSD.ORG Tue Apr 22 01:18:17 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 00A2A37B401 for ; Tue, 22 Apr 2003 01:18:17 -0700 (PDT) Received: from park.rambler.ru (park.rambler.ru [81.19.64.101]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6078243FAF for ; Tue, 22 Apr 2003 01:18:15 -0700 (PDT) (envelope-from is@rambler-co.ru) Received: from is.park.rambler.ru (is.park.rambler.ru [81.19.64.102]) by park.rambler.ru (8.12.6/8.12.6) with ESMTP id h3M8IAmF080489; Tue, 22 Apr 2003 12:18:10 +0400 (MSD) Date: Tue, 22 Apr 2003 12:18:10 +0400 (MSD) From: Igor Sysoev X-Sender: is@is To: Daniel Eischen In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 08:18:17 -0000 On Mon, 21 Apr 2003, Daniel Eischen wrote: > On Mon, 21 Apr 2003, Terry Lambert wrote: > > > Jeff Roberson wrote: > > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > > Since libkse seems to be generally useful, anyone mind if I > > > > go back to installing it as libpthread? > > > > > > There is some question over whether kse or thr will be the default > > > threading implementation for 5.1. I think this should be discussed before > > > we decide on what lib uses libpthread. > > > > Isn't libthr really just libkse with N = M? > > Yes, I wasn't going to bring that up, though. > > > These should behave exactly the same, right? It should even be > > possible to drop the UTS out of the picture for everything but > > signal handling, I think (i.e. it would never get upcalled) in > > the 1:1 case, if this were done? > > Libpthread can be made to behave the same as libthr just > by forcing every thread to be scope system. Currently, > the implementation for scope system threads does have > a small amount of overhead in that they still get upcalls > after the thread blocks in the kernel (in this case > the KSE just reenters the kernel with kse_release() > and waits for the thread to become unblocked). These > KSEs also require a small stack separate from the > thread's stack. The code is in place (in the UTS) to > not require a separate stack and not get any upcalls > for these threads, but we just need a bit more > kernel work to optimize this overhead away. But why is not it implemented via setting kse_mailbox.km_curthread to NULL ? As I understand it's way to disable upcalls when UTS is preempted by the kernel (the time slice ended, the page in operation, etc.) i.e. UTS should always run as 1:1 thread. Had it been changed ? How is UTS protected now ? Igor Sysoev http://sysoev.ru/en/ From owner-freebsd-threads@FreeBSD.ORG Tue Apr 22 05:55:53 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EFD5E37B401 for ; Tue, 22 Apr 2003 05:55:53 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 36DDE43F3F for ; Tue, 22 Apr 2003 05:55:53 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3MCtqBg027967; Tue, 22 Apr 2003 08:55:52 -0400 (EDT) Received: from localhost (eischen@localhost)h3MCtpYR027964; Tue, 22 Apr 2003 08:55:52 -0400 (EDT) Date: Tue, 22 Apr 2003 08:55:51 -0400 (EDT) From: Daniel Eischen To: Igor Sysoev In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 12:55:54 -0000 On Tue, 22 Apr 2003, Igor Sysoev wrote: > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > Libpthread can be made to behave the same as libthr just > > by forcing every thread to be scope system. Currently, > > the implementation for scope system threads does have > > a small amount of overhead in that they still get upcalls > > after the thread blocks in the kernel (in this case > > the KSE just reenters the kernel with kse_release() > > and waits for the thread to become unblocked). These > > KSEs also require a small stack separate from the > > thread's stack. The code is in place (in the UTS) to > > not require a separate stack and not get any upcalls > > for these threads, but we just need a bit more > > kernel work to optimize this overhead away. > > But why is not it implemented via setting kse_mailbox.km_curthread to NULL ? > As I understand it's way to disable upcalls when UTS is preempted > by the kernel (the time slice ended, the page in operation, etc.) > i.e. UTS should always run as 1:1 thread. > Had it been changed ? How is UTS protected now ? We can't use this method yet for 1:1 because we don't have a way to redirect a signal to a thread running like this. We need kse_thr_interrupt() to have a signal argument and for the kernel to install the signal frame on the stack of the target thread (or keep it pending on the thread if masked). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Apr 22 14:51:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A9DF37B401 for ; Tue, 22 Apr 2003 14:51:19 -0700 (PDT) Received: from mail.chesapeake.net (chesapeake.net [205.130.220.14]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2DC9F43F75 for ; Tue, 22 Apr 2003 14:51:18 -0700 (PDT) (envelope-from jroberson@chesapeake.net) Received: from localhost (jroberson@localhost) by mail.chesapeake.net (8.11.6/8.11.6) with ESMTP id h3MLpFw87155; Tue, 22 Apr 2003 17:51:15 -0400 (EDT) (envelope-from jroberson@chesapeake.net) Date: Tue, 22 Apr 2003 17:51:15 -0400 (EDT) From: Jeff Roberson To: Terry Lambert In-Reply-To: <3EA4E66B.52980656@mindspring.com> Message-ID: <20030422175008.P76635-100000@mail.chesapeake.net> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 21:51:19 -0000 I think your keyboard needs a rate limiter. ;-) So the point of all this is that the vm performed better with processes because it was better able to determine the resident set of each? On Mon, 21 Apr 2003, Terry Lambert wrote: > Jeff Roberson wrote: > > On Mon, 21 Apr 2003, Terry Lambert wrote: > > > It wouldn't. The main issue as far as performance went, and why > > > we (Novell USG) used processes instead of SVR4 threads, and did > > > file descriptor table sharing, and shared client context data in > > > a shared memory segment (8-)) is that SVR4-derived systems without > > > a unified VM and buffer cache do a lot of page thrashing. > > > > Please explain how using processes instead of threads improves page > > thrashing. > > SVR4.0.2 (Dell UNIX) and SVR4.2 (UnixWare 2.x) have a seperate VM > and buffer cache. Because of this, you tend to get page thrashing > under any overload condition, even for nominally shared code pages, > if you are doing a lot of data pages work. > > The problem is most easily seen in the UnixWare 1.x, prior to the > introduction of the "fixed" scheduling class. In order to put > memory pressure on the system, in a UI-visible way, run X Windows, > and then perform a compilation on a large project. When the ld > program is run, it will mmap() all of the .o files, and then randomly > access them in quick succession, in order to perform symbol resolution > for the large project. When this happens, the UI will "lock up", and > you will effectively lose the ability to move the mouse. As you > attempt to move the mouse, the mouse will not move, and it will trigger > paging in of the X server, and then paging in of the application, both > of whose code pages were forced out of core (and will be forced back > out of core again, immediately) by the ld's access to data pages. So > once every one or two seconds or so, it will move, generate expose > events, and lock up again. > > The net effect is the system appears to lock up, either completely, > or for multiple seconds at a time. > > The UnixWare 2.x/SVR4.2 solution to this problem is to introduce a > "fixed" scheduling class, so that a fixed percentage of the CPU time > is dedicated to processing the X server code. This doesn't stop it > from being paged out, but it does provide a fixed amount of CPU to > spend paging it back in, and then doing some processing on top of > that (basically, I/O is accounted on a per process basis). This > was basically a lazy way of introducing a "precious" working set low > watermark for the X server pages. Much better to have established > a per-file quota for the .o files themselves, so ld might thrash, > but the only program that would get hurt by it would be ld. > > > Now consider the specific case we were dealing with, which is the > NetWare for UNIX (formerly Portable NetWare) problem. In this case, > if we were to use the SVR4/Solaris threading (this was after the > merge of the Solaris and SVR4.2 code bases, as part of a joint > project between Sun and USL, in which Sun got VM and FS code, and > USL got the threads and some other code, in trade). > > The implementation paradigm for this code was as "anonymous work > to do engines" -- essentially, the server consisted of a number of > specific tasks (an intention mode transaction based long manager, > a monitoring daemon, some miscellaneous tasks), and a number of > identical tasks which implemented "work to do engines" -- all the > latter tasks were identical, in that the client context for any > client session was known to all of them. As a result, any of these > tasks could service any request. Since the NCP packets are, with > the sole exception of delayed lock grants, which are reported async > via a covert channel, request/response in architecture, the number > of concurrent client requests is limited by the number of tasks that > are available to service the requests. Our intent was to be able to > service a large number of clients. > > Now consider that, while maximum concurrency was an issue, so was > locality of file data sets, and locality of code pages, with the > two contending for the limited available divided memory pools that > were contended between the VM and buffer cache (effectively, there > was a total set size, with a reserve held back for each type of > pages, and the remaining pages were contended). > > Use of the "fixed" scheduling class was not an option. > > Using threads would not allow prefferential scheduling between the > tasks, neither would it have allowed sharing of all client context > (though it would have allowed descriptor sharing) without some form > of marshalling and locking. This is because a client that did not > believe the server was responding "fast enough" would repeat the > request. It was necessary to respond to these clients with a > "server busy" message. The reason behind this because IPX is a > unreliable datagram protocol, like UDP, and does not have a retry > mechanism built into it. > > The upshot of this is that, with threads, the per process working > set would be very large, and would be fragmented across the process > address space. This increased contention, well above what a process > could withstand, without forcing VM pressure on the buffer cache. > But the reason for the existance of the software was a *file* server, > so this was unacceptable. > > By seperating the address spaces, this pressure was reduced, and > the amount of overall contention was reduced, thus reducing the > buffer cache pressure from the processes. > > In the limit, with all processes fully utilized (i.e. a request > backlog at the stream MUX), it equalled out in performance. In > the common case, however, not all tasks were utilized all the > time, and it was possible to allow them to be paged out. > > On top of this, there were a number of speed benefits to System V > shared memory for the client contexts; if you have read "The Magic > Garden Explained", these should be pretty obvious. John Dyson > made a number of similar changes to the FreeBSD implementation for > Oracle Corporation, when Oracle was using FreeBSD as the basis of > its "Network Computer" server. Basically, the pages are VM pages > only, with no write-through to the backing store; in the SVR4 case, > this would have been buffer cache pages, backed by swap, if this > were anonymous memory instead (the kind you get in a threads heap). > > Thus we come to part 2, which is that we modified the streams MUX > to ensure that requests were assigned to engines as they entered > the stream mux FD with a write+read request in LIFO, rather than > FIFO order. By doing this, were able to ensure that, most likely, > the pages which were going to be requested were "in core" in the > process making the request (performing default FIFO ordering would > have resulted in a guarantee that the pages were not in core). I > dubbed this approach "hot engine scheduling". > > Attempting to use a similar approach in the threads case, besides > the completely fragmented process memory that caused a much larger > number of pages to need to be resident to do the same work, the > MUX assignment of "work to do" would in fact have been effectively > "random". > > Anything less than total utilization of the system was *worse* with > random allocation of work units, and *better* with LIFO allocation. > > > And that's why using processes instead of threads resulted in less > page thrashing. > > > There were, of course, other reasons for using processes, instead > of threads, the primary among which was "better quantum utilization" > (SVR4.2/UnixWare 2.0 did not support thread group affinity in the > scheduler; as you have discovered, supporting that is NP-hard, unless > you get tricky, and make migration explicit and initial selection > intentional). > > > -- Terry > From owner-freebsd-threads@FreeBSD.ORG Tue Apr 22 16:46:30 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C269137B401 for ; Tue, 22 Apr 2003 16:46:30 -0700 (PDT) Received: from sccrmhc01.attbi.com (sccrmhc01.attbi.com [204.127.202.61]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6470C43F93 for ; Tue, 22 Apr 2003 16:46:20 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by sccrmhc01.attbi.com (sccrmhc01) with ESMTP id <2003042223461900100e654ie>; Tue, 22 Apr 2003 23:46:19 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA88081; Tue, 22 Apr 2003 16:46:17 -0700 (PDT) Date: Tue, 22 Apr 2003 16:46:15 -0700 (PDT) From: Julian Elischer To: Daniel Eischen In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: libkse -> libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 22 Apr 2003 23:46:31 -0000 On Tue, 22 Apr 2003, Daniel Eischen wrote: > On Tue, 22 Apr 2003, Igor Sysoev wrote: > > > On Mon, 21 Apr 2003, Daniel Eischen wrote: > > > Libpthread can be made to behave the same as libthr just > > > by forcing every thread to be scope system. Currently, > > > the implementation for scope system threads does have > > > a small amount of overhead in that they still get upcalls > > > after the thread blocks in the kernel (in this case > > > the KSE just reenters the kernel with kse_release() > > > and waits for the thread to become unblocked). These > > > KSEs also require a small stack separate from the > > > thread's stack. The code is in place (in the UTS) to > > > not require a separate stack and not get any upcalls > > > for these threads, but we just need a bit more > > > kernel work to optimize this overhead away. > > > > But why is not it implemented via setting kse_mailbox.km_curthread to NULL ? > > As I understand it's way to disable upcalls when UTS is preempted > > by the kernel (the time slice ended, the page in operation, etc.) > > i.e. UTS should always run as 1:1 thread. > > Had it been changed ? How is UTS protected now ? UTS is protected.. this way.. but other threads can disable upcalls by setting a flag in the mailbox > > We can't use this method yet for 1:1 because we don't > have a way to redirect a signal to a thread running > like this. We need kse_thr_interrupt() to have a signal > argument and for the kernel to install the signal frame > on the stack of the target thread (or keep it pending > on the thread if masked). > > -- > Dan Eischen > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Apr 22 22:22:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DF23E37B401 for ; Tue, 22 Apr 2003 22:22:00 -0700 (PDT) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id B9BEC43F85 for ; Tue, 22 Apr 2003 22:21:59 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0352.cvx22-bradley.dialup.earthlink.net ([209.179.199.97] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 198Chc-0001Nk-00; Tue, 22 Apr 2003 22:21:52 -0700 Message-ID: <3EA6229B.A67A58AD@mindspring.com> Date: Tue, 22 Apr 2003 22:20:27 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Jeff Roberson References: <20030422175008.P76635-100000@mail.chesapeake.net> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4c6be438b07462648b6a4cceb64f99012350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: Daniel Eischen Subject: Re: libkse -> libpthreads X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Apr 2003 05:22:01 -0000 Jeff Roberson wrote: > > I think your keyboard needs a rate limiter. ;-) Ask a question, get an answer. 8-) > So the point of all this is that the vm performed better with processes > because it was better able to determine the resident set of each? Not really. It's all relevent to the story. Complex things don't always have simple summaries. That's why you and I get more money than the counter person at McDonalds. If I had to give a short summary, it would be "SVR4-derived systems without a unified VM and buffer cache do a lot of page thrashing". That's what I said in the first place; don't blame me if you asked "how?". 8-) 8-). -- Terry From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 09:38:34 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2EFD237B401 for ; Thu, 24 Apr 2003 09:38:34 -0700 (PDT) Received: from sdf-eu.org (sdf-eu.org [192.94.73.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2AC9543FDF for ; Thu, 24 Apr 2003 09:38:33 -0700 (PDT) (envelope-from meyers@sdf-eu.org) Received: from sdf-eu.org (IDENT:meyers@localhost [127.0.0.1] (may be forged)) by sdf-eu.org (8.12.8/8.11.6) with ESMTP id h3OGadfd025586 for ; Thu, 24 Apr 2003 16:36:39 GMT Received: (from meyers@localhost) by sdf-eu.org (8.12.8/8.12.8/Submit) id h3OGad2V025585 for freebsd-threads@freebsd.org; Thu, 24 Apr 2003 16:36:39 GMT Date: Thu, 24 Apr 2003 16:36:39 +0000 From: John Meyers To: freebsd-threads@freebsd.org Message-ID: <20030424163639.GA24365@SDF-EU.ORG> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.4i Subject: KSE contribution X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2003 16:38:34 -0000 Hi, I want to be guided by someone.I would like to take up the multiple KSEs one per CPU and deliver signals to threads. I am not an experienced hacker but would like to learn. Waiting for you cooperation. -- meyers@sdf-eu.org SDF-EU Public Access UNIX System - http://sdf-eu.org From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 12:28:27 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BBE8637B401 for ; Thu, 24 Apr 2003 12:28:27 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1B24943F93 for ; Thu, 24 Apr 2003 12:28:27 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3OJSQBg013459; Thu, 24 Apr 2003 15:28:26 -0400 (EDT) Received: from localhost (eischen@localhost)h3OJSPc3013455; Thu, 24 Apr 2003 15:28:25 -0400 (EDT) Date: Thu, 24 Apr 2003 15:28:25 -0400 (EDT) From: Daniel Eischen To: John Meyers In-Reply-To: <20030424163639.GA24365@SDF-EU.ORG> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: KSE contribution X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Apr 2003 19:28:28 -0000 On Thu, 24 Apr 2003, John Meyers wrote: > Hi, > I want to be guided by someone.I would like to take up the > multiple KSEs one per CPU and deliver signals to threads. > I am not an experienced hacker but would like to learn. > Waiting for you cooperation. We have multiple KSEs currently. Enable them with pthread_setconcurrency(). Or use scope system threads. We're currently working out issues with signaling, so stay tuned for that. The most help now would be to help test and debug. Run it with real-world applications if you have any. Use it with KDE, konqueror, mozilla, etc in your daily usage and see if anything breaks. Find some performance tests, compare with libc_r, libthr, linuxthreads, etc. Tell us what you find, where you think we need to focus. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 19:42:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 86F0937B401 for ; Thu, 24 Apr 2003 19:42:03 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2290543FBF for ; Thu, 24 Apr 2003 19:42:03 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) Received: from freefall.freebsd.org (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h3P2g2Up085405 for ; Thu, 24 Apr 2003 19:42:02 -0700 (PDT) (envelope-from davidxu@freefall.freebsd.org) Received: (from davidxu@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h3P2g2F6085399 for freebsd-threads@freebsd.org; Thu, 24 Apr 2003 19:42:02 -0700 (PDT) Date: Thu, 24 Apr 2003 19:42:02 -0700 (PDT) From: David Xu Message-Id: <200304250242.h3P2g2F6085399@freefall.freebsd.org> To: freebsd-threads@freebsd.org Subject: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 02:42:03 -0000 I have put a patch to enable userland support SMP scheduling. http://people.freebsd.org/~davidxu/libpthread_smp.diff The patch works on my SMP machine, but not fully tested, and I will work on idle kses stuffs. At least, it seems the SMP speed is not slower than UP. :-) David Xu From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 20:07:46 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9D7D337B401; Thu, 24 Apr 2003 20:07:46 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id DF20843FE0; Thu, 24 Apr 2003 20:07:45 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3P37jBg017127; Thu, 24 Apr 2003 23:07:45 -0400 (EDT) Received: from localhost (eischen@localhost)h3P37i8t017123; Thu, 24 Apr 2003 23:07:44 -0400 (EDT) Date: Thu, 24 Apr 2003 23:07:44 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <200304250242.h3P2g2F6085399@freefall.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 03:07:46 -0000 On Thu, 24 Apr 2003, David Xu wrote: > I have put a patch to enable userland support SMP scheduling. > http://people.freebsd.org/~davidxu/libpthread_smp.diff > The patch works on my SMP machine, but not fully tested, > and I will work on idle kses stuffs. At least, it seems > the SMP speed is not slower than UP. :-) Cool, I just found the bug with scope process threads that was causing Token_Strategy_Test to fail. Now the 2 ACE tests that were SIGBUS'ing (with scope system threads enabled) are now running. Only Token_Strategy_Test fails, but it is just an expected result where some mean value is less than the stddev. http://people.freebsd.org/~deischen/kse/libpthread.diffs I was willing to work on the idle KSEs; I was just asking your opinion :-) Feel free to tackle it though. This is fun stuff :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 20:16:50 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A287C37B401 for ; Thu, 24 Apr 2003 20:16:50 -0700 (PDT) Received: from smtp01.syd.iprimus.net.au (smtp01.syd.iprimus.net.au [210.50.30.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 2A6A743FE1 for ; Thu, 24 Apr 2003 20:16:50 -0700 (PDT) (envelope-from tim@robbins.dropbear.id.au) Received: from dilbert.robbins.dropbear.id.au (210.50.218.164) by smtp01.syd.iprimus.net.au (7.0.012) id 3E96D1D200222E64 for freebsd-threads@freebsd.org; Fri, 25 Apr 2003 13:16:48 +1000 Received: by dilbert.robbins.dropbear.id.au (Postfix, from userid 1000) id 8EE7FC90D; Fri, 25 Apr 2003 13:16:56 +1000 (EST) Date: Fri, 25 Apr 2003 13:16:56 +1000 From: Tim Robbins To: freebsd-threads@freebsd.org Message-ID: <20030425131656.A93380@dilbert.robbins.dropbear.id.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i Subject: Threading and credential changes X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 03:16:50 -0000 I'm trying to understand the way a thread's td_ucred is updated. In ast(), cred_update_thread() is called to update it when it doesn't match the one stored in the proc. However, setuid() etc. don't set the TDF_ASTPENDING flag on any threads, not even the current thread. Am I misunderstanding what ast() does, does the thread's cred get updated in some other way, or is this a bug? Tim From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 21:09:59 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F0A4837B404; Thu, 24 Apr 2003 21:09:58 -0700 (PDT) Received: from HAL9000.homeunix.com (12-233-57-131.client.attbi.com [12.233.57.131]) by mx1.FreeBSD.org (Postfix) with ESMTP id EC32543FD7; Thu, 24 Apr 2003 21:09:57 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: from HAL9000.homeunix.com (localhost [127.0.0.1]) by HAL9000.homeunix.com (8.12.9/8.12.5) with ESMTP id h3P49tjC020539; Thu, 24 Apr 2003 21:09:55 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Received: (from das@localhost) by HAL9000.homeunix.com (8.12.9/8.12.5/Submit) id h3P49tXc020538; Thu, 24 Apr 2003 21:09:55 -0700 (PDT) (envelope-from das@FreeBSD.ORG) Date: Thu, 24 Apr 2003 21:09:55 -0700 From: David Schultz To: Tim Robbins Message-ID: <20030425040955.GA20513@HAL9000.homeunix.com> Mail-Followup-To: Tim Robbins , freebsd-threads@freebsd.org References: <20030425131656.A93380@dilbert.robbins.dropbear.id.au> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030425131656.A93380@dilbert.robbins.dropbear.id.au> cc: freebsd-threads@FreeBSD.ORG Subject: Re: Threading and credential changes X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 04:09:59 -0000 On Fri, Apr 25, 2003, Tim Robbins wrote: > I'm trying to understand the way a thread's td_ucred is updated. In ast(), > cred_update_thread() is called to update it when it doesn't match the one > stored in the proc. However, setuid() etc. don't set the TDF_ASTPENDING flag > on any threads, not even the current thread. Am I misunderstanding what ast() > does, does the thread's cred get updated in some other way, or is this a bug? This may not answer your question entirely, but rwatson gave a very nice explanation of how and why ucreds are cached in a post to current@ a while ago: http://www.freebsd.org/cgi/getmsg.cgi?fetch=637312+0+/usr/local/www/db/text/2003/freebsd-current/20030209.freebsd-current From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 23:08:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6A41B37B401 for ; Thu, 24 Apr 2003 23:08:00 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5673A43F75 for ; Thu, 24 Apr 2003 23:07:58 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from DAVIDW2K ([192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id JM648883; Fri, 25 Apr 2003 11:10:55 +0800 Message-ID: <027e01c30ada$87d57f10$f001a8c0@davidw2k> From: "David Xu" To: "Daniel Eischen" References: Date: Fri, 25 Apr 2003 11:25:56 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 06:08:00 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Friday, April 25, 2003 11:07 AM Subject: Re: SMPing libpthread > On Thu, 24 Apr 2003, David Xu wrote: >=20 > > I have put a patch to enable userland support SMP scheduling.=20 > > http://people.freebsd.org/~davidxu/libpthread_smp.diff > > The patch works on my SMP machine, but not fully tested, > > and I will work on idle kses stuffs. At least, it seems > > the SMP speed is not slower than UP. :-) >=20 > Cool, I just found the bug with scope process threads that was > causing Token_Strategy_Test to fail. Now the 2 ACE tests that > were SIGBUS'ing (with scope system threads enabled) are now > running. Only Token_Strategy_Test fails, but it is just an > expected result where some mean value is less than the stddev. >=20 > http://people.freebsd.org/~deischen/kse/libpthread.diffs >=20 > I was willing to work on the idle KSEs; I was just asking > your opinion :-) Feel free to tackle it though. >=20 I have a simple idea, in kse_wait(), a kse inserts itself into an idle kse queue under scheduler lock held(it is already),=20 then release scheduler lock and call kse_release(). any other kses enters in _thr_sched_multi and got a completed context=20 list should look the idle kses queue and wakeup those idle kses accordingly, also _thr_set_runnable_unlocked etc should look the idle queue as well, because all things are done under scheduler lock, it seems very nice for me. :-) > This is fun stuff :-) >=20 True! > --=20 > Dan Eischen >=20 > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to = "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 23:08:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C5F4337B407; Thu, 24 Apr 2003 23:08:03 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id DFA8F43F75; Thu, 24 Apr 2003 23:08:00 -0700 (PDT) (envelope-from davidxu@viatech.com.cn) Received: from DAVIDW2K ([192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id JM648884; Fri, 25 Apr 2003 11:18:11 +0800 Message-ID: <028801c30adb$8b9095d0$f001a8c0@davidw2k> From: "David Xu" To: "Tim Robbins" , References: <20030425131656.A93380@dilbert.robbins.dropbear.id.au> Date: Fri, 25 Apr 2003 11:33:41 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4807.1700 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4910.0300 Subject: Re: Threading and credential changes X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 06:08:04 -0000 cred is updated when process enters kernel, if the current thread cred is not same as process's cred, it will be updated to use process's cred. when process is in user mode, it is not important if the cred is right one. Am I right ? David Xu ----- Original Message -----=20 From: "Tim Robbins" To: Sent: Friday, April 25, 2003 11:16 AM Subject: Threading and credential changes > I'm trying to understand the way a thread's td_ucred is updated. In = ast(), > cred_update_thread() is called to update it when it doesn't match the = one > stored in the proc. However, setuid() etc. don't set the = TDF_ASTPENDING flag > on any threads, not even the current thread. Am I misunderstanding = what ast() > does, does the thread's cred get updated in some other way, or is this = a bug? >=20 >=20 > Tim > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to = "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Thu Apr 24 23:26:50 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 2A6D337B405; Thu, 24 Apr 2003 23:26:50 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 667CE43F3F; Thu, 24 Apr 2003 23:26:49 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3P6QmBg013293; Fri, 25 Apr 2003 02:26:48 -0400 (EDT) Received: from localhost (eischen@localhost)h3P6QmDu013290; Fri, 25 Apr 2003 02:26:48 -0400 (EDT) Date: Fri, 25 Apr 2003 02:26:48 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <02c001c30add$7377ff40$f001a8c0@davidw2k> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 06:26:50 -0000 On Fri, 25 Apr 2003, David Xu wrote: > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Friday, April 25, 2003 11:07 AM > Subject: Re: SMPing libpthread > > > > On Thu, 24 Apr 2003, David Xu wrote: > > > > > I have put a patch to enable userland support SMP scheduling. > > > http://people.freebsd.org/~davidxu/libpthread_smp.diff > > > The patch works on my SMP machine, but not fully tested, > > > and I will work on idle kses stuffs. At least, it seems > > > the SMP speed is not slower than UP. :-) > > > > Cool, I just found the bug with scope process threads that was > > causing Token_Strategy_Test to fail. Now the 2 ACE tests that > > were SIGBUS'ing (with scope system threads enabled) are now > > running. Only Token_Strategy_Test fails, but it is just an > > expected result where some mean value is less than the stddev. > > > > http://people.freebsd.org/~deischen/kse/libpthread.diffs > > > > I was willing to work on the idle KSEs; I was just asking > > your opinion :-) Feel free to tackle it though. > > > I have a simple idea, in kse_wait(), a kse inserts itself into > an idle kse queue under scheduler lock held(it is already), > then release scheduler lock and call kse_release(). any > other kses enters in _thr_sched_multi and got a completed context > list should look the idle kses queue and wakeup those idle kses > accordingly, also _thr_set_runnable_unlocked etc should look > the idle queue as well, because all things are done under scheduler > lock, it seems very nice for me. :-) Yeah, very similar to what I was going to do. I was just going to increment the KSEG's kg_idle_kses and add an idle flag to the KSE (and set it in kse_wait()). The KSE's are already in a list hung off the KSEG, so you could just walk the list until you found an idle KSE to wakeup. But you could add another "idle KSE" list just as well. I also thought about adding a _pq_count() (or _pq_depth()) function to the priority queues. So you can tell if there is more than one thread. There's no sense in waking idle KSEs if there's not enough runnable threads for them to run. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Apr 25 03:15:13 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9288537B401 for ; Fri, 25 Apr 2003 03:15:12 -0700 (PDT) Received: from smtp01.syd.iprimus.net.au (smtp01.syd.iprimus.net.au [210.50.30.52]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0AB9243FBF for ; Fri, 25 Apr 2003 03:15:12 -0700 (PDT) (envelope-from tim@robbins.dropbear.id.au) Received: from dilbert.robbins.dropbear.id.au (210.50.248.139) by smtp01.syd.iprimus.net.au (7.0.012) id 3E96D1D20022F5E3; Fri, 25 Apr 2003 20:15:10 +1000 Received: by dilbert.robbins.dropbear.id.au (Postfix, from userid 1000) id 59936C90D; Fri, 25 Apr 2003 20:15:20 +1000 (EST) Date: Fri, 25 Apr 2003 20:15:20 +1000 From: Tim Robbins To: David Xu Message-ID: <20030425201520.A4699@dilbert.robbins.dropbear.id.au> References: <20030425131656.A93380@dilbert.robbins.dropbear.id.au> <028801c30adb$8b9095d0$f001a8c0@davidw2k> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.2.5.1i In-Reply-To: <028801c30adb$8b9095d0$f001a8c0@davidw2k>; from davidxu@viatech.com.cn on Fri, Apr 25, 2003 at 11:33:41AM +0800 cc: freebsd-threads@FreeBSD.ORG Subject: Re: Threading and credential changes X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Apr 2003 10:15:13 -0000 On Fri, Apr 25, 2003 at 11:33:41AM +0800, David Xu wrote: > cred is updated when process enters kernel, > if the current thread cred is not same as process's cred, > it will be updated to use process's cred. > when process is in user mode, it is not important if the > cred is right one. Am I right ? Thanks. I didn't think to look at the MD code in sys/i386/i386. I knew I must have been overlooking something obvious. Tim From owner-freebsd-threads@FreeBSD.ORG Fri Apr 25 21:52:22 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A1E9C37B401; Fri, 25 Apr 2003 21:52:22 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id B881E43FBF; Fri, 25 Apr 2003 21:52:21 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3Q4qKBg026384; Sat, 26 Apr 2003 00:52:20 -0400 (EDT) Received: from localhost (eischen@localhost)h3Q4qKgC026380; Sat, 26 Apr 2003 00:52:20 -0400 (EDT) Date: Sat, 26 Apr 2003 00:52:20 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <200304250242.h3P2g2F6085399@freefall.freebsd.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 04:52:23 -0000 On Thu, 24 Apr 2003, David Xu wrote: > I have put a patch to enable userland support SMP scheduling. > http://people.freebsd.org/~davidxu/libpthread_smp.diff > The patch works on my SMP machine, but not fully tested, > and I will work on idle kses stuffs. At least, it seems > the SMP speed is not slower than UP. :-) David, I noticed that we hold the scheduling lock before and after calling the scheduler. Is this necessary? And if so, is it necessary to hold hold it after return from the scheduling switch? One you're in the scheduler, and choose another thread (releasing the lock after doing so), shouldn't you be able to switch to it without having the lock? -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 05:24:36 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D58DB37B401; Sat, 26 Apr 2003 05:24:36 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 246CC43FA3; Sat, 26 Apr 2003 05:24:35 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with SMTP id h3QCOXUp051035; Sat, 26 Apr 2003 05:24:34 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <004001c30bef$648ff0b0$0701a8c0@tiger> From: "David Xu" To: "Daniel Eischen" References: Date: Sat, 26 Apr 2003 20:28:49 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 12:24:37 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Saturday, April 26, 2003 12:52 PM Subject: Re: SMPing libpthread > On Thu, 24 Apr 2003, David Xu wrote: >=20 > > I have put a patch to enable userland support SMP scheduling.=20 > > http://people.freebsd.org/~davidxu/libpthread_smp.diff > > The patch works on my SMP machine, but not fully tested, > > and I will work on idle kses stuffs. At least, it seems > > the SMP speed is not slower than UP. :-) >=20 > David, I noticed that we hold the scheduling lock before and > after calling the scheduler. Is this necessary? And if so, > is it necessary to hold hold it after return from the > scheduling switch? One you're in the scheduler, and choose > another thread (releasing the lock after doing so), shouldn't > you be able to switch to it without having the lock? >=20 I am trying to do things in atomic way. because I found on SMP, thread state is always out of synchronized in current code. I want to make state change and context switch in an atomic operation, when thread is in state transit, no other threads can use _thr_setrunnable_unlocked() = etc to change its state, this avoids lots of "thread already in priority = queue" messages, and if I test the "in priority queue" flag every where, I = sometimes lose chance to wakeup a thread because of some unknown races, whole process just suddenly stops there. I am just copying the idea from current kernel code, I don't intend to = inventing new things. :-) One deadlock I found is in kse_sched_multi(), there is an optimization: if (curthread =3D=3D NULL) ; /* Nothing to do here. */ else if ((curthread->need_switchout =3D=3D 0) && (curthread->blocked =3D=3D 0) && (THR_IN_CRITICAL(curthread))) { /* * Resume the thread and tell it to yield when * it leaves the critical region. */ These code cause deadlock, at least there is a deadlock between = malloc/free and static mutex initializing code, I think unless we believe that thread = is locking a leaf=20 node, it is not safe to give cpu to the thread again. I suggest comment = out these=20 optimization code. I am still working on it, hope that I can post a patch soon. A bad news is kernel seems too unstable when testing KSE program, it = just silently locks up, not panic, no dump, just locks up, this is newest kernel = source code. :( > --=20 > Dan Eischen >=20 >=20 From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 07:15:31 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DCE9937B401 for ; Sat, 26 Apr 2003 07:15:30 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id CAB6143F93 for ; Sat, 26 Apr 2003 07:15:29 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3QEFSBg005322; Sat, 26 Apr 2003 10:15:28 -0400 (EDT) Received: from localhost (eischen@localhost)h3QEFRYr005319; Sat, 26 Apr 2003 10:15:28 -0400 (EDT) Date: Sat, 26 Apr 2003 10:15:27 -0400 (EDT) From: Daniel Eischen To: David Xu In-Reply-To: <004001c30bef$648ff0b0$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 14:15:31 -0000 On Sat, 26 Apr 2003, David Xu wrote: > ----- Original Message ----- > From: "Daniel Eischen" > To: "David Xu" > Cc: > Sent: Saturday, April 26, 2003 12:52 PM > Subject: Re: SMPing libpthread > > > > On Thu, 24 Apr 2003, David Xu wrote: > > > > > I have put a patch to enable userland support SMP scheduling. > > > http://people.freebsd.org/~davidxu/libpthread_smp.diff > > > The patch works on my SMP machine, but not fully tested, > > > and I will work on idle kses stuffs. At least, it seems > > > the SMP speed is not slower than UP. :-) > > > > David, I noticed that we hold the scheduling lock before and > > after calling the scheduler. Is this necessary? And if so, > > is it necessary to hold hold it after return from the > > scheduling switch? One you're in the scheduler, and choose > > another thread (releasing the lock after doing so), shouldn't > > you be able to switch to it without having the lock? > > > > I am trying to do things in atomic way. because I found on SMP, thread > state is always out of synchronized in current code. I want to make > state change and context switch in an atomic operation, when thread is > in state transit, no other threads can use _thr_setrunnable_unlocked() etc > to change its state, this avoids lots of "thread already in priority queue" > messages, and if I test the "in priority queue" flag every where, I sometimes > lose chance to wakeup a thread because of some unknown races, whole > process just suddenly stops there. I think the problem is that the signal code doesn't properly check the thread's state and whether it's currently active or not. Sometimes, the signal code thinks that if the state is not PS_RUNNING that the thread is inactive. It should really be checking td->active and the UTS should make sure it sets and clears this when switching the thread in and out. The other problem is that the mutex and cond variable code doesn't take the scheduling lock when changing the thread's state (it relies on holding the mutex or cv low-level lock to protect the thread state). This is one of the things I was going to fix. It is OK to take the scheduling lock while holding the low-level mutex lock; we don't ever take them in reverse (that I can see). > I am just copying the idea from current kernel code, I don't intend to inventing > new things. :-) > > One deadlock I found is in kse_sched_multi(), there is an optimization: > if (curthread == NULL) > ; /* Nothing to do here. */ > else if ((curthread->need_switchout == 0) && > (curthread->blocked == 0) && (THR_IN_CRITICAL(curthread))) { > /* > * Resume the thread and tell it to yield when > * it leaves the critical region. > */ > > These code cause deadlock, at least there is a deadlock between malloc/free and > static mutex initializing code, I think unless we believe that thread is locking a leaf > node, it is not safe to give cpu to the thread again. I suggest comment out these > optimization code. I think that's because spinlock is really still a spinlock (we really have to get rid of these). I think this can be solved by putting the thread in a critical region while it holds the spinlock. Currently, a higher priority thread trying to get a spinlock that is held by a lower priority thread will cause a deadlock (if both threads are running on the same KSE). We could treat spinlocks as static mutexes or as low-level locks. There are 4 fields to the spinlock: typedef struct { volatile long access_lock; volatile long lock_owner; volatile char *fname; volatile int lineno; } spinlock_t; #define _SPINLOCK_INITIALIZER { 0, 0, 0, 0 } Since they are all initialized to 0, we could detect an uninitialized spinlock. Is there any platform where sizeof(long) != sizeof(void *)? Or I suppose we could use fname to point to a low-level lock or mutex. > I am still working on it, hope that I can post a patch soon. > A bad news is kernel seems too unstable when testing KSE program, it just silently > locks up, not panic, no dump, just locks up, this is newest kernel source code. :( Hmm, I haven't got that yet. I have had a lot of lock order reversals lately though. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 07:52:53 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C390437B401 for ; Sat, 26 Apr 2003 07:52:53 -0700 (PDT) Received: from mail.highway.ne.jp (mail.highway.ne.jp [210.166.100.33]) by mx1.FreeBSD.org (Postfix) with ESMTP id 613A943FA3 for ; Sat, 26 Apr 2003 07:52:52 -0700 (PDT) (envelope-from kaakun@highway.ne.jp) Received: from face.violasystems.net (ppp0635.vc-tyo.hdd.co.jp [219.100.26.35]) by mail.highway.ne.jp (8.9.3p2/3.7W03042516) with SMTP id XAA29861 for ; Sat, 26 Apr 2003 23:52:50 +0900 (JST) Date: Sat, 26 Apr 2003 23:52:40 +0900 From: Kazuaki Oda To: freebsd-threads@freebsd.org Message-Id: <20030426235240.2f950c8e.kaakun@highway.ne.jp> X-Mailer: Sylpheed version 0.8.11 (GTK+ 1.2.10; i386-portbld-freebsd5.0) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Subject: short libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 14:52:54 -0000 Hi, all. I'm not a hacker, but following patch resolves a signal delivering problem of my short test program. Is it correct? diff -cr libpthread.orig/thread/thr_sig.c libpthread/thread/thr_sig.c *** libpthread.orig/thread/thr_sig.c Fri Apr 18 14:04:16 2003 --- libpthread/thread/thr_sig.c Sat Apr 26 23:37:20 2003 *************** *** 291,297 **** * Grab the next thread before possibly destroying * the link entry. */ ! pthread_next = TAILQ_NEXT(pthread, pqe); if ((pthread->state == PS_SIGWAIT) && sigismember(pthread->data.sigwait, sig)) { --- 291,297 ---- * Grab the next thread before possibly destroying * the link entry. */ ! pthread_next = TAILQ_NEXT(pthread, tle); if ((pthread->state == PS_SIGWAIT) && sigismember(pthread->data.sigwait, sig)) { -- Kazuaki Oda From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 11:09:29 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D25B337B401; Sat, 26 Apr 2003 11:09:29 -0700 (PDT) Received: from bluejay.mail.pas.earthlink.net (bluejay.mail.pas.earthlink.net [207.217.120.218]) by mx1.FreeBSD.org (Postfix) with ESMTP id 11FF543FBF; Sat, 26 Apr 2003 11:09:29 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0174.cvx40-bradley.dialup.earthlink.net ([216.244.42.174] helo=mindspring.com) by bluejay.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 199U74-000245-00; Sat, 26 Apr 2003 11:09:27 -0700 Message-ID: <3EAACB08.BFD0B8F5@mindspring.com> Date: Sat, 26 Apr 2003 11:08:08 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Daniel Eischen References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a4725a00e406b7611270dcb04c44f550ae350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 18:09:30 -0000 Daniel Eischen wrote: > On Thu, 24 Apr 2003, David Xu wrote: > > I have put a patch to enable userland support SMP scheduling. > > http://people.freebsd.org/~davidxu/libpthread_smp.diff > > The patch works on my SMP machine, but not fully tested, > > and I will work on idle kses stuffs. At least, it seems > > the SMP speed is not slower than UP. :-) > > David, I noticed that we hold the scheduling lock before and > after calling the scheduler. Is this necessary? And if so, > is it necessary to hold hold it after return from the > scheduling switch? One you're in the scheduler, and choose > another thread (releasing the lock after doing so), shouldn't > you be able to switch to it without having the lock? IMO, this depends entirely on the scheduler in question; this is something Jeff and Julian were working on divorcing from the threads API entirely, and putting into a scheduler API that was supposed to be seperate and deal with policy issues like scheduling locks. It's very easy to envision an SMP scheduler that has zero locks, for example, so holding a lock in threads code before calling into the scheduler API would not only be counter-productive, it would be referencing a non-existant lock. Jeff and Julian probably have more to say on their implementation ideas. -- Terry From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 11:35:14 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1711737B401 for ; Sat, 26 Apr 2003 11:35:14 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 81CEF43FB1 for ; Sat, 26 Apr 2003 11:35:13 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0291.cvx40-bradley.dialup.earthlink.net ([216.244.43.36] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 199UVv-0000Nr-00; Sat, 26 Apr 2003 11:35:08 -0700 Message-ID: <3EAAD0EE.49EDA0D@mindspring.com> Date: Sat, 26 Apr 2003 11:33:18 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: David Xu References: <004001c30bef$648ff0b0$0701a8c0@tiger> Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a409aa9fc588579c26b94469d10b09e1cf350badd9bab72f9c350badd9bab72f9c350badd9bab72f9c cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 18:35:14 -0000 David Xu wrote: > From: "Daniel Eischen" > > David, I noticed that we hold the scheduling lock before and > > after calling the scheduler. Is this necessary? And if so, > > is it necessary to hold hold it after return from the > > scheduling switch? One you're in the scheduler, and choose > > another thread (releasing the lock after doing so), shouldn't > > you be able to switch to it without having the lock? > > I am trying to do things in atomic way. because I found on SMP, thread > state is always out of synchronized in current code. I want to make > state change and context switch in an atomic operation, when thread is > in state transit, no other threads can use _thr_setrunnable_unlocked() etc > to change its state, this avoids lots of "thread already in priority queue" > messages, and if I test the "in priority queue" flag every where, I sometimes > lose chance to wakeup a thread because of some unknown races, whole > process just suddenly stops there. I am just copying the idea from > current kernel code, I don't intend to inventing new things. :-) Note that I think you want idempotence, not atomicity, in this case, anyway. Jeff and Julian should comment; but my gut feeling is that you should be passing a function to the scheduler for it to call on your behalf, while it holds any locks it feels are necessary. This is probably an interface layering violation for the scheduler API. I can't help feeling, though, that this should be seperated out. I don't know how to deal with the idempotence issues "correctly", and still maintain a seperation between threads implementation and scheduler implementation off the top of my head (at least, without changing the scheduler API), but I'm sure it's doable, with a little thought. -- Terry From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 11:38:15 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4097137B401 for ; Sat, 26 Apr 2003 11:38:15 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B54B43FBF for ; Sat, 26 Apr 2003 11:38:14 -0700 (PDT) (envelope-from eischen@pcnet1.pcnet.com) Received: from pcnet1.pcnet.com (localhost [127.0.0.1]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h3QIcDBg009755; Sat, 26 Apr 2003 14:38:13 -0400 (EDT) Received: from localhost (eischen@localhost)h3QIcDkA009751; Sat, 26 Apr 2003 14:38:13 -0400 (EDT) Date: Sat, 26 Apr 2003 14:38:13 -0400 (EDT) From: Daniel Eischen To: Kazuaki Oda In-Reply-To: <20030426235240.2f950c8e.kaakun@highway.ne.jp> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org Subject: Re: short libpthread patch X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 18:38:15 -0000 On Sat, 26 Apr 2003, Kazuaki Oda wrote: > Hi, all. > > I'm not a hacker, but following patch resolves a signal delivering > problem of my short test program. > > Is it correct? Yes. Thanks! Keep hacking :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 14:12:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5514137B401 for ; Sat, 26 Apr 2003 14:12:35 -0700 (PDT) Received: from rwcrmhc52.attbi.com (rwcrmhc52.attbi.com [216.148.227.88]) by mx1.FreeBSD.org (Postfix) with ESMTP id B8CC843FB1 for ; Sat, 26 Apr 2003 14:12:34 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org (12-232-168-4.client.attbi.com[12.232.168.4]) by rwcrmhc52.attbi.com (rwcrmhc52) with ESMTP id <2003042621123405200nr5v8e>; Sat, 26 Apr 2003 21:12:34 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA28514; Sat, 26 Apr 2003 14:12:33 -0700 (PDT) Date: Sat, 26 Apr 2003 14:12:33 -0700 (PDT) From: Julian Elischer To: Terry Lambert In-Reply-To: <3EAAD0EE.49EDA0D@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: David Xu cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 21:12:35 -0000 terry thet are talking userland.. On Sat, 26 Apr 2003, Terry Lambert wrote: > David Xu wrote: > > From: "Daniel Eischen" > > > David, I noticed that we hold the scheduling lock before and > [...] > Note that I think you want idempotence, not atomicity, in this > case, anyway. > [...] > I can't help feeling, though, that this should be seperated > out. I don't know how to deal with the idempotence issues > "correctly", and still maintain a seperation between threads > implementation and scheduler implementation off the top of > my head (at least, without changing the scheduler API), but > I'm sure it's doable, with a little thought. > > -- Terry > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 15:23:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9667137B401 for ; Sat, 26 Apr 2003 15:23:03 -0700 (PDT) Received: from heron.mail.pas.earthlink.net (heron.mail.pas.earthlink.net [207.217.120.189]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1AB8443FA3 for ; Sat, 26 Apr 2003 15:23:03 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from pool0335.cvx21-bradley.dialup.earthlink.net ([209.179.193.80] helo=mindspring.com) by heron.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 199Y4C-0003jL-00; Sat, 26 Apr 2003 15:22:44 -0700 Message-ID: <3EAB0666.F3F9FD62@mindspring.com> Date: Sat, 26 Apr 2003 15:21:26 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: Julian Elischer References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a49824c257e7815ebcbb59d1088c583e97a2d4e88014a4647c350badd9bab72f9c350badd9bab72f9c cc: David Xu cc: Daniel Eischen cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Apr 2003 22:23:03 -0000 Julian Elischer wrote: > terry thet are talking userland.. Sorry; that wasn't clear from context; I had just watched a kernel commit roll by... never mind. 8-). -- Terry From owner-freebsd-threads@FreeBSD.ORG Sat Apr 26 18:46:52 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 843CF37B404; Sat, 26 Apr 2003 18:46:52 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id CA94943FBF; Sat, 26 Apr 2003 18:46:51 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with SMTP id h3R1kmUp077174; Sat, 26 Apr 2003 18:46:48 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <006501c30c5f$77fe5950$0701a8c0@tiger> From: "David Xu" To: "Daniel Eischen" , "David Xu" References: Date: Sun, 27 Apr 2003 09:51:04 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 5.50.4522.1200 X-MimeOLE: Produced By Microsoft MimeOLE V5.50.4522.1200 cc: freebsd-threads@freebsd.org Subject: Re: SMPing libpthread X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: David Xu List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 27 Apr 2003 01:46:52 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "David Xu" Cc: Sent: Saturday, April 26, 2003 10:15 PM Subject: Re: SMPing libpthread > On Sat, 26 Apr 2003, David Xu wrote: > > ----- Original Message -----=20 > > From: "Daniel Eischen" > > To: "David Xu" > > Cc: > > Sent: Saturday, April 26, 2003 12:52 PM > > Subject: Re: SMPing libpthread > >=20 > >=20 > > > On Thu, 24 Apr 2003, David Xu wrote: > > >=20 > > > > I have put a patch to enable userland support SMP scheduling.=20 > > > > http://people.freebsd.org/~davidxu/libpthread_smp.diff > > > > The patch works on my SMP machine, but not fully tested, > > > > and I will work on idle kses stuffs. At least, it seems > > > > the SMP speed is not slower than UP. :-) > > >=20 > > > David, I noticed that we hold the scheduling lock before and > > > after calling the scheduler. Is this necessary? And if so, > > > is it necessary to hold hold it after return from the > > > scheduling switch? One you're in the scheduler, and choose > > > another thread (releasing the lock after doing so), shouldn't > > > you be able to switch to it without having the lock? > > >=20 > >=20 > > I am trying to do things in atomic way. because I found on SMP, = thread > > state is always out of synchronized in current code. I want to make > > state change and context switch in an atomic operation, when thread = is > > in state transit, no other threads can use = _thr_setrunnable_unlocked() etc > > to change its state, this avoids lots of "thread already in = priority queue" > > messages, and if I test the "in priority queue" flag every where, = I sometimes > > lose chance to wakeup a thread because of some unknown races, whole > > process just suddenly stops there. >=20 > I think the problem is that the signal code doesn't properly > check the thread's state and whether it's currently active > or not. Sometimes, the signal code thinks that if the state > is not PS_RUNNING that the thread is inactive. It should > really be checking td->active and the UTS should make sure > it sets and clears this when switching the thread in and > out. I havn't tested signal code. >=20 > The other problem is that the mutex and cond variable code > doesn't take the scheduling lock when changing the thread's > state (it relies on holding the mutex or cv low-level lock > to protect the thread state). This is one of the things > I was going to fix. It is OK to take the scheduling lock > while holding the low-level mutex lock; we don't ever take > them in reverse (that I can see). >=20 One thing causes wrong thread state and SIGSEGV in mutex code under SMP is after a thread A put itself on a sync queue and it is then swapped = out by kernel,=20 a thread B on another CPU unlocks a mutex and tries to = _thr_setrunnable_unlocked thread A, and put it on RUNQ, next time thread A is selected to run, = because it is still in kernel not in userland, it causes SIGSEGV, other thread = state code have same problem. My code prevents this from happen, there is no = window opened for this issue. Thread A holds a low level lock and then holds a = schedule lock, unlocks low lever mutex lock, then does context switch, because when schedule = lock is held,=20 it is in critical region, kernel won't schedule an upcall, and thread B = want to=20 _thr_setrunnable_unlocked A should hold schedule lock, but it is = impossible until A is=20 switchout, so there is no race. In my patch, only a PS_RUNNING thread = will be blocked in kernel, however because thread state is still PS_RUNNING, =20 _thr_setrunnable_unlocked needn't to do any thing about it, I had code = already in that patch. > > I am just copying the idea from current kernel code, I don't intend = to inventing > > new things. :-) > >=20 > > One deadlock I found is in kse_sched_multi(), there is an = optimization: > > if (curthread =3D=3D NULL) > > ; /* Nothing to do here. */ > > else if ((curthread->need_switchout =3D=3D 0) && > > (curthread->blocked =3D=3D 0) && (THR_IN_CRITICAL(curthread))) { > > /* > > * Resume the thread and tell it to yield when > > * it leaves the critical region. > > */ > >=20 > > These code cause deadlock, at least there is a deadlock between = malloc/free and > > static mutex initializing code, I think unless we believe that = thread is locking a leaf=20 > > node, it is not safe to give cpu to the thread again. I suggest = comment out these=20 > > optimization code. >=20 > I think that's because spinlock is really still a spinlock (we really = have > to get rid of these). I think this can be solved by putting the = thread > in a critical region while it holds the spinlock. Currently, a higher > priority thread trying to get a spinlock that is held by a lower > priority thread will cause a deadlock (if both threads are running > on the same KSE). >=20 > We could treat spinlocks as static mutexes or as low-level locks. > There are 4 fields to the spinlock: >=20 > typedef struct { > volatile long access_lock; > volatile long lock_owner; > volatile char *fname; > volatile int lineno; > } spinlock_t; >=20 > #define _SPINLOCK_INITIALIZER { 0, 0, 0, 0 } >=20 > Since they are all initialized to 0, we could detect an uninitialized > spinlock. Is there any platform where sizeof(long) !=3D sizeof(void = *)? > Or I suppose we could use fname to point to a low-level lock or > mutex. >=20 Static mutex still needs malloc to initiailize its internal low level = lock,=20 it would recursively call malloc. I think only criticial count may work. > > I am still working on it, hope that I can post a patch soon. > > A bad news is kernel seems too unstable when testing KSE program, = it just silently > > locks up, not panic, no dump, just locks up, this is newest = kernel source code. :( >=20 > Hmm, I haven't got that yet. I have had a lot of lock order reversals > lately though. >=20 > --=20 > Dan Eischen >=20 > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to = "freebsd-threads-unsubscribe@freebsd.org" >=20