From owner-freebsd-threads@FreeBSD.ORG Mon Aug 4 11:01:33 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B786737B401 for ; Mon, 4 Aug 2003 11:01:33 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0927943FF2 for ; Mon, 4 Aug 2003 11:01:25 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h74I1PUp066160 for ; Mon, 4 Aug 2003 11:01:25 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h74I1P4A066152 for freebsd-threads@freebsd.org; Mon, 4 Aug 2003 11:01:25 -0700 (PDT) Date: Mon, 4 Aug 2003 11:01:25 -0700 (PDT) Message-Id: <200308041801.h74I1P4A066152@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2003 18:01:34 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2002/01/16] kern/33951 threads pthread_cancel is ignored 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/01/19] bin/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] bin/24632 threads libc_r delicate deviation from libc in ha o [2001/04/02] bin/26307 threads libc_r aborts when using the KDE media pl o [2001/10/31] bin/31661 threads pthread_kill signal handler doesn't get s o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/03/07] bin/35622 threads sigaltstack is missing in libc_r o [2002/06/27] bin/39922 threads [PATCH?] Threaded applications executed w o [2003/03/02] bin/48856 threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] bin/49087 threads Signals lost in programs linked with libc o [2003/05/07] bin/51949 threads thread in accept cannot be cancelled o [2002/02/01] i386/34536 threads accept() blocks other threads o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/10/10] kern/43887 threads abnormal CPU useage when use pthread_mute o [2003/05/30] kern/52817 threads top(1) shows garbage for threaded process o [2000/08/26] misc/20861 threads libc_r does not honor socket timeouts o [2001/01/25] misc/24641 threads pthread_rwlock_rdlock can deadlock o [2002/08/04] misc/41331 threads Pthread library open sets O_NONBLOCK flag 18 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/09/09] bin/30464 threads pthread mutex attributes -- pshared o [2002/05/02] bin/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwri o [2000/05/25] misc/18824 threads gethostbyname is not thread safe o [2000/10/21] misc/22190 threads A threaded read(2) from a socketpair(2) f o [2002/07/16] misc/40671 threads pthread_cancel doesn't remove thread from 5 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Aug 4 11:03:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CC62E37B401 for ; Mon, 4 Aug 2003 11:03:19 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5B88C43F3F for ; Mon, 4 Aug 2003 11:03:19 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h74I3JUp068333 for ; Mon, 4 Aug 2003 11:03:19 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h74I3I9B068327 for threads@freebsd.org; Mon, 4 Aug 2003 11:03:18 -0700 (PDT) Date: Mon, 4 Aug 2003 11:03:18 -0700 (PDT) Message-Id: <200308041803.h74I3I9B068327@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2003 18:03:20 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin 1 problem total. Non-critical problems From owner-freebsd-threads@FreeBSD.ORG Mon Aug 4 11:24:50 2003 Return-Path: Delivered-To: freebsd-threads@hub.freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CCB837B401; Mon, 4 Aug 2003 11:24:50 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9E7543FD7; Mon, 4 Aug 2003 11:24:49 -0700 (PDT) (envelope-from arved@FreeBSD.org) Received: from freefall.freebsd.org (arved@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h74IOnUp073034; Mon, 4 Aug 2003 11:24:49 -0700 (PDT) (envelope-from arved@freefall.freebsd.org) Received: (from arved@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h74IOnnJ073030; Mon, 4 Aug 2003 11:24:49 -0700 (PDT) Date: Mon, 4 Aug 2003 11:24:49 -0700 (PDT) From: Tilman Linneweh Message-Id: <200308041824.h74IOnnJ073030@freefall.freebsd.org> To: arved@FreeBSD.org, threads@FreeBSD.org, freebsd-threads@FreeBSD.org Subject: Re: bin/50733: buildworld won't build, because of linking problem X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2003 18:24:50 -0000 Synopsis: buildworld won't build, because of linking problem Responsible-Changed-From-To: threads->freebsd-threads Responsible-Changed-By: arved Responsible-Changed-When: Mon Aug 4 11:24:29 PDT 2003 Responsible-Changed-Why: fix responsible http://www.freebsd.org/cgi/query-pr.cgi?pr=50733 From owner-freebsd-threads@FreeBSD.ORG Mon Aug 4 11:24:50 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1CCB837B401; Mon, 4 Aug 2003 11:24:50 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id A9E7543FD7; Mon, 4 Aug 2003 11:24:49 -0700 (PDT) (envelope-from arved@FreeBSD.org) Received: from freefall.freebsd.org (arved@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h74IOnUp073034; Mon, 4 Aug 2003 11:24:49 -0700 (PDT) (envelope-from arved@freefall.freebsd.org) Received: (from arved@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h74IOnnJ073030; Mon, 4 Aug 2003 11:24:49 -0700 (PDT) Date: Mon, 4 Aug 2003 11:24:49 -0700 (PDT) From: Tilman Linneweh Message-Id: <200308041824.h74IOnnJ073030@freefall.freebsd.org> To: arved@FreeBSD.org, threads@FreeBSD.org, freebsd-threads@FreeBSD.org Subject: Re: bin/50733: buildworld won't build, because of linking problem X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 04 Aug 2003 18:24:50 -0000 Synopsis: buildworld won't build, because of linking problem Responsible-Changed-From-To: threads->freebsd-threads Responsible-Changed-By: arved Responsible-Changed-When: Mon Aug 4 11:24:29 PDT 2003 Responsible-Changed-Why: fix responsible http://www.freebsd.org/cgi/query-pr.cgi?pr=50733 From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 09:58:19 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C42D237B401 for ; Tue, 5 Aug 2003 09:58:19 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.bayarea.net [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E47843FAF for ; Tue, 5 Aug 2003 09:58:19 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75GwIwO031965 for ; Tue, 5 Aug 2003 09:58:18 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75GwICn000833 for ; Tue, 5 Aug 2003 09:58:18 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75GwI6A000832 for threads@FreeBSD.org; Tue, 5 Aug 2003 09:58:18 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 09:58:17 -0700 From: Marcel Moolenaar To: threads@FreeBSD.org Message-ID: <20030805165817.GA796@dhcp01.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 16:58:20 -0000 Ok, I have a couple of changes and fixes in my tree that makes KSE on ia64 work. I have a couple of questions before I can commit it: The current implementation defines the thread pointer to be per-KSE, like on i386. This is not how ia64 is supposed to work, so I have some hacks to cpu_set_upcall_kse() to preserve TP and a hack in _ia64_restore_context() to not restore TP (I still save TP in the context for now). Am I right that this is currently the only way libkse can work and that I have to wait until the patches that are floating around get committed? When libkse (then libpthread) does support a per-thread pointer. Does it expect the thread-pointer to set/switched by getcontext() setcontext() or will it assume that the thread-pointer is not itself part of the context? Thanks, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 10:22:15 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E606E37B401 for ; Tue, 5 Aug 2003 10:22:15 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 733AC43FA3 for ; Tue, 5 Aug 2003 10:22:15 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <20030805172215015003j466e>; Tue, 5 Aug 2003 17:22:15 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id KAA92003; Tue, 5 Aug 2003 10:22:14 -0700 (PDT) Date: Tue, 5 Aug 2003 10:22:13 -0700 (PDT) From: Julian Elischer To: Marcel Moolenaar In-Reply-To: <20030805165817.GA796@dhcp01.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@FreeBSD.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 17:22:16 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > Ok, > > I have a couple of changes and fixes in my tree that makes KSE on > ia64 work. I have a couple of questions before I can commit it: On ia64 the TP will point to the TDB, as the spec says. there is only one case where this negatively impacts libKSE and that is where the thread is setting itself into a critical region. in i386 it is movl #0, %gs(offset) (clear a value in the KCB) which can be done atomically since on ia64 TP points to teh THREAD structure, we have a second method of setting a critical region which is inthe TCB instead of being in the KCB, because using pdp11 syntax, offset(TCB)->reg NULL -> offset2(reg) cannot be done atomically and pre-emption may result in teh thread switching KSEs between teh two instructions. So on ia64 the critical region is defined as being enterred when the alternative flag in the TCB is used.. David and Dan have more info and patches > > The current implementation defines the thread pointer to be per-KSE, > like on i386. This is not how ia64 is supposed to work, so I have > some hacks to cpu_set_upcall_kse() to preserve TP and a hack in > _ia64_restore_context() to not restore TP (I still save TP in the > context for now). > > Am I right that this is currently the only way libkse can work and > that I have to wait until the patches that are floating around get > committed? I belive the patches are required, but are "near". > > When libkse (then libpthread) does support a per-thread pointer. > Does it expect the thread-pointer to set/switched by getcontext() > setcontext() or will it assume that the thread-pointer is not > itself part of the context? > > Thanks, > > -- > Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 10:33:16 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A954737B401 for ; Tue, 5 Aug 2003 10:33:16 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id F0E5F43F85 for ; Tue, 5 Aug 2003 10:33:15 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75HXEnj004145; Tue, 5 Aug 2003 13:33:14 -0400 (EDT) Date: Tue, 5 Aug 2003 13:33:14 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030805165817.GA796@dhcp01.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 17:33:17 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > Ok, > > I have a couple of changes and fixes in my tree that makes KSE on > ia64 work. I have a couple of questions before I can commit it: > > The current implementation defines the thread pointer to be per-KSE, > like on i386. This is not how ia64 is supposed to work, so I have > some hacks to cpu_set_upcall_kse() to preserve TP and a hack in > _ia64_restore_context() to not restore TP (I still save TP in the > context for now). Right, you shouldn't need to do that. The patches I have at: http://people.freebsd.org/~deischen/kse/libpthread.diffs should work for ia64 and other archs that have per-thread private doohickies. > Am I right that this is currently the only way libkse can work and > that I have to wait until the patches that are floating around get > committed? I'm almost ready to commit them. Trying to figure out why sysarch(AMD64_SET_FSBASE, foo) doesn't work on sledge (amd64). > When libkse (then libpthread) does support a per-thread pointer. > Does it expect the thread-pointer to set/switched by getcontext() > setcontext() or will it assume that the thread-pointer is not > itself part of the context? See the ia64 part of the above patch. Since getcontext() and setcontext() are not meant to be used by applications to switch between contexts in different threads, they shouldn't need to save and restore TP. But, the MD parts of libpthread, _thread_enter_uts() and _thread_switch(), do need to handle this. I _think_ I took care of this for ia64, but you might want to take a close look at the patch to make sure. The nice thing about per-thread private storage is that it makes the MD part of libpthread really simple :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 10:50:32 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F3B6E37B401; Tue, 5 Aug 2003 10:50:31 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.bayarea.net [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0B50F43FB1; Tue, 5 Aug 2003 10:50:31 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75HoUwO032249; Tue, 5 Aug 2003 10:50:30 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75HoUCn000992; Tue, 5 Aug 2003 10:50:30 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75HoUdm000991; Tue, 5 Aug 2003 10:50:30 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 10:50:30 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030805175030.GA901@dhcp01.pn.xcllnt.net> References: <20030805165817.GA796@dhcp01.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 17:50:32 -0000 On Tue, Aug 05, 2003 at 01:33:14PM -0400, Daniel Eischen wrote: > > > Am I right that this is currently the only way libkse can work and > > that I have to wait until the patches that are floating around get > > committed? > > I'm almost ready to commit them. Trying to figure out why > sysarch(AMD64_SET_FSBASE, foo) doesn't work on sledge (amd64). Ok. I wont commit the per-KSE hacks then. > See the ia64 part of the above patch. Since getcontext() > and setcontext() are not meant to be used by applications > to switch between contexts in different threads, they > shouldn't need to save and restore TP. But, the MD parts > of libpthread, _thread_enter_uts() and _thread_switch(), > do need to handle this. I _think_ I took care of this > for ia64, but you might want to take a close look at the > patch to make sure. The patch does not contain ia64 (yet), but libpthread.ia64.diffs does indeed have the code that deals with TP. Thanks, I now know what to commit and what not. I'll be testing the patch after committing... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 11:02:05 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 9FD6837B401 for ; Tue, 5 Aug 2003 11:02:05 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E0C8843F93 for ; Tue, 5 Aug 2003 11:02:04 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75I24nj009022; Tue, 5 Aug 2003 14:02:04 -0400 (EDT) Date: Tue, 5 Aug 2003 14:02:04 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030805175030.GA901@dhcp01.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 18:02:05 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > On Tue, Aug 05, 2003 at 01:33:14PM -0400, Daniel Eischen wrote: > > > > > Am I right that this is currently the only way libkse can work and > > > that I have to wait until the patches that are floating around get > > > committed? > > > > I'm almost ready to commit them. Trying to figure out why > > sysarch(AMD64_SET_FSBASE, foo) doesn't work on sledge (amd64). > > Ok. I wont commit the per-KSE hacks then. > > > See the ia64 part of the above patch. Since getcontext() > > and setcontext() are not meant to be used by applications > > to switch between contexts in different threads, they > > shouldn't need to save and restore TP. But, the MD parts > > of libpthread, _thread_enter_uts() and _thread_switch(), > > do need to handle this. I _think_ I took care of this > > for ia64, but you might want to take a close look at the > > patch to make sure. > > The patch does not contain ia64 (yet), but libpthread.ia64.diffs > does indeed have the code that deals with TP. Oh shoot. Terribly sorry :-( I didn't change the link to point to the latest patch file. If you grab it again, it should be updated. > Thanks, I now know what to commit and what not. I'll be testing > the patch after committing... Thanks. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 13:05:55 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 839DD37B404; Tue, 5 Aug 2003 13:05:55 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 51C6243FA3; Tue, 5 Aug 2003 13:05:54 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75K5swO032933; Tue, 5 Aug 2003 13:05:54 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75K5oYn000848; Tue, 5 Aug 2003 13:05:50 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75K5kwL000847; Tue, 5 Aug 2003 13:05:46 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 13:05:46 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030805200546.GA825@athlon.pn.xcllnt.net> References: <20030805175030.GA901@dhcp01.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="jRHKVT23PllUwdXP" Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 20:05:55 -0000 --jRHKVT23PllUwdXP Content-Type: text/plain; charset=us-ascii Content-Disposition: inline On Tue, Aug 05, 2003 at 02:02:04PM -0400, Daniel Eischen wrote: > > > > The patch does not contain ia64 (yet), but libpthread.ia64.diffs > > does indeed have the code that deals with TP. > > Oh shoot. Terribly sorry :-( I didn't change the link > to point to the latest patch file. If you grab it again, > it should be updated. Attached the diff again pthread_md.h (ia64) after my commits. Mostly merge conflict resolutions. FYI, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net --jRHKVT23PllUwdXP Content-Type: text/plain; charset=us-ascii Content-Disposition: attachment; filename="libpthread.diff" Index: pthread_md.h =================================================================== RCS file: /home/ncvs/src/lib/libpthread/arch/ia64/include/pthread_md.h,v retrieving revision 1.4 diff -u -r1.4 pthread_md.h --- pthread_md.h 5 Aug 2003 19:37:20 -0000 1.4 +++ pthread_md.h 5 Aug 2003 20:01:51 -0000 @@ -29,32 +29,188 @@ #ifndef _PTHREAD_MD_H_ #define _PTHREAD_MD_H_ +#include +#include + #define THR_GETCONTEXT(ucp) _ia64_save_context(&(ucp)->uc_mcontext) #define THR_SETCONTEXT(ucp) _ia64_restore_context(&(ucp)->uc_mcontext, \ 0, NULL) -#define THR_ALIGNBYTES 0 -#define THR_ALIGN(td) (td) +#define PER_THREAD + +struct kcb; +struct kse; +struct pthread; +struct tcb; +struct tdv; /* We don't know what this is yet? */ + +/* + * tp points to one of these. + */ +struct ia64_tp { + struct tdv *tp_tdv; /* dynamic TLS */ + struct tcb *tp_self; + char tp_tls[0]; /* static TLS */ +}; + +struct tcb { + struct kse_thr_mailbox tcb_tmbx; + struct pthread *tcb_thread; + struct kcb *tcb_curkcb; + long tcb_isfake; + struct ia64_tp tcb_tp; +}; -/* KSE Specific Data. */ -struct ksd { - void *ksd_base; - int ksd_size; +struct kcb { + struct kse_mailbox kcb_kmbx; + struct tcb kcb_faketcb; + struct tcb *kcb_curtcb; + struct kse *kcb_kse; }; +register struct ia64_tp *_tp __asm("%r13"); + +#ifndef TMF_NOUPCALL +#define TMF_NOUPCALL 0x1 +#endif + +/* + * The kcb and tcb constructors. + */ +struct tcb *_tcb_ctor(struct pthread *); +void _tcb_dtor(struct tcb *); +struct kcb *_kcb_ctor(struct kse *kse); +void _kcb_dtor(struct kcb *); + +/* Called from the KSE to set its private data. */ +static __inline void +_kcb_set(struct kcb *kcb) +{ + /* There is no thread yet; use the fake tcb. */ + _tp = &kcb->kcb_faketcb.tcb_tp; +} + +/* + * Get the current kcb. + * + * This can only be called while in a critical region; don't + * worry about having the kcb changed out from under us. + */ +static __inline struct kcb * +_kcb_get(void) +{ + return (_tp->tp_self->tcb_curkcb); +} + +/* + * Enter a critical region. + * + * Read and clear km_curthread in the kse mailbox. + */ +static __inline struct kse_thr_mailbox * +_kcb_critical_enter(void) +{ + struct kse_thr_mailbox *crit; + struct tcb *tcb; + uint32_t flags; + + tcb = _tp->tp_self; + if (tcb->tcb_isfake != 0) { + /* + * We already are in a critical region since + * there is no current thread. + */ + crit = NULL; + } else { + flags = tcb->tcb_tmbx.tm_flags; + tcb->tcb_tmbx.tm_flags |= TMF_NOUPCALL; + crit = tcb->tcb_curkcb->kcb_kmbx.km_curthread; + tcb->tcb_curkcb->kcb_kmbx.km_curthread = NULL; + tcb->tcb_tmbx.tm_flags = flags; + } + return (crit); +} + +static __inline void +_kcb_critical_leave(struct kse_thr_mailbox *crit) +{ + struct tcb *tcb; + + tcb = _tp->tp_self; + /* No need to do anything if this is a fake tcb. */ + if (tcb->tcb_isfake == 0) + tcb->tcb_curkcb->kcb_kmbx.km_curthread = crit; +} + +static __inline int +_kcb_in_critical(void) +{ + struct tcb *tcb; + uint32_t flags; + int ret; + + tcb = _tp->tp_self; + if (tcb->tcb_isfake != 0) { + /* + * We are in a critical region since there is no + * current thread. + */ + ret = 1; + } else { + flags = tcb->tcb_tmbx.tm_flags; + tcb->tcb_tmbx.tm_flags |= TMF_NOUPCALL; + ret = (tcb->tcb_curkcb->kcb_kmbx.km_curthread == NULL); + tcb->tcb_tmbx.tm_flags = flags; + } + return (ret); +} + +static __inline void +_tcb_set(struct kcb *kcb, struct tcb *tcb) +{ + kcb->kcb_curtcb = tcb; + tcb->tcb_curkcb = kcb; + _tp = &tcb->tcb_tp; +} + +static __inline struct tcb * +_tcb_get(void) +{ + return (_tp->tp_self); +} + +static __inline struct pthread * +_get_curthread(void) +{ + return (_tp->tp_self->tcb_thread); +} + +/* + * Get the current kse. + * + * Line _kcb_get(), this can only be called while in a critical region. + */ +static __inline struct kse * +_get_curkse(void) +{ + return (_tp->tp_self->tcb_curkcb->kcb_kse); +} + void _ia64_enter_uts(kse_func_t uts, struct kse_mailbox *km, void *stack, size_t stacksz); int _ia64_restore_context(mcontext_t *mc, intptr_t val, intptr_t *loc); int _ia64_save_context(mcontext_t *mc); static __inline int -_thread_enter_uts(struct kse_thr_mailbox *tm, struct kse_mailbox *km) +_thread_enter_uts(struct tcb *tcb, struct kcb *kcb) { - if (tm == NULL) - return (-1); - if (!_ia64_save_context(&tm->tm_context.uc_mcontext)) { - _ia64_enter_uts(km->km_func, km, km->km_stack.ss_sp, - km->km_stack.ss_size); + if (_ia64_save_context(&tcb->tcb_tmbx.tm_context.uc_mcontext) == 0) { + /* Make the fake tcb the current thread. */ + kcb->kcb_curtcb = &kcb->kcb_faketcb; + _tp = &kcb->kcb_faketcb.tcb_tp; + _ia64_enter_uts(kcb->kcb_kmbx.km_func, &kcb->kcb_kmbx, + kcb->kcb_kmbx.km_stack.ss_sp, + kcb->kcb_kmbx.km_stack.ss_size); /* We should not reach here. */ return (-1); } @@ -62,12 +218,18 @@ } static __inline int -_thread_switch(struct kse_thr_mailbox *tm, struct kse_thr_mailbox **thrp) +_thread_switch(struct kcb *kcb, struct tcb *tcb, int setmbox) { - if (tm == NULL) - return (-1); - _ia64_restore_context(&tm->tm_context.uc_mcontext, (intptr_t)tm, - (intptr_t*)thrp); + kcb->kcb_curtcb = tcb; + tcb->tcb_curkcb = kcb; + _tp = &tcb->tcb_tp; + if (setmbox != 0) + _ia64_restore_context(&tcb->tcb_tmbx.tm_context.uc_mcontext, + (intptr_t)&tcb->tcb_tmbx, + (intptr_t *)&kcb->kcb_kmbx.km_curthread); + else + _ia64_restore_context(&tcb->tcb_tmbx.tm_context.uc_mcontext, + 0, NULL); /* We should not reach here. */ return (-1); } --jRHKVT23PllUwdXP-- From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 13:12:02 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D846737B40C for ; Tue, 5 Aug 2003 13:12:02 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 27FB643F3F for ; Tue, 5 Aug 2003 13:12:02 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75KC1nj001059; Tue, 5 Aug 2003 16:12:01 -0400 (EDT) Date: Tue, 5 Aug 2003 16:12:01 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030805200546.GA825@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 20:12:03 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > On Tue, Aug 05, 2003 at 02:02:04PM -0400, Daniel Eischen wrote: > > > > > > The patch does not contain ia64 (yet), but libpthread.ia64.diffs > > > does indeed have the code that deals with TP. > > > > Oh shoot. Terribly sorry :-( I didn't change the link > > to point to the latest patch file. If you grab it again, > > it should be updated. > > Attached the diff again pthread_md.h (ia64) after my commits. > Mostly merge conflict resolutions. Got it, thanks :) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 13:32:15 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4B7BB37B401; Tue, 5 Aug 2003 13:32:15 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 42C8F43FA3; Tue, 5 Aug 2003 13:32:14 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75KWEwO033053; Tue, 5 Aug 2003 13:32:14 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75KWDYn000918; Tue, 5 Aug 2003 13:32:13 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75KWDb3000917; Tue, 5 Aug 2003 13:32:13 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 13:32:13 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030805203213.GA879@athlon.pn.xcllnt.net> References: <20030805200546.GA825@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 20:32:15 -0000 On Tue, Aug 05, 2003 at 04:12:01PM -0400, Daniel Eischen wrote: > On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > > > On Tue, Aug 05, 2003 at 02:02:04PM -0400, Daniel Eischen wrote: > > > > > > > > The patch does not contain ia64 (yet), but libpthread.ia64.diffs > > > > does indeed have the code that deals with TP. > > > > > > Oh shoot. Terribly sorry :-( I didn't change the link > > > to point to the latest patch file. If you grab it again, > > > it should be updated. > > > > Attached the diff again pthread_md.h (ia64) after my commits. > > Mostly merge conflict resolutions. > > Got it, thanks :) I got a fix: In struct ia64_tp we define tp_tls as an array of char. If we define it as an array of long double we automaticly have 16-byte alignment of the static TLS, struct ia64_tp, struct tcb and struct kcb. Allocating the TCB will then automaticly ensure that the static TLS is properly aligned. I'm currently testing with the following (re)definition of struct ia64_tp: struct ia64_tp { struct tdv *tp_tdv; /* dynamic TLS */ struct tcb *tp_self; long double tp_tls[0]; /* static TLS */ }; BTW: I'm also thinking about replacing tp_self by tp_thread and use the following to get to the TCB: #define _CURTCB(tp) (struct tcb*)((uintptr_t)tp - offsetof(struct tcb, tcb_tp)) This should remove some double-indirections to get to struct pthread. I haven't made the change locally yet. I first want to get the show on the road. FYI, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 13:43:02 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 0A5A537B401 for ; Tue, 5 Aug 2003 13:43:02 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 358F443FA3 for ; Tue, 5 Aug 2003 13:43:01 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75Kh0nj007005; Tue, 5 Aug 2003 16:43:00 -0400 (EDT) Date: Tue, 5 Aug 2003 16:43:00 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030805203213.GA879@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 20:43:02 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > On Tue, Aug 05, 2003 at 04:12:01PM -0400, Daniel Eischen wrote: > > On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > > > > > On Tue, Aug 05, 2003 at 02:02:04PM -0400, Daniel Eischen wrote: > > > > > > > > > > The patch does not contain ia64 (yet), but libpthread.ia64.diffs > > > > > does indeed have the code that deals with TP. > > > > > > > > Oh shoot. Terribly sorry :-( I didn't change the link > > > > to point to the latest patch file. If you grab it again, > > > > it should be updated. > > > > > > Attached the diff again pthread_md.h (ia64) after my commits. > > > Mostly merge conflict resolutions. > > > > Got it, thanks :) > > I got a fix: > In struct ia64_tp we define tp_tls as an array of char. If we > define it as an array of long double we automaticly have 16-byte > alignment of the static TLS, struct ia64_tp, struct tcb and > struct kcb. Allocating the TCB will then automaticly ensure that > the static TLS is properly aligned. I'm currently testing with > the following (re)definition of struct ia64_tp: > > struct ia64_tp { > struct tdv *tp_tdv; /* dynamic TLS */ > struct tcb *tp_self; > long double tp_tls[0]; /* static TLS */ > }; Sure; that was merely a placeholder so one (you) could replace it with whatever is needed. I assume this (static TLS) will have some predetermined size... > BTW: I'm also thinking about replacing tp_self by tp_thread and use > the following to get to the TCB: > > #define _CURTCB(tp) (struct tcb*)((uintptr_t)tp - offsetof(struct tcb, tcb_tp)) > > This should remove some double-indirections to get to struct pthread. > I haven't made the change locally yet. I first want to get the > show on the road. Sure, that works for me as well. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 14:00:12 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id D778E37B401; Tue, 5 Aug 2003 14:00:12 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1511B43F3F; Tue, 5 Aug 2003 14:00:12 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75L0BwO033189; Tue, 5 Aug 2003 14:00:11 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75L0BYn001000; Tue, 5 Aug 2003 14:00:11 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75L0BAU000999; Tue, 5 Aug 2003 14:00:11 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 14:00:11 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030805210011.GB879@athlon.pn.xcllnt.net> References: <20030805203213.GA879@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 21:00:13 -0000 On Tue, Aug 05, 2003 at 04:43:00PM -0400, Daniel Eischen wrote: > > In struct ia64_tp we define tp_tls as an array of char. If we > > define it as an array of long double we automaticly have 16-byte > > alignment of the static TLS, struct ia64_tp, struct tcb and > > struct kcb. Allocating the TCB will then automaticly ensure that > > the static TLS is properly aligned. I'm currently testing with > > the following (re)definition of struct ia64_tp: > > > > struct ia64_tp { > > struct tdv *tp_tdv; /* dynamic TLS */ > > struct tcb *tp_self; > > long double tp_tls[0]; /* static TLS */ > > }; > > Sure; that was merely a placeholder so one (you) could replace > it with whatever is needed. I assume this (static TLS) will > have some predetermined size... It's a runtime constant yes. We'll know the size of the static TLS when we initialize libkse/libpthread and TLS support has been added. I expect that kcb_faketcb doesn't need any TLS, because it's not used for running user code, just an internal "doohicky", right? BTW: Feel free to commit your patch at your earliest convenience (with or without the change described above). I see a slight regression after applying the patch, but much rather see it committed than having to work with a large patch... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 14:12:20 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 85F9337B401 for ; Tue, 5 Aug 2003 14:12:20 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id E26C643F93 for ; Tue, 5 Aug 2003 14:12:19 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75LCInj011940; Tue, 5 Aug 2003 17:12:19 -0400 (EDT) Date: Tue, 5 Aug 2003 17:12:18 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030805210011.GB879@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 21:12:20 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > On Tue, Aug 05, 2003 at 04:43:00PM -0400, Daniel Eischen wrote: > > > In struct ia64_tp we define tp_tls as an array of char. If we > > > define it as an array of long double we automaticly have 16-byte > > > alignment of the static TLS, struct ia64_tp, struct tcb and > > > struct kcb. Allocating the TCB will then automaticly ensure that > > > the static TLS is properly aligned. I'm currently testing with > > > the following (re)definition of struct ia64_tp: > > > > > > struct ia64_tp { > > > struct tdv *tp_tdv; /* dynamic TLS */ > > > struct tcb *tp_self; > > > long double tp_tls[0]; /* static TLS */ > > > }; > > > > Sure; that was merely a placeholder so one (you) could replace > > it with whatever is needed. I assume this (static TLS) will > > have some predetermined size... > > It's a runtime constant yes. We'll know the size of the static TLS > when we initialize libkse/libpthread and TLS support has been added. Do we need an additional parameter to _tcb_ctor() to specify the static TLS size? > I expect that kcb_faketcb doesn't need any TLS, because it's not > used for running user code, just an internal "doohicky", right? Right; the UTS needs some way to get to the current KSE. > BTW: Feel free to commit your patch at your earliest convenience > (with or without the change described above). I see a slight > regression after applying the patch, but much rather see it > committed than having to work with a large patch... Hmm, ok. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 14:17:01 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 141AD37B401; Tue, 5 Aug 2003 14:17:01 -0700 (PDT) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id D8B9343FA3; Tue, 5 Aug 2003 14:16:59 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc12) with ESMTP id <20030805211659014007mv0fe>; Tue, 5 Aug 2003 21:16:59 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA94067; Tue, 5 Aug 2003 14:16:57 -0700 (PDT) Date: Tue, 5 Aug 2003 14:16:55 -0700 (PDT) From: Julian Elischer To: Marcel Moolenaar In-Reply-To: <20030805210011.GB879@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 21:17:01 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > On Tue, Aug 05, 2003 at 04:43:00PM -0400, Daniel Eischen wrote: > > > In struct ia64_tp we define tp_tls as an array of char. If we > > > define it as an array of long double we automaticly have 16-byte > > > alignment of the static TLS, struct ia64_tp, struct tcb and > > > struct kcb. Allocating the TCB will then automaticly ensure that > > > the static TLS is properly aligned. I'm currently testing with > > > the following (re)definition of struct ia64_tp: > > > > > > struct ia64_tp { > > > struct tdv *tp_tdv; /* dynamic TLS */ > > > struct tcb *tp_self; > > > long double tp_tls[0]; /* static TLS */ > > > }; > > > > Sure; that was merely a placeholder so one (you) could replace > > it with whatever is needed. I assume this (static TLS) will > > have some predetermined size... > > It's a runtime constant yes. We'll know the size of the static TLS > when we initialize libkse/libpthread and TLS support has been added. > I expect that kcb_faketcb doesn't need any TLS, because it's not > used for running user code, just an internal "doohicky", right? > > BTW: Feel free to commit your patch at your earliest convenience > (with or without the change described above). I see a slight > regression after applying the patch, but much rather see it > committed than having to work with a large patch... you mean a 'speed' regression? don't forget that you can make sure you only do extra work if p->p_flag & P_SA is true.. > > -- > Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 14:49:58 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 878FC37B401; Tue, 5 Aug 2003 14:49:58 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 72E9043F3F; Tue, 5 Aug 2003 14:49:57 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75LnNwO033386; Tue, 5 Aug 2003 14:49:23 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75LnNCn001674; Tue, 5 Aug 2003 14:49:23 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75LnN3E001673; Tue, 5 Aug 2003 14:49:23 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 14:49:23 -0700 From: Marcel Moolenaar To: Julian Elischer Message-ID: <20030805214923.GA1633@dhcp01.pn.xcllnt.net> References: <20030805210011.GB879@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: deischen@freebsd.org cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 21:49:58 -0000 On Tue, Aug 05, 2003 at 02:16:55PM -0700, Julian Elischer wrote: > > > BTW: Feel free to commit your patch at your earliest convenience > > (with or without the change described above). I see a slight > > regression after applying the patch, but much rather see it > > committed than having to work with a large patch... > > you mean a 'speed' regression? A functional regression. There's a bug in _tcb_set() The tcb argument can be NULL and we unconditionally dereference it. Hence, kse_sched_multi() now causes segfaults. It did not do that before :-) Something else is still fishy though: itanium% ./kse Using 5 threads (default) bar 0 Segmentation fault (core dumped) It should be something like: itanium% ./thr Using 5 threads (default) bar 1 bar 2 bar 3 bar 0 bar 4 The bar # lines are randomized, so the order does not have to be the same. But, I'm almost there... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 14:53:08 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id CB54137B401; Tue, 5 Aug 2003 14:53:08 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1195643F75; Tue, 5 Aug 2003 14:53:08 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from dhcp01.pn.xcllnt.net (dhcp01.pn.xcllnt.net [192.168.4.201]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75Lr7wO033416; Tue, 5 Aug 2003 14:53:07 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from dhcp01.pn.xcllnt.net (localhost [127.0.0.1]) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h75Lr7Cn001684; Tue, 5 Aug 2003 14:53:07 -0700 (PDT) (envelope-from marcel@dhcp01.pn.xcllnt.net) Received: (from marcel@localhost) by dhcp01.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h75Lr7Ca001683; Tue, 5 Aug 2003 14:53:07 -0700 (PDT) (envelope-from marcel) Date: Tue, 5 Aug 2003 14:53:07 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030805215307.GB1633@dhcp01.pn.xcllnt.net> References: <20030805210011.GB879@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 21:53:09 -0000 On Tue, Aug 05, 2003 at 05:12:18PM -0400, Daniel Eischen wrote: > > > it with whatever is needed. I assume this (static TLS) will > > > have some predetermined size... > > > > It's a runtime constant yes. We'll know the size of the static TLS > > when we initialize libkse/libpthread and TLS support has been added. > > Do we need an additional parameter to _tcb_ctor() to specify > the static TLS size? It's a global contant. We don't have to pass it around. I expect that for dynamicly linked programs the dynamic linker will provide it. So, I think _tcb_ctor can just grab it directly or indirectly using a well-known function. > > BTW: Feel free to commit your patch at your earliest convenience > > (with or without the change described above). I see a slight > > regression after applying the patch, but much rather see it > > committed than having to work with a large patch... > > Hmm, ok. The regression is caused by _tcb_set(). The tcb argument can be NULL, but we derefernce it unconditionally. See also my reply to Julian... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 15:06:45 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 214E437B401 for ; Tue, 5 Aug 2003 15:06:45 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7AF9A43F3F for ; Tue, 5 Aug 2003 15:06:44 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75M6cnj021214; Tue, 5 Aug 2003 18:06:38 -0400 (EDT) Date: Tue, 5 Aug 2003 18:06:38 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030805214923.GA1633@dhcp01.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Julian Elischer Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 22:06:45 -0000 On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > On Tue, Aug 05, 2003 at 02:16:55PM -0700, Julian Elischer wrote: > > > > > BTW: Feel free to commit your patch at your earliest convenience > > > (with or without the change described above). I see a slight > > > regression after applying the patch, but much rather see it > > > committed than having to work with a large patch... > > > > you mean a 'speed' regression? > > A functional regression. There's a bug in _tcb_set() The tcb > argument can be NULL and we unconditionally dereference it. > Hence, kse_sched_multi() now causes segfaults. It did not do > that before :-) Yup, you're right. I think you want something like this: static __inline void _tcb_set(struct kcb *kcb, struct tcb *tcb) { if (tcb == NULL { kcb->kcb_curtcb = &kcb->kcb_faketcb; _tp = &kcb->kcb_faketcb.tcb_tp; } else { kcb->kcb_curtcb = tcb; tcb->tcb_curkcb = kcb; _tp = &tcb->tcb_tp; } } > > Something else is still fishy though: > > itanium% ./kse > Using 5 threads (default) > bar 0 > Segmentation fault (core dumped) > > It should be something like: > > itanium% ./thr > Using 5 threads (default) > bar 1 > bar 2 > bar 3 > bar 0 > bar 4 > > The bar # lines are randomized, so the order does not have to be > the same. But, I'm almost there... Great, let me know what you find. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 15:22:09 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5CE2E37B401; Tue, 5 Aug 2003 15:22:09 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id EEF4C43F3F; Tue, 5 Aug 2003 15:22:08 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) Received: from localhost (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h75MM6Up085138; Tue, 5 Aug 2003 15:22:07 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) From: David Xu To: Julian Elischer , Marcel Moolenaar Date: Wed, 6 Aug 2003 06:25:09 +0800 User-Agent: KMail/1.5.2 References: In-Reply-To: MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 7bit Content-Disposition: inline Message-Id: <200308060625.09817.davidxu@FreeBSD.org> cc: threads@FreeBSD.org Subject: Re: Good news: KSE on ia64 is starting to work X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: davidxu@FreeBSD.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 22:22:09 -0000 On Wednesday 06 August 2003 01:22, Julian Elischer wrote: > On Tue, 5 Aug 2003, Marcel Moolenaar wrote: > > Ok, > > > > I have a couple of changes and fixes in my tree that makes KSE on > > ia64 work. I have a couple of questions before I can commit it: > > On ia64 the TP will point to the TDB, as the spec says. > > there is only one case where this negatively impacts > libKSE and that is where the thread is setting itself into a critical > region. > > in i386 it is > > movl #0, %gs(offset) (clear a value in the KCB) > > which can be done atomically > since on ia64 TP points to teh THREAD structure, > we have a second method of setting a critical region which is inthe TCB > instead of being in the KCB, because > > using pdp11 syntax, > offset(TCB)->reg > NULL -> offset2(reg) > > cannot be done atomically and pre-emption may result in teh thread > switching KSEs between teh two instructions. > So on ia64 the critical region is defined as being enterred when > the alternative flag in the TCB is used.. > > David and Dan have more info and patches > I had committed a patch for kernel, the patch introduces a TMF_NOUPCALL flag which can be used for ia64 and alpha in libkse. I think Dan can commit his libkse patch now. David Xu From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 16:16:40 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7A37737B401 for ; Tue, 5 Aug 2003 16:16:40 -0700 (PDT) Received: from pool-151-200-10-97.res.east.verizon.net (pool-138-88-38-229.res.east.verizon.net [138.88.38.229]) by mx1.FreeBSD.org (Postfix) with ESMTP id C0BD743FA3 for ; Tue, 5 Aug 2003 16:16:38 -0700 (PDT) (envelope-from mtm@identd.net) Received: from kokeb.ambesa.net (vq3ew69v45uqt1ja@localhost [127.0.0.1]) id h75NGb1h025102; Tue, 5 Aug 2003 19:16:37 -0400 (EDT) (envelope-from mtm@identd.net) Received: (from mtm@localhost) by kokeb.ambesa.net (8.12.9/8.12.9/Submit) id h75NGYjK025101; Tue, 5 Aug 2003 19:16:34 -0400 (EDT) (envelope-from mtm@identd.net) X-Authentication-Warning: kokeb.ambesa.net: mtm set sender to mtm@identd.net using -f Date: Tue, 5 Aug 2003 19:16:34 -0400 From: Mike Makonnen To: Marcel Moolenaar Message-ID: <20030805231633.GA24684@kokeb.ambesa.net> References: <002e01c358ce$6df2ecd0$0701a8c0@tiger> <20030802195243.GA3396@kokeb.ambesa.net> <20030802210339.GB25026@dhcp01.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030802210339.GB25026@dhcp01.pn.xcllnt.net> User-Agent: Mutt/1.4.1i X-Operating-System: FreeBSD/5.1-CURRENT (i386) cc: freebsd-threads@FreeBSD.Org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 23:16:40 -0000 On Sat, Aug 02, 2003 at 02:03:39PM -0700, Marcel Moolenaar wrote: > On Sat, Aug 02, 2003 at 03:52:44PM -0400, Mike Makonnen wrote: > > > > As I indicated to Julian, I don't have time to do this now. I will > > be going off-line in a couple of weeks for I don't know how long. > > In the mean time I have a bunch of other stuff I have to get to. > > But, if someone else wants to do it I can tell you exactly > > what you need to do. It's relatively simple. > > Can you post the description to threads@? > I'll see if I can find a spare moment... When I went back over the code, to write this summary I found it is going to be much easier than I thought at first. Assuming, i386_set_ldt() returns the auto-allocated LDT slot we just have to load that into %gs, instead of the ldt_index (which points into the ldt_entries array). So, if someone can tell me what the magic number for auto-allocation is I can take care of this tonight. But, because I'll be offline soon I'll go ahead and post the background on how that code works just in case someone needs to mess with it. The pointers to pthread structures are kept in the array ldt_entries[MAX_THR]. Currently, MAX_THR is set to 128, but since we start at index NLDT the maximum number of simultaneous threads in an application is MAX_THR - NLDT. The global variable ldt_free is used to keep track of the next free index in the array. To facilitate this, when libthr is first initialized each entry in the array contains a pointer to the next free index. The ldt_free variable is then set to point to the first free entry in the array (ldt_entries[NLDT]) and the contents of that index (as preveiously stated) will be a pointer to the next free entry in the array (in this case ldt_entries[NLDT + 1]). When a new pthread is created, ldt_free is dereferenced to find the next empty entry. It is set to this entry and the contents of the previous entry it pointed to are overwritten with the address of the newly created pthread. When a thread exits the array index it occupied is overwritten by a pointer to ldt_free and ldt_free is pointed to the array index that held the just freed pthread. Another thing that happens when a new pthread is created is that we setup a segment descriptor structure which points into the ldt_entries entry we just setup for our newly created pthread. This segment descripter pointing into ldt_entries is what is passed into i386_set_ldt(). If the call to i386_set_ldt() succeeds we then load the LDT index into %gs. So, there should be no need to bump up any limits or anything to accomodate this change. Cheers. -- Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc mtm@identd.net | D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 mtm@FreeBSD.Org| FreeBSD - Unleash the Daemon! From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 16:44:53 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8D60837B401 for ; Tue, 5 Aug 2003 16:44:53 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id D767743FAF for ; Tue, 5 Aug 2003 16:44:52 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h75NipuN007481; Tue, 5 Aug 2003 19:44:52 -0400 (EDT) Date: Tue, 5 Aug 2003 19:44:51 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Mike Makonnen In-Reply-To: <20030805231633.GA24684@kokeb.ambesa.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@freebsd.org cc: Marcel Moolenaar Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 05 Aug 2003 23:44:53 -0000 On Tue, 5 Aug 2003, Mike Makonnen wrote: > On Sat, Aug 02, 2003 at 02:03:39PM -0700, Marcel Moolenaar wrote: > > On Sat, Aug 02, 2003 at 03:52:44PM -0400, Mike Makonnen wrote: > > > > > > As I indicated to Julian, I don't have time to do this now. I will > > > be going off-line in a couple of weeks for I don't know how long. > > > In the mean time I have a bunch of other stuff I have to get to. > > > But, if someone else wants to do it I can tell you exactly > > > what you need to do. It's relatively simple. > > > > Can you post the description to threads@? > > I'll see if I can find a spare moment... > > When I went back over the code, to write this summary > I found it is going to be much easier than I thought > at first. Assuming, i386_set_ldt() returns the auto-allocated > LDT slot we just have to load that into %gs, instead of > the ldt_index (which points into the ldt_entries array). > > So, if someone can tell me what the magic number for > auto-allocation is I can take care of this tonight. But, > because I'll be offline soon I'll go ahead and post > the background on how that code works just in case someone > needs to mess with it. > > The pointers to pthread structures are kept in the array > ldt_entries[MAX_THR]. Currently, MAX_THR is set to 128, but > since we start at index NLDT the maximum number of simultaneous > threads in an application is MAX_THR - NLDT. The global variable > ldt_free is used to keep track of the next free index in the > array. To facilitate this, when libthr is first initialized > each entry in the array contains a pointer to the next free > index. The ldt_free variable is then set to point to the first > free entry in the array (ldt_entries[NLDT]) and the contents > of that index (as preveiously stated) will be a pointer to > the next free entry in the array (in this case > ldt_entries[NLDT + 1]). When a new pthread is created, ldt_free > is dereferenced to find the next empty entry. It is set to this > entry and the contents of the previous entry it pointed to > are overwritten with the address of the newly created pthread. > When a thread exits the array index it occupied is overwritten > by a pointer to ldt_free and ldt_free is pointed to the > array index that held the just freed pthread. > > Another thing that happens when a new pthread is created is that > we setup a segment descriptor structure which points into the > ldt_entries entry we just setup for our newly created pthread. > This segment descripter pointing into ldt_entries is what is > passed into i386_set_ldt(). If the call to i386_set_ldt() succeeds > we then load the LDT index into %gs. > > So, there should be no need to bump up any limits or anything > to accomodate this change. Code says it so much better than words. Here's a quick attempt to show you what is needed. I took the liberty of simplifying it a bit since with auto ldt allocation, I don't think there's any need to have a predetermined number of slots (it also eliminates the need to lock the entry array). Watch out for any line wraps... -- Dan Eischen Index: arch/i386/i386/_setcurthread.c =================================================================== RCS file: /opt/FreeBSD/cvs/src/lib/libthr/arch/i386/i386/_setcurthread.c,v retrieving revision 1.10 diff -u -r1.10 _setcurthread.c --- arch/i386/i386/_setcurthread.c 29 Jun 2003 00:12:39 -0000 1.10 +++ arch/i386/i386/_setcurthread.c 5 Aug 2003 23:36:10 -0000 @@ -39,105 +39,34 @@ #include "thr_private.h" -#define MAXTHR 128 - -#define LDT_INDEX(x) (((long)(x) - (long)ldt_entries) / sizeof(ldt_entries[0])) - -void **ldt_free = NULL; -void *ldt_entries[MAXTHR]; -static int ldt_inited = 0; -static spinlock_t ldt_lock = _SPINLOCK_INITIALIZER; - -static void ldt_init(void); - /* in _curthread.S */ extern void _set_gs(int); -/* - * Initialize the array of ldt_entries and the next free slot. - * This routine must be called with the global ldt lock held. - */ -static void -ldt_init(void) -{ - int i; - - ldt_free = &ldt_entries[NLDT]; - - for (i = 0; i < MAXTHR - 1; i++) - ldt_entries[i] = (void *)&ldt_entries[i + 1]; - - ldt_entries[MAXTHR - 1] = NULL; - - ldt_inited = 1; -} - void _retire_thread(void *entry) { - _spinlock(&ldt_lock); - if (ldt_free == NULL) - *(void **)entry = NULL; - else - *(void **)entry = *ldt_free; - ldt_free = entry; - _spinunlock(&ldt_lock); + int ldt; + + ldt = (int)entry; + if (ldt >= 0) + i386_set_ldt(ldt, NULL, 1); } void * _set_curthread(ucontext_t *uc, struct pthread *thr, int *err) { union descriptor desc; - void **ldt_entry; int ldt_index; - int error; *err = 0; - - /* - * If we are setting up the initial thread, the gs register - * won't be setup for the current thread. In any case, we - * don't need protection from re-entrancy at this point in - * the life of the program. - */ - if (thr != _thread_initial) - _SPINLOCK(&ldt_lock); - - if (ldt_inited == NULL) - ldt_init(); - - if (ldt_free == NULL) { - /* Concurrent thread limit reached */ - *err = curthread->error = EAGAIN; - if (thr != _thread_initial) - _SPINUNLOCK(&ldt_lock); - return (NULL); - } - - /* - * Pull one off of the free list and update the free list pointer. - */ - ldt_entry = ldt_free; - ldt_free = (void **)*ldt_entry; - - if (thr != _thread_initial) - _SPINUNLOCK(&ldt_lock); - - /* - * Cache the address of the thread structure here. This is - * what the gs register will point to. - */ - *ldt_entry = (void *)thr; - ldt_index = LDT_INDEX(ldt_entry); - bzero(&desc, sizeof(desc)); /* * Set up the descriptor to point into the ldt table which contains * only a pointer to the thread. */ - desc.sd.sd_lolimit = sizeof(*ldt_entry); - desc.sd.sd_lobase = (unsigned int)ldt_entry & 0xFFFFFF; + desc.sd.sd_lolimit = sizeof(*thr); + desc.sd.sd_lobase = (unsigned int)thr & 0xFFFFFF; desc.sd.sd_type = SDT_MEMRO; desc.sd.sd_dpl = SEL_UPL; desc.sd.sd_p = 1; @@ -145,19 +74,19 @@ desc.sd.sd_xx = 0; desc.sd.sd_def32 = 1; desc.sd.sd_gran = 0; - desc.sd.sd_hibase = (unsigned int)ldt_entry >> 24; + desc.sd.sd_hibase = (unsigned int)thr >> 24; - error = i386_set_ldt(ldt_index, &desc, 1); - if (error == -1) - abort(); - - /* - * Set up our gs with the index into the ldt for this entry. - */ - if (uc != NULL) - uc->uc_mcontext.mc_gs = LSEL(ldt_index, SEL_UPL); - else - _set_gs(LSEL(ldt_index, SEL_UPL)); - - return (ldt_entry); + ldt_index = i386_set_ldt(LDT_AUTO_ALLOC, &desc, 1); + if (ldt_index < 0) + *err = EAGAIN; + else { + /* + * Set up our gs with the ldt index. + */ + if (uc != NULL) + uc->uc_mcontext.mc_gs = LSEL(ldt_index, SEL_UPL); + else + _set_gs(LSEL(ldt_index, SEL_UPL)); + } + return ((void *)ldt_index); } From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 17:15:24 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 48F9737B401 for ; Tue, 5 Aug 2003 17:15:24 -0700 (PDT) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id B84EB43F85 for ; Tue, 5 Aug 2003 17:15:23 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc12) with ESMTP id <20030806001523014007ojm4e>; Wed, 6 Aug 2003 00:15:23 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA95621; Tue, 5 Aug 2003 17:15:18 -0700 (PDT) Date: Tue, 5 Aug 2003 17:15:17 -0700 (PDT) From: Julian Elischer To: Mike Makonnen In-Reply-To: <20030805231633.GA24684@kokeb.ambesa.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-threads@FreeBSD.Org cc: Marcel Moolenaar Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 00:15:24 -0000 check the change Dan just checked in... FreeBSD src repository Modified files: lib/libpthread/arch/i386/i386 pthread_md.c Log: Use auto LDT allocation for i386. Revision Changes Path 1.2 +6 -63 src/lib/libpthread/arch/i386/i386/pthread_md.c I have been thinking about this.. I assume you have some cached threads around, so you'd not be calling this every time there is a new thread.. For KSE we only do this for a new KSE which is a less frequent operation. On Tue, 5 Aug 2003, Mike Makonnen wrote: > On Sat, Aug 02, 2003 at 02:03:39PM -0700, Marcel Moolenaar wrote: > > On Sat, Aug 02, 2003 at 03:52:44PM -0400, Mike Makonnen wrote: > > > > > > As I indicated to Julian, I don't have time to do this now. I will > > > be going off-line in a couple of weeks for I don't know how long. > > > In the mean time I have a bunch of other stuff I have to get to. > > > But, if someone else wants to do it I can tell you exactly > > > what you need to do. It's relatively simple. > > > > Can you post the description to threads@? > > I'll see if I can find a spare moment... > > When I went back over the code, to write this summary > I found it is going to be much easier than I thought > at first. Assuming, i386_set_ldt() returns the auto-allocated > LDT slot we just have to load that into %gs, instead of > the ldt_index (which points into the ldt_entries array). The value returned is the slot number so you need to << by 3 and set the Local bit for a legit value for %gs It is the same value that was returned before except that the kernel choses it for you if you specify a location of LDT_AUTO_ALLOC instead of a legal slot number. (i.e. Do whatever you are doing now with the return value before you put it into %gs.) > > So, if someone can tell me what the magic number for > auto-allocation is I can take care of this tonight. But, > because I'll be offline soon I'll go ahead and post > the background on how that code works just in case someone > needs to mess with it. > > The pointers to pthread structures are kept in the array > ldt_entries[MAX_THR]. Currently, MAX_THR is set to 128, but > since we start at index NLDT the maximum number of simultaneous > threads in an application is MAX_THR - NLDT. The global variable > ldt_free is used to keep track of the next free index in the > array. To facilitate this, when libthr is first initialized > each entry in the array contains a pointer to the next free > index. The ldt_free variable is then set to point to the first > free entry in the array (ldt_entries[NLDT]) and the contents > of that index (as preveiously stated) will be a pointer to > the next free entry in the array (in this case > ldt_entries[NLDT + 1]). When a new pthread is created, ldt_free > is dereferenced to find the next empty entry. It is set to this > entry and the contents of the previous entry it pointed to > are overwritten with the address of the newly created pthread. > When a thread exits the array index it occupied is overwritten > by a pointer to ldt_free and ldt_free is pointed to the > array index that held the just freed pthread. > > Another thing that happens when a new pthread is created is that > we setup a segment descriptor structure which points into the > ldt_entries entry we just setup for our newly created pthread. > This segment descripter pointing into ldt_entries is what is > passed into i386_set_ldt(). If the call to i386_set_ldt() succeeds > we then load the LDT index into %gs. > > So, there should be no need to bump up any limits or anything > to accomodate this change. > > Cheers. > -- > Mike Makonnen | GPG-KEY: http://www.identd.net/~mtm/mtm.asc > mtm@identd.net | D228 1A6F C64E 120A A1C9 A3AA DAE1 E2AF DBCC 68B9 > mtm@FreeBSD.Org| FreeBSD - Unleash the Daemon! > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 17:26:54 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B9F9D37B404; Tue, 5 Aug 2003 17:26:54 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 7247343F3F; Tue, 5 Aug 2003 17:26:53 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc11) with ESMTP id <2003080600265201300hus49e>; Wed, 6 Aug 2003 00:26:52 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA95724; Tue, 5 Aug 2003 17:26:51 -0700 (PDT) Date: Tue, 5 Aug 2003 17:26:50 -0700 (PDT) From: Julian Elischer To: deischen@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Marcel Moolenaar cc: freebsd-threads@freebsd.org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 00:26:55 -0000 Warning warning warning.... The code that Dan shows ends up pointing %gs to the struct pthread structure.. however the ELF i386 (and amd64) ABI for TLS assumes that it points to a POINTER to the TCB (is that the same thing?). If the TCB (Thread control block) is the same thing as the struct pthread then you should probably make the first entry be a pointer to istelf or the TLS code generated by the linker (when enabled) will point to the wrong thing.. With the array there is now it just happens to come out right. On Tue, 5 Aug 2003, Daniel Eischen wrote: > On Tue, 5 Aug 2003, Mike Makonnen wrote: > > > On Sat, Aug 02, 2003 at 02:03:39PM -0700, Marcel Moolenaar wrote: > > > On Sat, Aug 02, 2003 at 03:52:44PM -0400, Mike Makonnen wrote: > > > > > > > > As I indicated to Julian, I don't have time to do this now. I will > > > > be going off-line in a couple of weeks for I don't know how long. > > > > In the mean time I have a bunch of other stuff I have to get to. > > > > But, if someone else wants to do it I can tell you exactly > > > > what you need to do. It's relatively simple. > > > > > > Can you post the description to threads@? > > > I'll see if I can find a spare moment... > > > > When I went back over the code, to write this summary > > I found it is going to be much easier than I thought > > at first. Assuming, i386_set_ldt() returns the auto-allocated > > LDT slot we just have to load that into %gs, instead of > > the ldt_index (which points into the ldt_entries array). > > > > So, if someone can tell me what the magic number for > > auto-allocation is I can take care of this tonight. But, > > because I'll be offline soon I'll go ahead and post > > the background on how that code works just in case someone > > needs to mess with it. > > > > The pointers to pthread structures are kept in the array > > ldt_entries[MAX_THR]. Currently, MAX_THR is set to 128, but > > since we start at index NLDT the maximum number of simultaneous > > threads in an application is MAX_THR - NLDT. The global variable > > ldt_free is used to keep track of the next free index in the > > array. To facilitate this, when libthr is first initialized > > each entry in the array contains a pointer to the next free > > index. The ldt_free variable is then set to point to the first > > free entry in the array (ldt_entries[NLDT]) and the contents > > of that index (as preveiously stated) will be a pointer to > > the next free entry in the array (in this case > > ldt_entries[NLDT + 1]). When a new pthread is created, ldt_free > > is dereferenced to find the next empty entry. It is set to this > > entry and the contents of the previous entry it pointed to > > are overwritten with the address of the newly created pthread. > > When a thread exits the array index it occupied is overwritten > > by a pointer to ldt_free and ldt_free is pointed to the > > array index that held the just freed pthread. > > > > Another thing that happens when a new pthread is created is that > > we setup a segment descriptor structure which points into the > > ldt_entries entry we just setup for our newly created pthread. > > This segment descripter pointing into ldt_entries is what is > > passed into i386_set_ldt(). If the call to i386_set_ldt() succeeds > > we then load the LDT index into %gs. > > > > So, there should be no need to bump up any limits or anything > > to accomodate this change. > > Code says it so much better than words. Here's a quick attempt > to show you what is needed. I took the liberty of simplifying > it a bit since with auto ldt allocation, I don't think there's > any need to have a predetermined number of slots (it also > eliminates the need to lock the entry array). > > Watch out for any line wraps... > > -- > Dan Eischen > > Index: arch/i386/i386/_setcurthread.c > =================================================================== > RCS file: /opt/FreeBSD/cvs/src/lib/libthr/arch/i386/i386/_setcurthread.c,v > retrieving revision 1.10 > diff -u -r1.10 _setcurthread.c > --- arch/i386/i386/_setcurthread.c 29 Jun 2003 00:12:39 -0000 1.10 > +++ arch/i386/i386/_setcurthread.c 5 Aug 2003 23:36:10 -0000 > @@ -39,105 +39,34 @@ > > #include "thr_private.h" > > -#define MAXTHR 128 > - > -#define LDT_INDEX(x) (((long)(x) - (long)ldt_entries) / sizeof(ldt_entries[0])) > - > -void **ldt_free = NULL; > -void *ldt_entries[MAXTHR]; > -static int ldt_inited = 0; > -static spinlock_t ldt_lock = _SPINLOCK_INITIALIZER; > - > -static void ldt_init(void); > - > /* in _curthread.S */ > extern void _set_gs(int); > > -/* > - * Initialize the array of ldt_entries and the next free slot. > - * This routine must be called with the global ldt lock held. > - */ > -static void > -ldt_init(void) > -{ > - int i; > - > - ldt_free = &ldt_entries[NLDT]; > - > - for (i = 0; i < MAXTHR - 1; i++) > - ldt_entries[i] = (void *)&ldt_entries[i + 1]; > - > - ldt_entries[MAXTHR - 1] = NULL; > - > - ldt_inited = 1; > -} > - > void > _retire_thread(void *entry) > { > - _spinlock(&ldt_lock); > - if (ldt_free == NULL) > - *(void **)entry = NULL; > - else > - *(void **)entry = *ldt_free; > - ldt_free = entry; > - _spinunlock(&ldt_lock); > + int ldt; > + > + ldt = (int)entry; > + if (ldt >= 0) > + i386_set_ldt(ldt, NULL, 1); > } > > void * > _set_curthread(ucontext_t *uc, struct pthread *thr, int *err) > { > union descriptor desc; > - void **ldt_entry; > int ldt_index; > - int error; > > *err = 0; > - > - /* > - * If we are setting up the initial thread, the gs register > - * won't be setup for the current thread. In any case, we > - * don't need protection from re-entrancy at this point in > - * the life of the program. > - */ > - if (thr != _thread_initial) > - _SPINLOCK(&ldt_lock); > - > - if (ldt_inited == NULL) > - ldt_init(); > - > - if (ldt_free == NULL) { > - /* Concurrent thread limit reached */ > - *err = curthread->error = EAGAIN; > - if (thr != _thread_initial) > - _SPINUNLOCK(&ldt_lock); > - return (NULL); > - } > - > - /* > - * Pull one off of the free list and update the free list pointer. > - */ > - ldt_entry = ldt_free; > - ldt_free = (void **)*ldt_entry; > - > - if (thr != _thread_initial) > - _SPINUNLOCK(&ldt_lock); > - > - /* > - * Cache the address of the thread structure here. This is > - * what the gs register will point to. > - */ > - *ldt_entry = (void *)thr; > - ldt_index = LDT_INDEX(ldt_entry); > - > bzero(&desc, sizeof(desc)); > > /* > * Set up the descriptor to point into the ldt table which contains > * only a pointer to the thread. > */ > - desc.sd.sd_lolimit = sizeof(*ldt_entry); > - desc.sd.sd_lobase = (unsigned int)ldt_entry & 0xFFFFFF; > + desc.sd.sd_lolimit = sizeof(*thr); > + desc.sd.sd_lobase = (unsigned int)thr & 0xFFFFFF; > desc.sd.sd_type = SDT_MEMRO; > desc.sd.sd_dpl = SEL_UPL; > desc.sd.sd_p = 1; > @@ -145,19 +74,19 @@ > desc.sd.sd_xx = 0; > desc.sd.sd_def32 = 1; > desc.sd.sd_gran = 0; > - desc.sd.sd_hibase = (unsigned int)ldt_entry >> 24; > + desc.sd.sd_hibase = (unsigned int)thr >> 24; > > - error = i386_set_ldt(ldt_index, &desc, 1); > - if (error == -1) > - abort(); > - > - /* > - * Set up our gs with the index into the ldt for this entry. > - */ > - if (uc != NULL) > - uc->uc_mcontext.mc_gs = LSEL(ldt_index, SEL_UPL); > - else > - _set_gs(LSEL(ldt_index, SEL_UPL)); > - > - return (ldt_entry); > + ldt_index = i386_set_ldt(LDT_AUTO_ALLOC, &desc, 1); > + if (ldt_index < 0) > + *err = EAGAIN; > + else { > + /* > + * Set up our gs with the ldt index. > + */ > + if (uc != NULL) > + uc->uc_mcontext.mc_gs = LSEL(ldt_index, SEL_UPL); > + else > + _set_gs(LSEL(ldt_index, SEL_UPL)); > + } > + return ((void *)ldt_index); > } > > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 17:55:35 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 07B8937B401 for ; Tue, 5 Aug 2003 17:55:35 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5EBA443F75 for ; Tue, 5 Aug 2003 17:55:34 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h760tTuN019950; Tue, 5 Aug 2003 20:55:29 -0400 (EDT) Date: Tue, 5 Aug 2003 20:55:29 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Marcel Moolenaar cc: freebsd-threads@freebsd.org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 00:55:35 -0000 On Tue, 5 Aug 2003, Julian Elischer wrote: > Warning warning warning.... > > The code that Dan shows ends up pointing %gs to the struct pthread > structure.. however the ELF i386 (and amd64) ABI for TLS assumes that it > points to a POINTER to the TCB (is that the same thing?). If the TCB > (Thread control block) is the same thing as the struct pthread then you > should probably make the first entry be a pointer to istelf or the TLS > code generated by the linker (when enabled) will point to the wrong > thing.. For libthr, the struct pthread isn't really the thread control block. struct pthread is MI, so you can't rely on it having a layout that will satisfy all the ABIs. It isn't correct now for i386. libthr will likely need the same sort of arch-dependent hooks that libpthread just grew (minus the struct kcb stuff). The patch I posted just addresses auto-ldt allocation. > > With the array there is now it just happens to come out right. But it doesn't really, since the first slot of struct pthread is not a pointer to the dynamic TLS. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 18:18:54 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8572F37B401; Tue, 5 Aug 2003 18:18:54 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 9001243FB1; Tue, 5 Aug 2003 18:18:53 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <20030806011848015009dcs8e>; Wed, 6 Aug 2003 01:18:48 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id SAA96141; Tue, 5 Aug 2003 18:18:47 -0700 (PDT) Date: Tue, 5 Aug 2003 18:18:45 -0700 (PDT) From: Julian Elischer To: deischen@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Marcel Moolenaar cc: freebsd-threads@freebsd.org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 01:18:54 -0000 On Tue, 5 Aug 2003, Daniel Eischen wrote: > On Tue, 5 Aug 2003, Julian Elischer wrote: > > > Warning warning warning.... > > > > The code that Dan shows ends up pointing %gs to the struct pthread > > structure.. however the ELF i386 (and amd64) ABI for TLS assumes that it > > points to a POINTER to the TCB (is that the same thing?). If the TCB > > (Thread control block) is the same thing as the struct pthread then you > > should probably make the first entry be a pointer to istelf or the TLS > > code generated by the linker (when enabled) will point to the wrong > > thing.. > > For libthr, the struct pthread isn't really the thread control > block. struct pthread is MI, so you can't rely on it having > a layout that will satisfy all the ABIs. It isn't correct > now for i386. > > libthr will likely need the same sort of arch-dependent > hooks that libpthread just grew (minus the struct kcb > stuff). The patch I posted just addresses auto-ldt > allocation. > > > > > With the array there is now it just happens to come out right. > > But it doesn't really, since the first slot of struct pthread > is not a pointer to the dynamic TLS. the i386 TLS spec doesn;t say it needs to be.. What it says is: %gs references a LDT entry that defines a segment, the forst word of which is a pointer to the Thread Control block, which, at a known offset (defined by a symbol), contains a pointer to the TLS vector array. This is satisfied now because %gs defiens a sebment that starts at one of the array entries in that array, and each array entry holds a pointer to the associated thread control block (struct pthread) which could hold a pointer to the TLS vector is it was added there.. if you bypass the array and make the segment contain the TCB directly then you need to add an artificial pointer to itself at the head so that the generated code does the right thing.. to get the address of the TLS vector it will do this: movl %gs:0 %eax ; get address of TCB movl tdpoff(%eax), %eax ; get address of vector in %eax This is the code that the linger expects to build.. if you don't have %gs pointed to a pointer, but instead, directly at the TCB you'd only need: movl gs:#tdpoff (or whatever the syntax for that is:) , %eax which MIGHT be a fraction faster, but is NOT what the linker is going to generate.. the code that the linker is going to generate is by chance also compatible with libc_r and with libkse as well. (assuming that we allocated a page at 0 and set %gs to be LUDATA). (or set a single LDT entry) (for KSE the indirection is in the KSE info) It also needs to be able to do -ve offsets from the base of the TCB and a segment (for normal data) cannot do -ve offsets and +ve offsets. so you MUST have the segment pointed to by %gs contain a POINTER to the TCB, not contain the TCB itself. (unless the pointer is in the tcb) (which is what the spec suggestes for the 1:1 case anyhow) > > -- > Dan Eischen > > From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 18:59:59 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1148C37B407 for ; Tue, 5 Aug 2003 18:59:59 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 334E443F3F for ; Tue, 5 Aug 2003 18:59:58 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h761xquN001581; Tue, 5 Aug 2003 21:59:52 -0400 (EDT) Date: Tue, 5 Aug 2003 21:59:52 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Marcel Moolenaar cc: freebsd-threads@freebsd.org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 01:59:59 -0000 On Tue, 5 Aug 2003, Julian Elischer wrote: > > > On Tue, 5 Aug 2003, Daniel Eischen wrote: > > > On Tue, 5 Aug 2003, Julian Elischer wrote: > > > > > Warning warning warning.... > > > > > > The code that Dan shows ends up pointing %gs to the struct pthread > > > structure.. however the ELF i386 (and amd64) ABI for TLS assumes that it > > > points to a POINTER to the TCB (is that the same thing?). If the TCB > > > (Thread control block) is the same thing as the struct pthread then you > > > should probably make the first entry be a pointer to istelf or the TLS > > > code generated by the linker (when enabled) will point to the wrong > > > thing.. > > > > For libthr, the struct pthread isn't really the thread control > > block. struct pthread is MI, so you can't rely on it having > > a layout that will satisfy all the ABIs. It isn't correct > > now for i386. > > > > libthr will likely need the same sort of arch-dependent > > hooks that libpthread just grew (minus the struct kcb > > stuff). The patch I posted just addresses auto-ldt > > allocation. > > > > > > > > With the array there is now it just happens to come out right. > > > > But it doesn't really, since the first slot of struct pthread > > is not a pointer to the dynamic TLS. > > the i386 TLS spec doesn;t say it needs to be.. > > > What it says is: > %gs references a LDT entry that defines a segment, the forst word of > which is a pointer to the Thread Control block, which, at a known > offset (defined by a symbol), contains a pointer to the TLS vector > array. > > This is satisfied now because %gs defiens a sebment that starts at one > of the array entries in that array, and each array entry holds a pointer > to the associated thread control block (struct pthread) which could hold > a pointer to the TLS vector is it was added there.. Well, it could for i386, but that doesn't hold water for other archs. Like I said, struct pthread is MI. I don't think you can satisfy all the ABIs with one layout of the MI struct pthread. Whatever is ABI-dependent should be independent of struct pthread. You could modify the current ldt entries in libthr so that they do contain the necessary dynamic and static TLS, and that would be fine. I just didn't worry about addressing it because it wasn't currently setup to handle it, and I didn't think it was necessary at this point. Here's a second pass that tries to handle the i386 ABI needs. There's no caching of old entries, but if you cache threads, you get the same effect and all archs benefit. -- Dan Eischen Index: arch/i386/i386/_curthread.S =================================================================== RCS file: /opt/FreeBSD/cvs/src/lib/libthr/arch/i386/i386/_curthread.S,v retrieving revision 1.2 diff -u -r1.2 _curthread.S --- arch/i386/i386/_curthread.S 2 Jun 2003 02:32:56 -0000 1.2 +++ arch/i386/i386/_curthread.S 6 Aug 2003 01:52:29 -0000 @@ -6,6 +6,7 @@ cmpl $0, _thread_initial je nothreads movl %gs:0, %eax + movl 4(%eax), %eax ret nothreads: xor %eax, %eax Index: arch/i386/i386/_setcurthread.c =================================================================== RCS file: /opt/FreeBSD/cvs/src/lib/libthr/arch/i386/i386/_setcurthread.c,v retrieving revision 1.10 diff -u -r1.10 _setcurthread.c --- arch/i386/i386/_setcurthread.c 29 Jun 2003 00:12:39 -0000 1.10 +++ arch/i386/i386/_setcurthread.c 6 Aug 2003 01:53:40 -0000 @@ -39,105 +39,64 @@ #include "thr_private.h" -#define MAXTHR 128 - -#define LDT_INDEX(x) (((long)(x) - (long)ldt_entries) / sizeof(ldt_entries[0])) - -void **ldt_free = NULL; -void *ldt_entries[MAXTHR]; -static int ldt_inited = 0; -static spinlock_t ldt_lock = _SPINLOCK_INITIALIZER; - -static void ldt_init(void); - /* in _curthread.S */ extern void _set_gs(int); +struct tcb { + struct tdv *tdv; /* dynamic TLS */ + struct pthread *thread; + /* static TLS ??? */ +}; + /* - * Initialize the array of ldt_entries and the next free slot. - * This routine must be called with the global ldt lock held. + * %gs points to one of these. */ -static void -ldt_init(void) -{ - int i; - - ldt_free = &ldt_entries[NLDT]; - - for (i = 0; i < MAXTHR - 1; i++) - ldt_entries[i] = (void *)&ldt_entries[i + 1]; +struct ldt_entry { + struct tcb *self; /* points to tcb */ + int ldt; + struct tcb tcb; +}; - ldt_entries[MAXTHR - 1] = NULL; - - ldt_inited = 1; -} void _retire_thread(void *entry) { - _spinlock(&ldt_lock); - if (ldt_free == NULL) - *(void **)entry = NULL; - else - *(void **)entry = *ldt_free; - ldt_free = entry; - _spinunlock(&ldt_lock); + struct ldt_entry *ldt; + + ldt = (struct ldt_entry *)entry; + if (ldt != NULL) { + if (ldt->ldt >= 0) { + i386_set_ldt(ldt->ldt, NULL, 1); + ldt->ldt = -1; /* just in case */ + } + free(ldt); + } } + void * _set_curthread(ucontext_t *uc, struct pthread *thr, int *err) { union descriptor desc; - void **ldt_entry; - int ldt_index; - int error; + struct ldt_entry *entry; *err = 0; + bzero(&desc, sizeof(desc)); - /* - * If we are setting up the initial thread, the gs register - * won't be setup for the current thread. In any case, we - * don't need protection from re-entrancy at this point in - * the life of the program. - */ - if (thr != _thread_initial) - _SPINLOCK(&ldt_lock); - - if (ldt_inited == NULL) - ldt_init(); - - if (ldt_free == NULL) { - /* Concurrent thread limit reached */ - *err = curthread->error = EAGAIN; - if (thr != _thread_initial) - _SPINUNLOCK(&ldt_lock); + if ((entry = malloc(sizeof(entry))) == NULL) { + *err = EAGAIN; return (NULL); } - - /* - * Pull one off of the free list and update the free list pointer. - */ - ldt_entry = ldt_free; - ldt_free = (void **)*ldt_entry; - - if (thr != _thread_initial) - _SPINUNLOCK(&ldt_lock); - - /* - * Cache the address of the thread structure here. This is - * what the gs register will point to. - */ - *ldt_entry = (void *)thr; - ldt_index = LDT_INDEX(ldt_entry); - - bzero(&desc, sizeof(desc)); + bzero(entry, sizeof(entry)); + entry->self = &entry->tcb; /* set up self reference */ + entry->tcb.thread = thr; /* * Set up the descriptor to point into the ldt table which contains * only a pointer to the thread. */ - desc.sd.sd_lolimit = sizeof(*ldt_entry); - desc.sd.sd_lobase = (unsigned int)ldt_entry & 0xFFFFFF; + desc.sd.sd_lolimit = sizeof(*entry); + desc.sd.sd_lobase = (unsigned int)entry & 0xFFFFFF; desc.sd.sd_type = SDT_MEMRO; desc.sd.sd_dpl = SEL_UPL; desc.sd.sd_p = 1; @@ -145,19 +104,20 @@ desc.sd.sd_xx = 0; desc.sd.sd_def32 = 1; desc.sd.sd_gran = 0; - desc.sd.sd_hibase = (unsigned int)ldt_entry >> 24; - - error = i386_set_ldt(ldt_index, &desc, 1); - if (error == -1) - abort(); + desc.sd.sd_hibase = (unsigned int)entry >> 24; - /* - * Set up our gs with the index into the ldt for this entry. - */ - if (uc != NULL) - uc->uc_mcontext.mc_gs = LSEL(ldt_index, SEL_UPL); - else - _set_gs(LSEL(ldt_index, SEL_UPL)); - - return (ldt_entry); + entry->ldt = i386_set_ldt(LDT_AUTO_ALLOC, &desc, 1); + if (entry->ldt < 0) { + *err = EAGAIN; + free(entry); + } else { + /* + * Set up our gs with the ldt index. + */ + if (uc != NULL) + uc->uc_mcontext.mc_gs = LSEL(entry->ldt, SEL_UPL); + else + _set_gs(LSEL(entry->ldt, SEL_UPL)); + } + return ((void *)entry); } From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 21:53:14 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1E76D37B401; Tue, 5 Aug 2003 21:53:14 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 88A4A43F75; Tue, 5 Aug 2003 21:53:13 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <20030806045312015009cbgie>; Wed, 6 Aug 2003 04:53:12 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id VAA97823; Tue, 5 Aug 2003 21:53:10 -0700 (PDT) Date: Tue, 5 Aug 2003 21:53:08 -0700 (PDT) From: Julian Elischer To: deischen@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Marcel Moolenaar cc: freebsd-threads@freebsd.org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 04:53:14 -0000 I have to look at your new patch later. I think I liked the first one more.... all that had to be changed was to add #if __i386__ struct pthread *self #endif at the head of struct pthread. (I think) From owner-freebsd-threads@FreeBSD.ORG Tue Aug 5 23:01:06 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id F347937B401 for ; Tue, 5 Aug 2003 23:01:05 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4308C43F3F for ; Tue, 5 Aug 2003 23:01:05 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h7660xuN013110; Wed, 6 Aug 2003 02:00:59 -0400 (EDT) Date: Wed, 6 Aug 2003 02:00:59 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Marcel Moolenaar cc: freebsd-threads@freebsd.org Subject: Re: cvs commit: src/sys/i386/i386 sys_machdep.c X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 06:01:06 -0000 On Tue, 5 Aug 2003, Julian Elischer wrote: > > I have to look at your new patch later. > > I think I liked the first one more.... > all that had to be changed was > to add > #if __i386__ > struct pthread *self > #endif > at the head of struct pthread. (I think) Yeah, I don't like that though because it dirties up MI code that would otherwise be clean. The second patch is very similar to how libpthread does it. But whatever; we've probably given Mike more than enough. We can just let him go off and do it _his_ way :-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 11:55:32 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7BCA037B401 for ; Wed, 6 Aug 2003 11:55:32 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 94AEF43FCB for ; Wed, 6 Aug 2003 11:55:31 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76ItVwO039333 for ; Wed, 6 Aug 2003 11:55:31 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76ItUG0000975 for ; Wed, 6 Aug 2003 11:55:30 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h76ItUdn000974 for threads@FreeBSD.org; Wed, 6 Aug 2003 11:55:30 -0700 (PDT) (envelope-from marcel) Date: Wed, 6 Aug 2003 11:55:30 -0700 From: Marcel Moolenaar To: threads@FreeBSD.org Message-ID: <20030806185530.GA893@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.5.4i Subject: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 18:55:32 -0000 Gang, Given the panics that Daniel is having on pluto1 it's probably a good idea to fill people in on the status of KSE/ia64: The groundwork is finished. I practical terms this means that I/O bound threads work to its fullest extend. There's just one tiny little annoying and complicated thing: thread_userret(), and consequently thread_export_context() can be called for interrupts, traps and faults as well. Since syscalls are not implemented as traps, we have two distinct paths into (and out from) the kernel. One (the syscall) is synchronous WRT to program execution and the other (interrupts) is asynchronous. Synchronous contexts don't have scratch registers in them. Asynchronous context need to have them. This is not the hard problem: just add some flags to indicate what parts of the context are valid and thus should be restored and we're ok. The problem is when we preempt an interrupted thread, export the context to the UTS and do an upcall. We end up having an async. context in userland. I'm not sure at this time what we should do with it. We have the following options: o Extend _ia64_restore_context() so that libkse can restore async contexts. The downside is that it will very likely cause a disabled high FP trap, which results in the process having the high FP registers enabled. A performance hit. (see also below) o Have _ia64_restore_context() call setcontext() for async contexts and do the work in the kernel. Restoring the high FP will not result in the enablement of the high FP registers, because we can restore them to the PCB. They will be loaded into the CPU when there's a need for them (which may be never). Both cases have the problem that we're using a synchronous method (the call/ret sequence) to restore an async context. I'm not sure how ugly it gets to change the return path and mimic an interrupt return. In short: The KSE framework works, as long as we don't preempt threads. I'm not sure how to solve that exactly... About the high FP: On ia64 the FP registers are split in two (2) sets: low and high. Both sets can be enabled and disabled independently from each other and each set has a modified bit to keep track of usage. The low FP registers are f0-f31 and are always enabled. The high FP registers are f32-f127 and are disabled by default. We use lazy context switching to save and restore these on a need to have basis. When a process uses a high FP register and the set is disabled, we take a trap, save the high FP registers currently on the CPU and load the high FP registers of the process that trapped. We then enable the high FP registers (for that process) and let it continue. As long as there's only 1 process using the high FP registers, there's no performance penalty when they are enabled. Note that compilers will avoid using the high FP registers as much as possible. AFAICT, none of the code in our source tree uses the high FP registers. Hence: they should only be enabled when the process is highly FP intensive. FYI, -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 12:01:33 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 8F5DE37B401 for ; Wed, 6 Aug 2003 12:01:33 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id D47D743F3F for ; Wed, 6 Aug 2003 12:01:32 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76J1WwO039372 for ; Wed, 6 Aug 2003 12:01:32 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76J1WG0001001 for ; Wed, 6 Aug 2003 12:01:32 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h76J1WMY001000 for threads@freebsd.org; Wed, 6 Aug 2003 12:01:32 -0700 (PDT) (envelope-from marcel) Date: Wed, 6 Aug 2003 12:01:32 -0700 From: Marcel Moolenaar To: threads@freebsd.org Message-ID: <20030806190132.GB893@athlon.pn.xcllnt.net> References: <20030806185530.GA893@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20030806185530.GA893@athlon.pn.xcllnt.net> User-Agent: Mutt/1.5.4i Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 19:01:33 -0000 On Wed, Aug 06, 2003 at 11:55:30AM -0700, Marcel Moolenaar wrote: > About the high FP: > > On ia64 the FP registers are split in two (2) sets: low and high. > Both sets can be enabled and disabled independently from each other > and each set has a modified bit to keep track of usage. The low > FP registers are f0-f31 and are always enabled. The high FP registers > are f32-f127 and are disabled by default. We use lazy context > switching to save and restore these on a need to have basis. When ^^^^^ These applies to the high FP registers only. We put the low FP registers in the trapframe or PCB. Only the high FP registers are lazily switched. -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 12:28:53 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 83CD837B401 for ; Wed, 6 Aug 2003 12:28:53 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id DB72D43FAF for ; Wed, 6 Aug 2003 12:28:52 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h76JSquN001604; Wed, 6 Aug 2003 15:28:52 -0400 (EDT) Date: Wed, 6 Aug 2003 15:28:52 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030806185530.GA893@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 19:28:53 -0000 On Wed, 6 Aug 2003, Marcel Moolenaar wrote: > Gang, > > Given the panics that Daniel is having on pluto1 it's probably a > good idea to fill people in on the status of KSE/ia64: First, thanks for your help on ia64. As I've found out, there still seems to be a lot more that needs to get done (in general, not KSE specific), and you seem to be the only one working on it. > The groundwork is finished. I practical terms this means that I/O > bound threads work to its fullest extend. There's just one tiny > little annoying and complicated thing: thread_userret(), and > consequently thread_export_context() can be called for interrupts, > traps and faults as well. Since syscalls are not implemented as > traps, we have two distinct paths into (and out from) the kernel. > > One (the syscall) is synchronous WRT to program execution and the > other (interrupts) is asynchronous. Synchronous contexts don't > have scratch registers in them. Asynchronous context need to have > them. This is not the hard problem: just add some flags to indicate > what parts of the context are valid and thus should be restored > and we're ok. > > The problem is when we preempt an interrupted thread, export the You mean when an interrupt thread preempts an SA thread? And this would be while it was in userland, not in the kernel? > context to the UTS and do an upcall. We end up having an async. > context in userland. I'm not sure at this time what we should do > with it. We have the following options: > > o Extend _ia64_restore_context() so that libkse can restore async > contexts. The downside is that it will very likely cause a > disabled high FP trap, which results in the process having the > high FP registers enabled. A performance hit. (see also below) Having to know about 2 different contexts isn't too new. Alpha may need the same thing since sigframes and trapframes are not the same (why?). > o Have _ia64_restore_context() call setcontext() for async contexts > and do the work in the kernel. Restoring the high FP will not > result in the enablement of the high FP registers, because we > can restore them to the PCB. They will be loaded into the CPU > when there's a need for them (which may be never). This can't really work without a special system call because you need to atomically set the thread mailbox pointer. If you want to add a syscall that does that, then that could be an option. Or, _ia64_restore_context() could be made to munge an async context so that it first returns to a function that can set the mailbox before returning to the interrupted context. The key is that once the mailbox pointer has been set, the threads context in the mailbox cannot be accessed again. A very slow method for doing that would be do use something like signalcontext() to copy the saved context out of the mailbox before setting the mailbox pointer and returning to it. Hmm, I guess you could just copy the context to a stack variable in _ia64_restore_context(): _ia64_restore_context(...) { if (context is synchronous) /* normal restore */ else { ucontext_t uc; uc.uc_sigmask = _thr_process_mask; bcopy(&ctx.uc_mcontext, &uc.uc_mcontext, sizeof(uc.uc_mcontext)); /* set mailbox pointer */ setcontext(&uc); } } If you chose this route, or something similar that needs a system call, you might optimize the kernel. You might only need to make an upcall when the KSE's quantum expired instead of whenever a thread was interrupted. Or perhaps when there are no unblocked threads in the kernel that require an upcall to notify the UTS, there's no need to make an upcall. In regards to the high FPU thing. Can't you tell the UTS via the exported context whether a high FPU restore is necessary? Just something that will let the restore avoid the trap into the kernel and getting tagged as being a high FPU process... -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 13:53:10 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4BC4937B401; Wed, 6 Aug 2003 13:53:10 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 267EF43F85; Wed, 6 Aug 2003 13:53:09 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76Kr8wO039937; Wed, 6 Aug 2003 13:53:08 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76Kr8G0001268; Wed, 6 Aug 2003 13:53:08 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h76Kr8pC001267; Wed, 6 Aug 2003 13:53:08 -0700 (PDT) (envelope-from marcel) Date: Wed, 6 Aug 2003 13:53:08 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030806205308.GA1179@athlon.pn.xcllnt.net> References: <20030806185530.GA893@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 20:53:10 -0000 On Wed, Aug 06, 2003 at 03:28:52PM -0400, Daniel Eischen wrote: > On Wed, 6 Aug 2003, Marcel Moolenaar wrote: > > > Gang, > > > > Given the panics that Daniel is having on pluto1 it's probably a > > good idea to fill people in on the status of KSE/ia64: > > First, thanks for your help on ia64. As I've found out, > there still seems to be a lot more that needs to get > done (in general, not KSE specific), and you seem to > be the only one working on it. Yes, unfortunately. The amount of work will not reduce for quite some time and it will only get worse before it gets better. > > The problem is when we preempt an interrupted thread, export the > > You mean when an interrupt thread preempts an SA thread? And > this would be while it was in userland, not in the kernel? I think so. I don't know exactly what the implications are for SA threads, but in general the problem is that we have a trap or interrupt while executing some thread and it results in an upcall. > > context to the UTS and do an upcall. We end up having an async. > > context in userland. I'm not sure at this time what we should do > > with it. We have the following options: > > > > o Extend _ia64_restore_context() so that libkse can restore async > > contexts. The downside is that it will very likely cause a > > disabled high FP trap, which results in the process having the > > high FP registers enabled. A performance hit. (see also below) > > Having to know about 2 different contexts isn't too new. Alpha > may need the same thing since sigframes and trapframes are not > the same (why?). But both are async frames on alpha. The only difference is the layout, not what's put in it. On ia64 the layout is always the same, but we don't populate everything all the time. > > o Have _ia64_restore_context() call setcontext() for async contexts > > and do the work in the kernel. Restoring the high FP will not > > result in the enablement of the high FP registers, because we > > can restore them to the PCB. They will be loaded into the CPU > > when there's a need for them (which may be never). > > This can't really work without a special system call because > you need to atomically set the thread mailbox pointer. Yes. That too. I had a brainwave: We still support the old syscall path, which is based on the break instruction. It's a trap. So, we can use the break instruction to trap into the kernel, executing the setcontext() syscall and go back to the interrupted context without any hackery. This at least resolves the problem of having 2 paths into the kernel: we take a trap to restore a context created by a trap. I guess I won't obsolete the old syscalls anymore :-) Alas, I forgot about the mailbox pointer... We don't have a syscall for that. I could probably put the info in the context itself, set a flag and enhance setcontext() to do it atomically as a possibly undocumented feature/extension to support KSE. Hmmm... > If you chose this route, or something similar that needs a > system call, you might optimize the kernel. You might > only need to make an upcall when the KSE's quantum expired > instead of whenever a thread was interrupted. Or perhaps > when there are no unblocked threads in the kernel that > require an upcall to notify the UTS, there's no need > to make an upcall. > > In regards to the high FPU thing. Can't you tell the > UTS via the exported context whether a high FPU restore > is necessary? It's always necessary. Even if the high FP wasn't enabled at the time of the interrupt, it doesn't mean that the high FP registers don't have valid values. It's just that they haven't been accessed in that timeslice yet (we need some heuristics to enable the high FP by default for that process until after some timeout -- that avoids a trap every time slice). Also, another thread may use the high FP registers and since user level threading is indistinguishable for the kernel, we need to either have synchronous context switches or save the whole lot for async context switches. I think I'll go with the following (for async contexts): o Use ifa and isr in the context to pass the pointer to the mailbox and the value we want to write to it. Both are only used by the kernel to pass information about the fault or interrupt to the kernel, so they are unused fields in contexts. o Set a flag in the context to indicate that we want the mailbox set (if applicable). o Use the break-based syscall path to call setcontext(2). o Have the kernel switch the context and set the mailbox. It's a bit of a hack, but I think other solutions are much more painful. Both implementation-wise and kludge-wise. -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 14:12:41 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id C1ABC37B401 for ; Wed, 6 Aug 2003 14:12:41 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 255F743F3F for ; Wed, 6 Aug 2003 14:12:41 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h76LCeuN020117; Wed, 6 Aug 2003 17:12:40 -0400 (EDT) Date: Wed, 6 Aug 2003 17:12:40 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Marcel Moolenaar In-Reply-To: <20030806205308.GA1179@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 21:12:42 -0000 On Wed, 6 Aug 2003, Marcel Moolenaar wrote: > On Wed, Aug 06, 2003 at 03:28:52PM -0400, Daniel Eischen wrote: > > > If you chose this route, or something similar that needs a > > system call, you might optimize the kernel. You might > > only need to make an upcall when the KSE's quantum expired > > instead of whenever a thread was interrupted. Or perhaps > > when there are no unblocked threads in the kernel that > > require an upcall to notify the UTS, there's no need > > to make an upcall. > > > > In regards to the high FPU thing. Can't you tell the > > UTS via the exported context whether a high FPU restore > > is necessary? > > It's always necessary. Even if the high FP wasn't enabled > at the time of the interrupt, it doesn't mean that the high > FP registers don't have valid values. It's just that they > haven't been accessed in that timeslice yet (we need some > heuristics to enable the high FP by default for that process > until after some timeout -- that avoids a trap every time > slice). I see. FWIW, we don't have a way to eliminate the trap on x86 either. Once the context is marked with the FPU registers being from the FPU itself or the PCB, we always restore them. The thing to realize I guess is that once you trap, you won't do it again until the FPU has been taken away again. So you may only trap the first time, and a few other times following that you may not. You can have to weigh that against always using a system call to restore the context. I'm not sure what the difference would be. > Also, another thread may use the high FP registers and since > user level threading is indistinguishable for the kernel, we > need to either have synchronous context switches or save the > whole lot for async context switches. > > I think I'll go with the following (for async contexts): > > o Use ifa and isr in the context to pass the pointer to the > mailbox and the value we want to write to it. Both are > only used by the kernel to pass information about the > fault or interrupt to the kernel, so they are unused > fields in contexts. > o Set a flag in the context to indicate that we want the > mailbox set (if applicable). > o Use the break-based syscall path to call setcontext(2). > o Have the kernel switch the context and set the mailbox. > > It's a bit of a hack, but I think other solutions are much > more painful. Both implementation-wise and kludge-wise. If there's anything more we can do, let us know. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 14:25:49 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5DE0637B401; Wed, 6 Aug 2003 14:25:49 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id 6B20A43F75; Wed, 6 Aug 2003 14:25:48 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76LPmwO040110; Wed, 6 Aug 2003 14:25:48 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76LPmG0001385; Wed, 6 Aug 2003 14:25:48 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h76LPmEU001384; Wed, 6 Aug 2003 14:25:48 -0700 (PDT) (envelope-from marcel) Date: Wed, 6 Aug 2003 14:25:47 -0700 From: Marcel Moolenaar To: deischen@freebsd.org Message-ID: <20030806212547.GC1179@athlon.pn.xcllnt.net> References: <20030806205308.GA1179@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 21:25:49 -0000 On Wed, Aug 06, 2003 at 05:12:40PM -0400, Daniel Eischen wrote: > > The thing to realize I guess is that once you trap, > you won't do it again until the FPU has been taken > away again. So you may only trap the first time, > and a few other times following that you may not. > You can have to weigh that against always using a > system call to restore the context. I'm not sure > what the difference would be. I haven't done any performance measurements. What I do know is that lazy switching of the high FP is a definite win, because they are hardly ever used. Once the ia64 port reaches some level of maturity I intend to use 1:1 threading to simulate multiple processes using the high FP registers and analyze how it behaves (including correctness). Based on that I probably want to make changes. > > I think I'll go with the following (for async contexts): > > > > o Use ifa and isr in the context to pass the pointer to the > > mailbox and the value we want to write to it. Both are > > only used by the kernel to pass information about the > > fault or interrupt to the kernel, so they are unused > > fields in contexts. > > o Set a flag in the context to indicate that we want the > > mailbox set (if applicable). > > o Use the break-based syscall path to call setcontext(2). > > o Have the kernel switch the context and set the mailbox. > > > > It's a bit of a hack, but I think other solutions are much > > more painful. Both implementation-wise and kludge-wise. > > If there's anything more we can do, let us know. With the MD bits in libkse revamped I don't have much to wish for. It would have taken us (me) a lot more work and time if we (I) had to do that at some later time. It looks like the async contexts are the only thing left and there's no revamp of KSE that can make that go away, so I guess I just have to solve it :-) I'll let you know when ACE fails to panic the kernel... -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 14:42:49 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5305937B401; Wed, 6 Aug 2003 14:42:49 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id A2D3143F3F; Wed, 6 Aug 2003 14:42:48 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <20030806214247015009dqj5e>; Wed, 6 Aug 2003 21:42:48 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id OAA06531; Wed, 6 Aug 2003 14:42:46 -0700 (PDT) Date: Wed, 6 Aug 2003 14:42:45 -0700 (PDT) From: Julian Elischer To: Marcel Moolenaar In-Reply-To: <20030806205308.GA1179@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 21:42:49 -0000 On Wed, 6 Aug 2003, Marcel Moolenaar wrote: > On Wed, Aug 06, 2003 at 03:28:52PM -0400, Daniel Eischen wrote: > > On Wed, 6 Aug 2003, Marcel Moolenaar wrote: > > > > > Gang, > > > > > > Given the panics that Daniel is having on pluto1 it's probably a > > > good idea to fill people in on the status of KSE/ia64: > > > > First, thanks for your help on ia64. As I've found out, > > there still seems to be a lot more that needs to get > > done (in general, not KSE specific), and you seem to > > be the only one working on it. > > Yes, unfortunately. The amount of work will not reduce for > quite some time and it will only get worse before it gets > better. > > > > The problem is when we preempt an interrupted thread, export the > > > > You mean when an interrupt thread preempts an SA thread? And > > this would be while it was in userland, not in the kernel? > > I think so. I don't know exactly what the implications are for > SA threads, but in general the problem is that we have a trap > or interrupt while executing some thread and it results in an > upcall. Originally we would not do an upcall unless the kernel was neterred from a syscall, however David added soem code so that at a clock interrupt, if the mailbox indicates that it has had enough time, An UPCALL context is belatedly made and saved and an upcall results.. > > > > context to the UTS and do an upcall. We end up having an async. > > > context in userland. I'm not sure at this time what we should do > > > with it. We have the following options: > > > > > > o Extend _ia64_restore_context() so that libkse can restore async > > > contexts. The downside is that it will very likely cause a > > > disabled high FP trap, which results in the process having the > > > high FP registers enabled. A performance hit. (see also below) > > > > Having to know about 2 different contexts isn't too new. Alpha > > may need the same thing since sigframes and trapframes are not > > the same (why?). > > But both are async frames on alpha. The only difference is the > layout, not what's put in it. On ia64 the layout is always the > same, but we don't populate everything all the time. So how does a returning thread know what to restore? > > > > o Have _ia64_restore_context() call setcontext() for async contexts > > > and do the work in the kernel. Restoring the high FP will not > > > result in the enablement of the high FP registers, because we > > > can restore them to the PCB. They will be loaded into the CPU > > > when there's a need for them (which may be never). > > > > This can't really work without a special system call because > > you need to atomically set the thread mailbox pointer. > > Yes. That too. I had a brainwave: We still support the old > syscall path, which is based on the break instruction. It's > a trap. So, we can use the break instruction to trap into > the kernel, executing the setcontext() syscall and go back > to the interrupted context without any hackery. This at > least resolves the problem of having 2 paths into the > kernel: we take a trap to restore a context created by > a trap. I guess I won't obsolete the old syscalls anymore :-) > > Alas, I forgot about the mailbox pointer... We don't have a > syscall for that. I could probably put the info in the context > itself, set a flag and enhance setcontext() to do it atomically > as a possibly undocumented feature/extension to support KSE. > Hmmm... you're somewhere near here now aren't you? (I think you said san Mateo but I could be wrong) Maybe we could get to gether some time and walk through this.. A whiteboard is always easier to understand. > > > If you chose this route, or something similar that needs a > > system call, you might optimize the kernel. You might > > only need to make an upcall when the KSE's quantum expired > > instead of whenever a thread was interrupted. Or perhaps > > when there are no unblocked threads in the kernel that > > require an upcall to notify the UTS, there's no need > > to make an upcall. > > > > In regards to the high FPU thing. Can't you tell the > > UTS via the exported context whether a high FPU restore > > is necessary? > > It's always necessary. Even if the high FP wasn't enabled > at the time of the interrupt, it doesn't mean that the high > FP registers don't have valid values. It's just that they > haven't been accessed in that timeslice yet (we need some > heuristics to enable the high FP by default for that process > until after some timeout -- that avoids a trap every time > slice). > Also, another thread may use the high FP registers and since > user level threading is indistinguishable for the kernel, we > need to either have synchronous context switches or save the > whole lot for async context switches. > > I think I'll go with the following (for async contexts): > > o Use ifa and isr in the context to pass the pointer to the > mailbox and the value we want to write to it. Both are > only used by the kernel to pass information about the > fault or interrupt to the kernel, so they are unused > fields in contexts. > o Set a flag in the context to indicate that we want the > mailbox set (if applicable). > o Use the break-based syscall path to call setcontext(2). > o Have the kernel switch the context and set the mailbox. > > It's a bit of a hack, but I think other solutions are much > more painful. Both implementation-wise and kludge-wise. > > -- > Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 15:04:56 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6F01037B401; Wed, 6 Aug 2003 15:04:56 -0700 (PDT) Received: from ns1.xcllnt.net (209-128-86-226.BAYAREA.NET [209.128.86.226]) by mx1.FreeBSD.org (Postfix) with ESMTP id C435943F75; Wed, 6 Aug 2003 15:04:54 -0700 (PDT) (envelope-from marcel@xcllnt.net) Received: from athlon.pn.xcllnt.net (athlon.pn.xcllnt.net [192.168.4.3]) by ns1.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76M4LwO040288; Wed, 6 Aug 2003 15:04:21 -0700 (PDT) (envelope-from marcel@piii.pn.xcllnt.net) Received: from athlon.pn.xcllnt.net (localhost [127.0.0.1]) by athlon.pn.xcllnt.net (8.12.9/8.12.9) with ESMTP id h76M4LG0001499; Wed, 6 Aug 2003 15:04:21 -0700 (PDT) (envelope-from marcel@athlon.pn.xcllnt.net) Received: (from marcel@localhost) by athlon.pn.xcllnt.net (8.12.9/8.12.9/Submit) id h76M4Ll4001498; Wed, 6 Aug 2003 15:04:21 -0700 (PDT) (envelope-from marcel) Date: Wed, 6 Aug 2003 15:04:21 -0700 From: Marcel Moolenaar To: Julian Elischer Message-ID: <20030806220421.GA1404@athlon.pn.xcllnt.net> References: <20030806205308.GA1179@athlon.pn.xcllnt.net> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.4i cc: deischen@freebsd.org cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 06 Aug 2003 22:04:56 -0000 On Wed, Aug 06, 2003 at 02:42:45PM -0700, Julian Elischer wrote: > > Originally we would not do an upcall unless the > kernel was neterred from a syscall, however David added soem code so > that at a clock interrupt, if the mailbox indicates that it has had > enough time, An UPCALL context is belatedly made and saved and an upcall > results.. Ah, so I have David to thank for the added complexity... :-) > > But both are async frames on alpha. The only difference is the > > layout, not what's put in it. On ia64 the layout is always the > > same, but we don't populate everything all the time. > > So how does a returning thread know what to restore? So far it has always been a sync context, so all that needed to be restored were the special and the preserved registers. I added the return registers just recently. The sync contexts don't have any optional fields. It's the async contexts that make things complicated. I just have to add flags and/or actually use the flags that have been defined already. > > Yes. That too. I had a brainwave: We still support the old > > syscall path, which is based on the break instruction. It's > > a trap. So, we can use the break instruction to trap into > > the kernel, executing the setcontext() syscall and go back > > to the interrupted context without any hackery. This at > > least resolves the problem of having 2 paths into the > > kernel: we take a trap to restore a context created by > > a trap. I guess I won't obsolete the old syscalls anymore :-) > > > > Alas, I forgot about the mailbox pointer... We don't have a > > syscall for that. I could probably put the info in the context > > itself, set a flag and enhance setcontext() to do it atomically > > as a possibly undocumented feature/extension to support KSE. > > Hmmm... > > you're somewhere near here now aren't you? > (I think you said san Mateo but I could be wrong) I live in Mountain View. San Mateo relates to BSDcon. > Maybe we could get to gether some time and walk through > this.. A whiteboard is always easier to understand. That's probably not a bad idea. I'll this off-list, unless it's beneficial for others as well and we can meet with one or two more. -- Marcel Moolenaar USPA: A-39004 marcel@xcllnt.net From owner-freebsd-threads@FreeBSD.ORG Wed Aug 6 17:45:55 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 65DB037B401; Wed, 6 Aug 2003 17:45:55 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id B398E43F85; Wed, 6 Aug 2003 17:45:54 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc11) with ESMTP id <2003080700455301300i0530e>; Thu, 7 Aug 2003 00:45:53 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA08114; Wed, 6 Aug 2003 17:45:48 -0700 (PDT) Date: Wed, 6 Aug 2003 17:45:47 -0700 (PDT) From: Julian Elischer To: Marcel Moolenaar In-Reply-To: <20030806220421.GA1404@athlon.pn.xcllnt.net> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: threads@freebsd.org Subject: Re: KSE/ia64: a quick update X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 07 Aug 2003 00:45:55 -0000 On Wed, 6 Aug 2003, Marcel Moolenaar wrote: > On Wed, Aug 06, 2003 at 02:42:45PM -0700, Julian Elischer wrote: > > > > Originally we would not do an upcall unless the > > kernel was neterred from a syscall, however David added soem code so > > that at a clock interrupt, if the mailbox indicates that it has had > > enough time, An UPCALL context is belatedly made and saved and an upcall > > results.. > > Ah, so I have David to thank for the added complexity... :-) not really.. we all discussed it.. From owner-freebsd-threads@FreeBSD.ORG Fri Aug 8 12:13:10 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 510F637B401; Fri, 8 Aug 2003 12:13:10 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 5BA7943F93; Fri, 8 Aug 2003 12:13:09 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h78JD8uN000731; Fri, 8 Aug 2003 15:13:08 -0400 (EDT) Date: Fri, 8 Aug 2003 15:13:08 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: davidxu@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: mutex_d again X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2003 19:13:10 -0000 On Fri, 8 Aug 2003, Daniel Eischen wrote: > On Fri, 8 Aug 2003, Daniel Eischen wrote: > > > It seems mutex_d is failing again in the priority inheritence tests. > > I'll try it on my system at work just to make sure it's not my kernel > > (newly built) at home that is screwing me somehow. > > It seems to be working here at work on a 3-day old > kernel. I am updating the kernel right now to see > if it works with a new kernel. Yes, it now fails on a new kernel cvsup'd from 2 hours ago. Something in the kernel is busted, and within the last 2 or 3 days. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Aug 8 13:20:11 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id A116E37B401; Fri, 8 Aug 2003 13:20:11 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 01EBC43F3F; Fri, 8 Aug 2003 13:20:11 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h78KKAuN012184; Fri, 8 Aug 2003 16:20:10 -0400 (EDT) Date: Fri, 8 Aug 2003 16:20:10 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: davidxu@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org Subject: Re: mutex_d again X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 08 Aug 2003 20:20:12 -0000 On Fri, 8 Aug 2003, Daniel Eischen wrote: > On Fri, 8 Aug 2003, Daniel Eischen wrote: > > > On Fri, 8 Aug 2003, Daniel Eischen wrote: > > > > > It seems mutex_d is failing again in the priority inheritence tests. > > > I'll try it on my system at work just to make sure it's not my kernel > > > (newly built) at home that is screwing me somehow. > > > > It seems to be working here at work on a 3-day old > > kernel. I am updating the kernel right now to see > > if it works with a new kernel. > > Yes, it now fails on a new kernel cvsup'd from 2 hours > ago. Something in the kernel is busted, and within > the last 2 or 3 days. Nevermind, found it. -- Dan Eischen Index: kern/kern_thread.c =================================================================== RCS file: /opt/FreeBSD/cvs/src/sys/kern/kern_thread.c,v retrieving revision 1.156 diff -u -r1.156 kern_thread.c --- kern/kern_thread.c 7 Aug 2003 15:04:26 -0000 1.156 +++ kern/kern_thread.c 8 Aug 2003 20:02:32 -0000 @@ -1601,7 +1601,7 @@ } else { if (td->td_standin == NULL) thread_alloc_spare(td, NULL); - tflags = fuword32(tmbx); + tflags = fuword32(&tmbx->tm_flags); /* * On some architectures, TP register points to thread * mailbox but not points to kse mailbox, and userland From owner-freebsd-threads@FreeBSD.ORG Sat Aug 9 10:46:36 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 7687937B401; Sat, 9 Aug 2003 10:46:36 -0700 (PDT) Received: from sccimhc02.asp.att.net (sccimhc02.asp.att.net [63.240.76.164]) by mx1.FreeBSD.org (Postfix) with ESMTP id B307A43FDF; Sat, 9 Aug 2003 10:46:35 -0700 (PDT) (envelope-from pwillia@insightbb.com) Received: from warthog (12-222-105-121.client.insightbb.com[12.222.105.121]) by sccimhc02.asp.att.net (sccimhc02) with SMTP id <20030809174634im200b4huge>; Sat, 9 Aug 2003 17:46:35 +0000 From: "Paul Williams" To: , Date: Sat, 9 Aug 2003 12:46:32 -0500 Message-ID: <000501c35e9e$2bcacaa0$6801a8c0@insightbb.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: quoted-printable X-Priority: 3 (Normal) X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook, Build 10.0.4510 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Importance: Normal Subject: Processor affinity and KSE sample code X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2003 17:46:36 -0000 Hello, I'm trying to pin a group of processes to a particular processor in = FreeBSD 5.1-CURRENT. I've seen some discussion on older mailing lists about = KSE's possibly supporting this, but I am not seeing how to do so using the = current interface. Is this possible now or planned for future work? Also, does someone have examples of how to use KSEs? The test code included with = the kernel isn't too enlightening (at least not for a KSE novice). Thank you, Paul Williams From owner-freebsd-threads@FreeBSD.ORG Sat Aug 9 11:22:07 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 596C137B401; Sat, 9 Aug 2003 11:22:07 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id A8E8B43FE9; Sat, 9 Aug 2003 11:22:06 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h79IM5uN014603; Sat, 9 Aug 2003 14:22:06 -0400 (EDT) Date: Sat, 9 Aug 2003 14:22:05 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Paul Williams In-Reply-To: <000501c35e9e$2bcacaa0$6801a8c0@insightbb.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: freebsd-smp@freebsd.org cc: freebsd-threads@freebsd.org Subject: Re: Processor affinity and KSE sample code X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 09 Aug 2003 18:22:07 -0000 On Sat, 9 Aug 2003, Paul Williams wrote: > Hello, > > I'm trying to pin a group of processes to a particular processor in FreeBSD > 5.1-CURRENT. I've seen some discussion on older mailing lists about KSE's > possibly supporting this, but I am not seeing how to do so using the current > interface. Is this possible now or planned for future work? Also, does It isn't implemented yet. I had always wanted to implement a kse_bind() though. > someone have examples of how to use KSEs? The test code included with the > kernel isn't too enlightening (at least not for a KSE novice). It's much easier just to use libpthread^Wlibkse and threads. Scope system threads get their own KSE, while scope process threads all run on a set of N kses (where N = kern.threads.virtual_cpu). -- Dan Eischen