From owner-freebsd-threads@FreeBSD.ORG Mon Jul 21 11:03:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 906E237B401 for ; Mon, 21 Jul 2003 11:03:00 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1882543FB1 for ; Mon, 21 Jul 2003 11:03:00 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LI2xUp059427 for ; Mon, 21 Jul 2003 11:02:59 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LI2x5m059422 for threads@freebsd.org; Mon, 21 Jul 2003 11:02:59 -0700 (PDT) Date: Mon, 21 Jul 2003 11:02:59 -0700 (PDT) Message-Id: <200307211802.h6LI2x5m059422@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 18:03:00 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin 1 problem total. Non-critical problems From owner-freebsd-threads@FreeBSD.ORG Mon Jul 21 14:35:54 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1BBBE37B407 for ; Mon, 21 Jul 2003 14:35:54 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3096F43FA3 for ; Mon, 21 Jul 2003 14:35:49 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLZmUp004086 for ; Mon, 21 Jul 2003 14:35:48 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLZmjp004080 for freebsd-threads@freebsd.org; Mon, 21 Jul 2003 14:35:48 -0700 (PDT) Date: Mon, 21 Jul 2003 14:35:48 -0700 (PDT) Message-Id: <200307212135.h6LLZmjp004080@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:35:54 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2002/01/16] kern/33951 threads pthread_cancel is ignored 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/01/19] bin/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] bin/24632 threads libc_r delicate deviation from libc in ha o [2001/04/02] bin/26307 threads libc_r aborts when using the KDE media pl o [2001/10/31] bin/31661 threads pthread_kill signal handler doesn't get s o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/03/07] bin/35622 threads sigaltstack is missing in libc_r o [2002/06/27] bin/39922 threads [PATCH?] Threaded applications executed w o [2003/03/02] bin/48856 threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] bin/49087 threads Signals lost in programs linked with libc o [2003/05/07] bin/51949 threads thread in accept cannot be cancelled o [2002/02/01] i386/34536 threads accept() blocks other threads o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/10/10] kern/43887 threads abnormal CPU useage when use pthread_mute o [2003/05/30] kern/52817 threads top(1) shows garbage for threaded process o [2000/08/26] misc/20861 threads libc_r does not honor socket timeouts o [2001/01/25] misc/24641 threads pthread_rwlock_rdlock can deadlock o [2002/08/04] misc/41331 threads Pthread library open sets O_NONBLOCK flag 18 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/09/09] bin/30464 threads pthread mutex attributes -- pshared o [2002/05/02] bin/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwri o [2000/05/25] misc/18824 threads gethostbyname is not thread safe o [2000/10/21] misc/22190 threads A threaded read(2) from a socketpair(2) f o [2002/07/16] misc/40671 threads pthread_cancel doesn't remove thread from 5 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Jul 21 14:37:29 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id B353337B401 for ; Mon, 21 Jul 2003 14:37:29 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C122943FD7 for ; Mon, 21 Jul 2003 14:37:26 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLbQUp005705 for ; Mon, 21 Jul 2003 14:37:26 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLbQvV005699 for threads@freebsd.org; Mon, 21 Jul 2003 14:37:26 -0700 (PDT) Date: Mon, 21 Jul 2003 14:37:26 -0700 (PDT) Message-Id: <200307212137.h6LLbQvV005699@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:37:30 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin 1 problem total. Non-critical problems From owner-freebsd-threads@FreeBSD.ORG Mon Jul 21 14:46:01 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 5610F37B401 for ; Mon, 21 Jul 2003 14:46:01 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id C502343FBD for ; Mon, 21 Jul 2003 14:46:00 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLk0Up008735 for ; Mon, 21 Jul 2003 14:46:00 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLk03M008729 for freebsd-threads@freebsd.org; Mon, 21 Jul 2003 14:46:00 -0700 (PDT) Date: Mon, 21 Jul 2003 14:46:00 -0700 (PDT) Message-Id: <200307212146.h6LLk03M008729@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: freebsd-threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:46:01 -0000 Current FreeBSD problem reports Critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2000/06/13] kern/19247 threads uthread_sigaction.c does not do anything o [2002/01/16] kern/33951 threads pthread_cancel is ignored 2 problems total. Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/01/19] bin/24472 threads libc_r does not honor SO_SNDTIMEO/SO_RCVT o [2001/01/25] bin/24632 threads libc_r delicate deviation from libc in ha o [2001/04/02] bin/26307 threads libc_r aborts when using the KDE media pl o [2001/10/31] bin/31661 threads pthread_kill signal handler doesn't get s o [2001/11/26] bin/32295 threads pthread dont dequeue signals o [2002/03/07] bin/35622 threads sigaltstack is missing in libc_r o [2002/06/27] bin/39922 threads [PATCH?] Threaded applications executed w o [2003/03/02] bin/48856 threads Setting SIGCHLD to SIG_IGN still leaves z o [2003/03/10] bin/49087 threads Signals lost in programs linked with libc o [2003/05/07] bin/51949 threads thread in accept cannot be cancelled o [2002/02/01] i386/34536 threads accept() blocks other threads o [2000/07/18] kern/20016 threads pthreads: Cannot set scheduling timer/Can o [2002/05/25] kern/38549 threads the procces compiled whith pthread stoppe o [2002/10/10] kern/43887 threads abnormal CPU useage when use pthread_mute o [2003/05/30] kern/52817 threads top(1) shows garbage for threaded process o [2000/08/26] misc/20861 threads libc_r does not honor socket timeouts o [2001/01/25] misc/24641 threads pthread_rwlock_rdlock can deadlock o [2002/08/04] misc/41331 threads Pthread library open sets O_NONBLOCK flag 18 problems total. Non-critical problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- o [2001/09/09] bin/30464 threads pthread mutex attributes -- pshared o [2002/05/02] bin/37676 threads libc_r: msgsnd(), msgrcv(), pread(), pwri o [2000/05/25] misc/18824 threads gethostbyname is not thread safe o [2000/10/21] misc/22190 threads A threaded read(2) from a socketpair(2) f o [2002/07/16] misc/40671 threads pthread_cancel doesn't remove thread from 5 problems total. From owner-freebsd-threads@FreeBSD.ORG Mon Jul 21 14:47:47 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 6D78D37B425 for ; Mon, 21 Jul 2003 14:47:47 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0800A43FD7 for ; Mon, 21 Jul 2003 14:47:43 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: from freefall.freebsd.org (peter@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6LLlhUp010809 for ; Mon, 21 Jul 2003 14:47:43 -0700 (PDT) (envelope-from owner-bugmaster@freebsd.org) Received: (from peter@localhost) by freefall.freebsd.org (8.12.9/8.12.9/Submit) id h6LLlhbV010803 for threads@freebsd.org; Mon, 21 Jul 2003 14:47:43 -0700 (PDT) Date: Mon, 21 Jul 2003 14:47:43 -0700 (PDT) Message-Id: <200307212147.h6LLlhbV010803@freefall.freebsd.org> X-Authentication-Warning: freefall.freebsd.org: peter set sender to owner-bugmaster@freebsd.org using -f From: FreeBSD bugmaster To: threads@FreeBSD.org Subject: Current problem reports assigned to you X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 21 Jul 2003 21:47:47 -0000 Current FreeBSD problem reports Critical problems Serious problems S Submitted Tracker Resp. Description ------------------------------------------------------------------------------- a [2003/04/08] bin/50733 threads buildworld won't build, because of linkin 1 problem total. Non-critical problems From owner-freebsd-threads@FreeBSD.ORG Wed Jul 23 13:16:14 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DC9CD37B405 for ; Wed, 23 Jul 2003 13:16:13 -0700 (PDT) Received: from rwcrmhc12.comcast.net (rwcrmhc12.comcast.net [216.148.227.85]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0D86043FB1 for ; Wed, 23 Jul 2003 13:16:13 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc12) with ESMTP id <2003072320161201400anvdse>; Wed, 23 Jul 2003 20:16:12 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA60627; Wed, 23 Jul 2003 13:16:10 -0700 (PDT) Date: Wed, 23 Jul 2003 13:16:08 -0700 (PDT) From: Julian Elischer To: threads@freebsd.org Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: Julian Elischer Subject: Porting KSE to a new architecture.. kernel part X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 Jul 2003 20:16:14 -0000 David, Dan, Marcel, peter Here is my first hack at a small doc to help $ARCH developers enable KSE threading on their architecture. comments? additions? There are several small functions that need to be rewritten to allow KSE threading to run on a new architecture. Firstly it is important to understand the extra interractions that KSE threading has between the userland and the kernel. When a thread blocks in the kernel, the (virtual) cpu is reallocated to another newly created context (thread) and it is allowed to proceed back to userland and runs a known routine with a known pointer as its argument. This is called an "upcall" and there need to be machine specific functions to set it up. The blocking thread is allowed to continue at a later time but since the userland has already been returned to, It must not return there. Instead, when it reaches the user boundary, its context is stored back into the userland thread storage for that thread, in exactly the same way that it would look as if it had returned from the kernel, and then called 'yield'. The format of the saved context is "struct mcontext" and it should already be defined for each architecture. **************** Implementation ****************** ## The UPCALL ## There are two Machine independent functions that do the upcall, (It's done in two stages) and each calls a machine dependent part to do such things as may need to be done for each architecture. The machine INdependent functions are: thread_schedule_upcall() and thread_userret() Thread_schedule_upcall() calls cpu_set_upcall() and thread_userret() calls cpu_set_upcall_kse() Thread_schedule_upcall() sets up a new 'unused' Thread and requires that cpu_set_upcall() set up its context so that that thread, when run will return towards the user boundary. It does this in almost the same manner that fork() sets up the child thread to do the same thing. As a result, cpu_set_upcall() is almost the same in logic as cpu_fork(). It even sets up the context so that the next code run by the new thread is in fact fork_trampoline() and the (new) thread proceeds towards the userland boundary pretty much believing that it has just done a fork(). The main difference is that the flag TDP_UPCALLING is set in the thread's flags. In i386 this is about 14 lines of C and in ia64 it is about 18 lines of C. When the thread reaches the user boundary it runs userret() which in turn calls thread_userret() as mentionned above. Thread_userret() notices that TDP_UPCALLING is set and take special actions before allowing it to proceed to userland. One of these actions is to load the userland context with a PC (instruction pointer) for the upcall function for that particular (virtual) cpu, as well as setting the stack pointer to the stack associated with it. It also fiddles the arguments on the stack that the upcall function will see, setting the (single) argument to the address of that Virtual CPU's mailbox. This is all done by the function cpu_set_upcall_kse(). In i386 it is 4 lines of C and in ia64 it is about 14 lines of C. ## The context export to userland ** The blocking thread needs to be able to store its context back to userland when it has completed its kernel task, and an Machine dependent function is required for that. The basic userret() calls thread_userret() and that in turn calls thread_export_context(). Thread_export_context() in turn uses the machiine dependent function get_mcontext to extract the saved context from the thread's stack (or wherever it is stored) and write it in the proscribed format (struct mcontext), which is then written out to the thread context storage area in userland. It is possible that this may be rewritten to avoid the intermadiate copy of the mcontext in kernel space and write it directly to userland, however this is not what is done at this time. get_mcontext() has code that is much like that used to write signals to userland and can largly be cribbed (in loginc if not in code) from there (sendsig()). ## Other ## KSE threads also need some small assistance with signals. a machine dependent function cpu_thread_siginfo() is needed to format the information being sent to the userland scheduler into the correct format for that architecture. From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 13:59:34 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE59537B40A; Thu, 24 Jul 2003 13:59:34 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id F358643F3F; Thu, 24 Jul 2003 13:59:33 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc11) with ESMTP id <2003072420593201300n892he>; Thu, 24 Jul 2003 20:59:32 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id NAA70280; Thu, 24 Jul 2003 13:59:30 -0700 (PDT) Date: Thu, 24 Jul 2003 13:59:28 -0700 (PDT) From: Julian Elischer To: deischen@freebsd.org In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: David Xu cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 20:59:35 -0000 On Thu, 24 Jul 2003, Daniel Eischen wrote: > > > > /* > > * Thread mailbox. > > * > > * This describes a user thread to the kernel scheduler. > > */ > > struct kse_thr_mailbox { > > uint32_t tm_flags; /* Thread flags */ > > struct kse_thr_mailbox *tm_next; /* Next thread in list */ > > void *tm_udata; /* For use by the UTS */ > > uint32_t tm_uticks; > > uint32_t tm_sticks; > > register_t tm_spare[4]; > > ucontext_t tm_context; /* User and machine context */ > > siginfo_t tm_syncsig; > > }; > > We need to allocate a few spare ptrs for TLS at the beginning. > I think tm_context should be at the end since it is the thing > most likely to change. tm_context will be aligned because > it contains an alignment clause for the FPU state, at least > for i386, and should for other archs if it is necessary. > We also need some way to tell if kse_thr_mailbox is a fake > mailbox or not. We could either use a version/type field > or a second tm_udata. No this is not quite right.. (CC'd to threads so that we have a record of this...) The TLS spec specifies a few things.. on x86, the memory segment defined by the LDT entry referenced by %gs must have at offset 0, the address of the thread control block (TCB), and at some offset in the TCB is a pointer to the Dynamic Thread Vector (dtv). The pointer is at offset 'dtpoff' (a symbol) from the base of the Thread Control block. "Static" (as in defined in modules linked statically with the main module) __thread variables are defined to be contiguously allocated BEFORE (i.e. at -ve offsets) the Thread control block.. The DTV contains pointers to subsections of the tls storage (even to noncontiguous parts) needed by the linker and to resolve addresses at runtime. in pictures: [LDT 1] (%gs=X*8) [LDT 2] ... [LDT X-1] [%gs]----(index)--->[LDT X]------+ [LDT X+1] | | +---------------+ V -------------------------------------------------------------- | | | | | | | | | | | | | | | Z -------------------------------------------------------------- | | XXX THIS AREA NOT DEFINED | Z segment 'g' ------------------------------------- | | +---------------+ | V --------------------------------------------------//------------ Z | | | | | | | | | | | | | | Z --------------------------------------------------//------------ "statically linked" TLS storage # TCB contents Z TCB --------------------------------------------------//------------ ^ <--- dtpoff-->| | | | +---------------+ | | +-------------------)------+ | | V | --------------------------------------------------//------------ Z | | | | | | | | | DTV entries | Z --------------------------------------------------//------------ | | +---------------------------------------+ | V --------------------------------------------------//------------ Z | | | NON-Contiguous (dynamically linked TLS storage)| --------------------------------------------------//------------ Under the KSE library, the area marked "XXX" is the KSE MAILBOX Under libthr it is part of an array of thread pointers. (i.e this works correctly for both libraries) On ia64 it is slightly differnt.. Ther eis an actual tp register (I believe) TP | V --------------------------------------------------//------------ Z | | | | | | | | | | | | | | Z --------------------------------------------------//------------ | TCB contents (*) | |XXX|"statically linked" TLS storage --------------------------------------------------//------------ | | ^ | | | V +---------------+ | [KSE mbox] | | | +--+ | | V | --------------------------------------------------//------------ Z | | | | | | | | | DTV entries Z --------------------------------------------------//------------ | | +---------------------------------------+ | V --------------------------------------------------//------------ Z | | | NON-Contiguous (dynamically linked TLS storage)| --------------------------------------------------//------------ Under the KSE library, the area marked "XXX" is undefined and could be either a pointer to the KSE mailbox or the TCB. (which would contain a pointer to the KSE mailbox). In either case the pointer to the KSE mailbox would be set by the UTS when sceduling that thread. it is important to note that the kernel doesn't need or read this pointer as it already knows where the KSE mailbox is, and the UTS upcall doesn't need it as it gets the mailbox address as an argument when upcalled. It is only required by the UTS in the case that it is enterred from the thread directly. (*) The TCB could be anywhere, and the XXX area could be used to point to it, however it is probably more efficient to allocate the (static) TLS immediatly following the TCB and do it in a single allocation, and use -ve offsets from the TP to get to it. (i.e. the TP points between the TCB and the (static) TLS. ON non-i386, and non ia64 systems there is a hybrid. as follows TP | +--------------------+ | V --------------------------------------------------//------------ Z | | | | | | | | | | | | | | Z --------------------------------------------------//------------ "statically linked" TLS storage # TCB contents Z TCB --------------------------------------------------//------------ ^ <--- dtpoff-->| | | | | | +---------------+ | | | V +-------------------)------+ [KSE mbox] | | V | --------------------------------------------------//------------ Z | | | | | | | | | DTV entries Z --------------------------------------------------//------------ | | +---------------------------------------+ | V --------------------------------------------------//------------ Z | | | NON-Contiguous (dynamically linked TLS storage)| --------------------------------------------------//------------ > > struct kse_thr_mailbox { > void *tm_tls[4]; /* reserved for TLS */ > uint32_t tm_flags; > uint32_t tm_version; > struct kse_thr_mailbox *tm_next; > void *tm_udata; > uint32_t tm_uticks; > uint32_t tm_sticks; > register_t tm_spare[4]; > siginfo_t tm_syncsig; > ucontext_t tm_context; > }; looking at the above diagrams, we see: struct kse_thr_mailbox need not have any TLS stuff. The offset to the DTV pointer is relative to whatever the TP is pointing to, which MAY be the mailbox or it may be the larger TCB (which might contian the thread mailbox but may not) In anycase there is no reason for the mailbox to contain a TLS entry becauee the TLS is not something that we are communicating to the kernel. In the i386 case, (and only the i386 case (unless the amd-64 case is the same)) the KSE MAILBOX is what we are pointing %gs:0 at, and in that case, the pointer to the TCB (not the thread mailbox) is stored there. (and set there by the UTS when scheduling a thread). Thus the struct kse_mailbox would have: struct kse_mailbox { #ifdef __i386__ void *TLS_tcb; /* current TCB for TLS */ #endif uint32_t km_version; /* Mailbox version */ uint32_t km_flags; /* KSE flags */ struct kse_thr_mailbox *km_curthread; /* Currently running thread */ struct kse_thr_mailbox *km_completed; /* Threads back from kernel */ [...] > > > /* > > * KSE mailbox. > > * > > * Communication path between the UTS and the kernel scheduler specific to > > * a single KSE. > > */ > > struct kse_mailbox { > > uint32_t km_version; /* Mailbox version */ > > uint32_t km_flags; /* KSE flags */ > > struct kse_thr_mailbox *km_curthread; /* Currently running thread */ > > struct kse_thr_mailbox *km_completed; /* Threads back from kernel */ > > sigset_t km_sigscaught; /* Caught signals */ > > Do we need to have all that. Isn't setting TMF_NOUPCALL enough? yes, it should be enough... Setting TMF_NOUPCALL ensures that any pre-emption will not result in a change in KSE, which is all that is needed to ensure that you can guarantee that finding your KSE is possible. > > tmbx = TP; > ret = (kse_critical_t)tmbx->tm_flags; > tmbx->tm_flags |= TMF_NOUPCALL; > return (ret); > > -- > Dan Eischen > > From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 15:05:03 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 81D1E37B401; Thu, 24 Jul 2003 15:05:03 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 1F2A443F3F; Thu, 24 Jul 2003 15:05:03 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with SMTP id h6OM50Up026628; Thu, 24 Jul 2003 15:05:01 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <002501c35230$1205be60$0701a8c0@tiger> From: "David Xu" To: "Julian Elischer" , References: Date: Fri, 25 Jul 2003 06:08:08 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 cc: threads@freebsd.org cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 22:05:03 -0000 ----- Original Message -----=20 From: "Julian Elischer" To: Cc: ; "David Xu" ; = Sent: Friday, July 25, 2003 4:59 AM Subject: Re: KSD/TSD take 2 (was: KSE critical regions) to the kernel. >=20 > In the i386 case, (and only the i386 case (unless the amd-64 case is = the > same)) the KSE MAILBOX is what we are pointing %gs:0 at, and in that > case, the pointer to the TCB (not the thread mailbox) is stored there. > (and set there by the UTS when scheduling a thread). >=20 > Thus the struct kse_mailbox would have: >=20 > struct kse_mailbox {=20 > #ifdef __i386__ > void *TLS_tcb; /* current TCB for TLS = */ > #endif > uint32_t km_version; /* Mailbox version */ > uint32_t km_flags; /* KSE flags */ > struct kse_thr_mailbox *km_curthread; /* Currently running = thread */ > struct kse_thr_mailbox *km_completed; /* Threads back from = kernel */ > [...] >=20 userland can always adapt the layout by: struct lib_kse_mailbox { void *TLS_tcb; struct kse_mailbox kmbx; }; and set base address to lib_kse_mailbox, userland can do whatever it wants to do. same thing can be done for thread mailbox. I don't think too many fields not related to interaction between kernel and userland should be pushed into mailbox, it is too ugly. David Xu From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 15:34:46 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 744A437B401; Thu, 24 Jul 2003 15:34:46 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id E10E843FBD; Thu, 24 Jul 2003 15:34:45 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc11) with ESMTP id <2003072422344401300na1v1e>; Thu, 24 Jul 2003 22:34:44 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id PAA70866; Thu, 24 Jul 2003 15:34:41 -0700 (PDT) Date: Thu, 24 Jul 2003 15:34:40 -0700 (PDT) From: Julian Elischer To: David Xu In-Reply-To: <002501c35230$1205be60$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: threads@freebsd.org cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 22:34:46 -0000 On Fri, 25 Jul 2003, David Xu wrote: > > ----- Original Message ----- > From: "Julian Elischer" > To: > Cc: ; "David Xu" ; > Sent: Friday, July 25, 2003 4:59 AM > Subject: Re: KSD/TSD take 2 (was: KSE critical regions) > to the kernel. > > > > In the i386 case, (and only the i386 case (unless the amd-64 case is the > > same)) the KSE MAILBOX is what we are pointing %gs:0 at, and in that > > case, the pointer to the TCB (not the thread mailbox) is stored there. > > (and set there by the UTS when scheduling a thread). > > > > Thus the struct kse_mailbox would have: > > > > struct kse_mailbox { > > #ifdef __i386__ > > void *TLS_tcb; /* current TCB for TLS */ > > #endif > > uint32_t km_version; /* Mailbox version */ > > uint32_t km_flags; /* KSE flags */ > > struct kse_thr_mailbox *km_curthread; /* Currently running thread */ > > struct kse_thr_mailbox *km_completed; /* Threads back from kernel */ > > [...] > > > > userland can always adapt the layout by: > > struct lib_kse_mailbox { > void *TLS_tcb; > struct kse_mailbox kmbx; > }; > > and set base address to lib_kse_mailbox, userland can > do whatever it wants to do. same thing can be done for > thread mailbox. > > I don't think too many fields not related to interaction > between kernel and userland should be pushed into mailbox, > it is too ugly. I was hoping to optimise by making the kse_create(mboxaddr) call allocate the LDT entry and set the gs register to point to the mailbox. making the two things differnt makes it less likely that making that optimisation makes sense. presently it requires special code to do the segments.. (in teh library) > > David Xu > > > > From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 16:01:04 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id BE31537B401; Thu, 24 Jul 2003 16:01:04 -0700 (PDT) Received: from rwcrmhc13.comcast.net (rwcrmhc13.comcast.net [204.127.198.39]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F81743F3F; Thu, 24 Jul 2003 16:01:04 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc13) with ESMTP id <2003072423005801500o821pe>; Thu, 24 Jul 2003 23:00:58 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id QAA71059; Thu, 24 Jul 2003 16:00:57 -0700 (PDT) Date: Thu, 24 Jul 2003 16:00:55 -0700 (PDT) From: Julian Elischer To: David Xu In-Reply-To: <002501c35230$1205be60$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: threads@freebsd.org cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 23:01:05 -0000 On Fri, 25 Jul 2003, David Xu wrote: > > ----- Original Message ----- > From: "Julian Elischer" > To: > Cc: ; "David Xu" ; > Sent: Friday, July 25, 2003 4:59 AM > Subject: Re: KSD/TSD take 2 (was: KSE critical regions) > to the kernel. > > > > userland can always adapt the layout by: > > struct lib_kse_mailbox { > void *TLS_tcb; > struct kse_mailbox kmbx; > }; > > and set base address to lib_kse_mailbox, userland can > do whatever it wants to do. same thing can be done for > thread mailbox. > > I don't think too many fields not related to interaction > between kernel and userland should be pushed into mailbox, > it is too ugly. I agree.. as long as we state very strongly that the segment register points to the TCB and NOT the mailbox, and that the mailbox may not be the first item in the TCB, then it works ok.. From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 16:07:59 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 04ADE37B401; Thu, 24 Jul 2003 16:07:59 -0700 (PDT) Received: from sledge.freebsd.org (sledge.freebsd.org [216.136.204.103]) by mx1.FreeBSD.org (Postfix) with ESMTP id 798EC43F3F; Thu, 24 Jul 2003 16:07:58 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (localhost [127.0.0.1]) by sledge.freebsd.org (8.12.9/8.12.9) with SMTP id h6ON7uv8034293; Thu, 24 Jul 2003 16:07:57 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <005701c35238$dc713a00$0701a8c0@tiger> From: "David Xu" To: "Julian Elischer" References: Date: Fri, 25 Jul 2003 07:11:04 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="ISO-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 cc: deischen@freebsd.org cc: threads@freebsd.org cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 24 Jul 2003 23:07:59 -0000 ----- Original Message -----=20 From: "Julian Elischer" To: "David Xu" Cc: ; ; Sent: Friday, July 25, 2003 6:34 AM Subject: Re: KSD/TSD take 2 (was: KSE critical regions) >=20 >=20 > On Fri, 25 Jul 2003, David Xu wrote: >=20 > >=20 > > ----- Original Message -----=20 > > From: "Julian Elischer" > > To: > > Cc: ; "David Xu" ; = > > Sent: Friday, July 25, 2003 4:59 AM > > Subject: Re: KSD/TSD take 2 (was: KSE critical regions) > > to the kernel. > > >=20 > > > In the i386 case, (and only the i386 case (unless the amd-64 case = is the > > > same)) the KSE MAILBOX is what we are pointing %gs:0 at, and in = that > > > case, the pointer to the TCB (not the thread mailbox) is stored = there. > > > (and set there by the UTS when scheduling a thread). > > >=20 > > > Thus the struct kse_mailbox would have: > > >=20 > > > struct kse_mailbox {=20 > > > #ifdef __i386__ > > > void *TLS_tcb; /* current TCB for = TLS */ > > > #endif > > > uint32_t km_version; /* Mailbox version = */ > > > uint32_t km_flags; /* KSE flags */ > > > struct kse_thr_mailbox *km_curthread; /* Currently = running thread */ > > > struct kse_thr_mailbox *km_completed; /* Threads back = from kernel */ > > > [...] > > >=20 > >=20 > > userland can always adapt the layout by: > >=20 > > struct lib_kse_mailbox { > > void *TLS_tcb; > > struct kse_mailbox kmbx; > > }; > >=20 > > and set base address to lib_kse_mailbox, userland can > > do whatever it wants to do. same thing can be done for > > thread mailbox. > >=20 > > I don't think too many fields not related to interaction > > between kernel and userland should be pushed into mailbox, > > it is too ugly. >=20 > I was hoping to optimise by making the kse_create(mboxaddr) > call allocate the LDT entry and set the gs register to point to the=20 > mailbox. making the two things differnt makes it less likely that > making that optimisation makes sense. >=20 Dan has two fields in kse_mailbox: void *km_ksdaddr; /* KSE specific data address */ uint32_t km_ksdsize; /* KSE specific data size */ I think it is used to tell kernel to map start address=20 of libkse's kse mailbox like above lib_kse_mailbox. > presently it requires special code to do the segments.. > (in teh library) >=20 >=20 Did you make some progresses in this area ? From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 17:46:06 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 24BD237B401; Thu, 24 Jul 2003 17:46:05 -0700 (PDT) Received: from rwcrmhc11.comcast.net (rwcrmhc11.comcast.net [204.127.198.35]) by mx1.FreeBSD.org (Postfix) with ESMTP id 13DA143F75; Thu, 24 Jul 2003 17:46:05 -0700 (PDT) (envelope-from julian@elischer.org) Received: from interjet.elischer.org ([12.233.125.100]) by attbi.com (rwcrmhc11) with ESMTP id <2003072500460301300n8g60e>; Fri, 25 Jul 2003 00:46:03 +0000 Received: from localhost (localhost.elischer.org [127.0.0.1]) by InterJet.elischer.org (8.9.1a/8.9.1) with ESMTP id RAA71730; Thu, 24 Jul 2003 17:45:58 -0700 (PDT) Date: Thu, 24 Jul 2003 17:45:57 -0700 (PDT) From: Julian Elischer To: David Xu In-Reply-To: <005701c35238$dc713a00$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: deischen@freebsd.org cc: threads@freebsd.org cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 00:46:07 -0000 On Fri, 25 Jul 2003, David Xu wrote: > > > > > Dan has two fields in kse_mailbox: > > void *km_ksdaddr; /* KSE specific data address */ > uint32_t km_ksdsize; /* KSE specific data size */ > > I think it is used to tell kernel to map start address > of libkse's kse mailbox like above lib_kse_mailbox. > > > presently it requires special code to do the segments.. > > (in teh library) > > > > > Did you make some progresses in this area ? It requires thatI rewrite the LDT code in WINE because wine just clobbers the LDT without looking to see if anyone else is using entries.. My aim was to have a call i386_set_ldt that takes an addr and length and returns teh value that needs to be set into a segment register in order to access that segment (or error if no more segments available). Wine could use it from userland and kse_create could use it from inside the kernel to set up a segment that covers the mailbox. kse_create would set it into the %gs register of the newly created kse. having it done by by the library with the 'shotgun' code is probably good enough for now > > > From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 17:53:59 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id E784737B401; Thu, 24 Jul 2003 17:53:59 -0700 (PDT) Received: from exchhz01.viatech.com.cn (ip-167-164-97-218.anlai.com [218.97.164.167]) by mx1.FreeBSD.org (Postfix) with ESMTP id 0E8E243F85; Thu, 24 Jul 2003 17:53:57 -0700 (PDT) (envelope-from davidxu@viatech.com.cn) Received: from davidw2k (ip-240-1-168-192.rev.dyxnet.com [192.168.1.240]) by exchhz01.viatech.com.cn with SMTP (Microsoft Exchange Internet Mail Service Version 5.5.2650.21) id P3SG7XQT; Fri, 25 Jul 2003 08:36:45 +0800 Message-ID: <001b01c35247$b8bd3870$f001a8c0@davidw2k> From: "David Xu" To: "Julian Elischer" References: Date: Fri, 25 Jul 2003 08:57:29 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 cc: deischen@freebsd.org cc: threads@freebsd.org cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 00:54:00 -0000 ----- Original Message -----=20 From: "Julian Elischer" To: "David Xu" Cc: ; ; Sent: Friday, July 25, 2003 8:45 AM Subject: Re: KSD/TSD take 2 (was: KSE critical regions) >=20 >=20 > On Fri, 25 Jul 2003, David Xu wrote: >=20 > >=20 > > >=20 > >=20 > > Dan has two fields in kse_mailbox: > >=20 > > void *km_ksdaddr; /* KSE specific data address */ > > uint32_t km_ksdsize; /* KSE specific data size */ > >=20 > > I think it is used to tell kernel to map start address=20 > > of libkse's kse mailbox like above lib_kse_mailbox. > >=20 > > > presently it requires special code to do the segments.. > > > (in teh library) > > >=20 > > >=20 > > Did you make some progresses in this area ? >=20 > It requires thatI rewrite the LDT code in WINE > because wine just clobbers the LDT without looking to see if anyone = else > is using entries.. >=20 >=20 > My aim was to have a call i386_set_ldt that takes an addr and length > and returns teh value that needs to be set into a segment register > in order to access that segment (or error if no more segments > available). Wine could use it from userland and kse_create could use = it > from inside the kernel to set up a segment that covers the mailbox. > kse_create would set it into the %gs register of the newly created = kse. > This sounds good to me. =20 > having it done by by the library with the 'shotgun' code > is probably good enough for now >=20 >=20 OK. From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 18:26:08 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 3256637B401; Thu, 24 Jul 2003 18:26:08 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 70EB043F93; Thu, 24 Jul 2003 18:26:07 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6P1Q2cJ011912; Thu, 24 Jul 2003 21:26:02 -0400 (EDT) Date: Thu, 24 Jul 2003 21:26:02 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: David Xu cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 01:26:08 -0000 On Thu, 24 Jul 2003, Julian Elischer wrote: > > On Thu, 24 Jul 2003, Daniel Eischen wrote: > > > > > struct kse_thr_mailbox { > > void *tm_tls[4]; /* reserved for TLS */ > > uint32_t tm_flags; > > uint32_t tm_version; > > struct kse_thr_mailbox *tm_next; > > void *tm_udata; > > uint32_t tm_uticks; > > uint32_t tm_sticks; > > register_t tm_spare[4]; > > siginfo_t tm_syncsig; > > ucontext_t tm_context; > > }; > > looking at the above diagrams, we see: > > struct kse_thr_mailbox need not have any TLS stuff. The offset to the It is only there because you suggested we need a reserve for TLS. It can easily be removed ;-) -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 18:37:40 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DAD2037B401; Thu, 24 Jul 2003 18:37:40 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3559C43FA3; Thu, 24 Jul 2003 18:37:40 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6P1bZcJ013791; Thu, 24 Jul 2003 21:37:35 -0400 (EDT) Date: Thu, 24 Jul 2003 21:37:35 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: David Xu In-Reply-To: <005701c35238$dc713a00$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Julian Elischer cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 01:37:41 -0000 On Fri, 25 Jul 2003, David Xu wrote: > > From: "Julian Elischer" > > > I was hoping to optimise by making the kse_create(mboxaddr) > > call allocate the LDT entry and set the gs register to point to the > > mailbox. making the two things differnt makes it less likely that > > making that optimisation makes sense. > > > > Dan has two fields in kse_mailbox: > > void *km_ksdaddr; /* KSE specific data address */ > uint32_t km_ksdsize; /* KSE specific data size */ > > I think it is used to tell kernel to map start address > of libkse's kse mailbox like above lib_kse_mailbox. Yes, that is what it is for. We really need this for amd64, so we might as well do it for x86. We'll need MD functions in the kernel to do this; they'll do nothing for all archs except x86 and amd64. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Thu Jul 24 18:40:08 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 4549E37B401; Thu, 24 Jul 2003 18:40:08 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 8B16B43F3F; Thu, 24 Jul 2003 18:40:07 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6P1e0cJ014182; Thu, 24 Jul 2003 21:40:00 -0400 (EDT) Date: Thu, 24 Jul 2003 21:40:00 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Julian Elischer In-Reply-To: Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: David Xu cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 01:40:08 -0000 On Thu, 24 Jul 2003, Julian Elischer wrote: > > On Fri, 25 Jul 2003, David Xu wrote: > > > > > > Did you make some progresses in this area ? > > It requires thatI rewrite the LDT code in WINE > because wine just clobbers the LDT without looking to see if anyone else > is using entries.. > > > My aim was to have a call i386_set_ldt that takes an addr and length > and returns teh value that needs to be set into a segment register > in order to access that segment (or error if no more segments > available). Wine could use it from userland and kse_create could use it > from inside the kernel to set up a segment that covers the mailbox. > kse_create would set it into the %gs register of the newly created kse. > > having it done by by the library with the 'shotgun' code > is probably good enough for now I'd like to try and get his implemented in this next round of changes. If you are going to add a system call, it's easy enough just to call the guts of it from kse_create(). -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Jul 25 00:35:13 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 739F137B401; Fri, 25 Jul 2003 00:35:13 -0700 (PDT) Received: from puffin.mail.pas.earthlink.net (puffin.mail.pas.earthlink.net [207.217.120.139]) by mx1.FreeBSD.org (Postfix) with ESMTP id 057AA43F85; Fri, 25 Jul 2003 00:35:11 -0700 (PDT) (envelope-from tlambert2@mindspring.com) Received: from user-2ivfi6b.dialup.mindspring.com ([165.247.200.203] helo=mindspring.com) by puffin.mail.pas.earthlink.net with asmtp (SSLv3:RC4-MD5:128) (Exim 3.33 #1) id 19fx6c-0007HS-00; Fri, 25 Jul 2003 00:35:10 -0700 Message-ID: <3F20DD72.A524688B@mindspring.com> Date: Fri, 25 Jul 2003 00:34:10 -0700 From: Terry Lambert X-Mailer: Mozilla 4.79 [en] (Win98; U) X-Accept-Language: en MIME-Version: 1.0 To: deischen@freebsd.org References: Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit X-ELNK-Trace: b1a02af9316fbb217a47c185c03b154d40683398e744b8a462e1b143b266ec56cdecb609ae1a9318387f7b89c61deb1d350badd9bab72f9c350badd9bab72f9c cc: threads@freebsd.org cc: marcel@xcllnt.net cc: Julian Elischer cc: David Xu Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 07:35:13 -0000 Daniel Eischen wrote: > I'd like to try and get his implemented in this next > round of changes. If you are going to add a system > call, it's easy enough just to call the guts of it > from kse_create(). FWIW: One trick would be to use an "invalid" value for one of the parameters on the existing call to mux you into an alternate implementsion (which could even take additional parameters, for obvious reasons). -- Terry From owner-freebsd-threads@FreeBSD.ORG Fri Jul 25 00:43:33 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 08D7137B401; Fri, 25 Jul 2003 00:43:33 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 4860743F85; Fri, 25 Jul 2003 00:43:32 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6P7hScJ009359; Fri, 25 Jul 2003 03:43:28 -0400 (EDT) Date: Fri, 25 Jul 2003 03:43:28 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Terry Lambert In-Reply-To: <3F20DD72.A524688B@mindspring.com> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: David Xu cc: Julian Elischer cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 07:43:33 -0000 On Fri, 25 Jul 2003, Terry Lambert wrote: > Daniel Eischen wrote: > > I'd like to try and get his implemented in this next > > round of changes. If you are going to add a system > > call, it's easy enough just to call the guts of it > > from kse_create(). > > FWIW: One trick would be to use an "invalid" value for one of the > parameters on the existing call to mux you into an alternate > implementsion (which could even take additional parameters, for > obvious reasons). Possibly... i386_set_ldt(int start_sel, union descriptor *descs, int num_sels); You could use start_sel = 0 and num_sels = 1 to have it automatically find the first one not already allocated. The function already returns the first selector allocated. -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Fri Jul 25 01:08:58 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 1C81637B401; Fri, 25 Jul 2003 01:08:58 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id AB82443F93; Fri, 25 Jul 2003 01:08:57 -0700 (PDT) (envelope-from davidxu@freebsd.org) Received: from tiger (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with SMTP id h6P88sUp088635; Fri, 25 Jul 2003 01:08:55 -0700 (PDT) (envelope-from davidxu@freebsd.org) Message-ID: <009101c35284$6f943350$0701a8c0@tiger> From: "David Xu" To: , "Terry Lambert" References: Date: Fri, 25 Jul 2003 16:12:02 +0800 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1106 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1106 cc: threads@freebsd.org cc: Julian Elischer cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 08:08:58 -0000 ----- Original Message -----=20 From: "Daniel Eischen" To: "Terry Lambert" Cc: ; "David Xu" ; "Julian = Elischer" ; Sent: Friday, July 25, 2003 3:43 PM Subject: Re: KSD/TSD take 2 (was: KSE critical regions) > On Fri, 25 Jul 2003, Terry Lambert wrote: >=20 > > Daniel Eischen wrote: > > > I'd like to try and get his implemented in this next > > > round of changes. If you are going to add a system > > > call, it's easy enough just to call the guts of it > > > from kse_create(). > >=20 > > FWIW: One trick would be to use an "invalid" value for one of the > > parameters on the existing call to mux you into an alternate > > implementsion (which could even take additional parameters, for > > obvious reasons). >=20 > Possibly... >=20 > i386_set_ldt(int start_sel, union descriptor *descs, int = num_sels); >=20 > You could use start_sel =3D 0 and num_sels =3D 1 to have it = automatically > find the first one not already allocated. The function already > returns the first selector allocated. >=20 Not only allocate, there needs a deallocate call. > --=20 > Dan Eischen >=20 > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to = "freebsd-threads-unsubscribe@freebsd.org" > From owner-freebsd-threads@FreeBSD.ORG Fri Jul 25 01:33:45 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id DD32037B401; Fri, 25 Jul 2003 01:33:45 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3F14843FAF; Fri, 25 Jul 2003 01:33:45 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6P8XfcJ017247; Fri, 25 Jul 2003 04:33:41 -0400 (EDT) Date: Fri, 25 Jul 2003 04:33:41 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: David Xu In-Reply-To: <009101c35284$6f943350$0701a8c0@tiger> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII cc: threads@freebsd.org cc: Julian Elischer cc: marcel@xcllnt.net Subject: Re: KSD/TSD take 2 (was: KSE critical regions) X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 25 Jul 2003 08:33:46 -0000 On Fri, 25 Jul 2003, David Xu wrote: > From: "Daniel Eischen" > > > > On Fri, 25 Jul 2003, Terry Lambert wrote: > > > > > Daniel Eischen wrote: > > > > I'd like to try and get his implemented in this next > > > > round of changes. If you are going to add a system > > > > call, it's easy enough just to call the guts of it > > > > from kse_create(). > > > > > > FWIW: One trick would be to use an "invalid" value for one of the > > > parameters on the existing call to mux you into an alternate > > > implementsion (which could even take additional parameters, for > > > obvious reasons). > > > > Possibly... > > > > i386_set_ldt(int start_sel, union descriptor *descs, int num_sels); > > > > You could use start_sel = 0 and num_sels = 1 to have it automatically > > find the first one not already allocated. The function already > > returns the first selector allocated. > > > Not only allocate, there needs a deallocate call. You could pass in NULL for descs... -- Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat Jul 26 06:00:42 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 261E437B401 for ; Sat, 26 Jul 2003 06:00:42 -0700 (PDT) Received: from silver.he.iki.fi (silver.he.iki.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id E996B43FBD for ; Sat, 26 Jul 2003 06:00:40 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from PETEX31 (h81.vuokselantie10.fi [193.64.42.129]) by silver.he.iki.fi (8.12.9/8.11.4) with SMTP id h6QD0csL039914 for ; Sat, 26 Jul 2003 16:00:38 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <069c01c35375$e9352880$812a40c1@PETEX31> From: "Petri Helenius" To: Date: Sat, 26 Jul 2003 16:00:37 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MIMEOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 Subject: libkse "wieght" X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 13:00:42 -0000 First of all, I´m very happy with the libkse since a few weeks ago, scheduling and signals seem to work for me and the application runs smoother than ever, although with only a handful of threads. I was wondering how "expensive" thread creation and termination designed to be with libkse, say should I just create and throw away tens or hundreds of threads in a small time or try to "recycle" the worker threads I already created? Most threads will be either waiting on network input/output,, condvar to flip or a sleep to expire. I would expect to have maximum of 5-10 threads executing at any time but I´m wondering if the traditional "large poll/select" approach is superior to creating, say, 500 threads? I´m not expecting you to design my app for me but just give insight of how expensive I should the thread maintenance and switching operations to be with libkse. By my previous experience with libkse, switching between threads is lightweight enough not to cause performance issues. I haven´t tested the creation/termination stuff yet. Pete From owner-freebsd-threads@FreeBSD.ORG Sat Jul 26 06:42:00 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id EAF7C37B401; Sat, 26 Jul 2003 06:42:00 -0700 (PDT) Received: from freefall.freebsd.org (freefall.freebsd.org [216.136.204.21]) by mx1.FreeBSD.org (Postfix) with ESMTP id 920EC43FA3; Sat, 26 Jul 2003 06:42:00 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) Received: from localhost (davidxu@localhost [127.0.0.1]) by freefall.freebsd.org (8.12.9/8.12.9) with ESMTP id h6QDfwUp062787; Sat, 26 Jul 2003 06:41:59 -0700 (PDT) (envelope-from davidxu@FreeBSD.org) From: David Xu Organization: Viatech To: "Petri Helenius" , Date: Sat, 26 Jul 2003 21:45:02 +0800 User-Agent: KMail/1.5.2 References: <069c01c35375$e9352880$812a40c1@PETEX31> In-Reply-To: <069c01c35375$e9352880$812a40c1@PETEX31> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Message-Id: <200307262145.02854.davidxu@FreeBSD.org> Subject: Re: libkse "wieght" X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 13:42:01 -0000 On Saturday 26 July 2003 21:00, Petri Helenius wrote: > First of all, I=B4m very happy with the libkse since a few weeks ago, > scheduling and signals seem to work for me and the application runs > smoother than ever, although with only a handful of threads. > > I was wondering how "expensive" thread creation and termination designed = to > be with libkse, say should I just create and throw away tens or hundreds = of > threads in a small time or try to "recycle" the worker threads I already > created?=20 With newest libkse source code, I can create 5000 threads and then pthread_join them in 0.6 seconds on my PIII 1Ghz machine. Although it is cheap enough to create thread and throw it away, but caching some threads i= s=20 still a good idea. > Most threads will be either waiting on network input/output,, > condvar to flip or a sleep to expire. I would expect to have maximum of > 5-10 threads executing at any time but I=B4m wondering if the traditional > "large poll/select" approach is superior to creating, say, 500 threads? > It depends on application, because poll/select on disk file has no effect, if you want to increase concurrent I/O for disk file, if you don't use AIO= ,=20 thread is still better then select/poll. > I=B4m not expecting you to design my app for me but just give insight of = how > expensive I should the thread maintenance and switching operations to be > with libkse. By my previous experience with libkse, switching between > threads is lightweight enough not to cause performance issues. I haven=B4t > tested the creation/termination stuff yet. > > Pete > > _______________________________________________ > freebsd-threads@freebsd.org mailing list > http://lists.freebsd.org/mailman/listinfo/freebsd-threads > To unsubscribe, send any mail to "freebsd-threads-unsubscribe@freebsd.org" From owner-freebsd-threads@FreeBSD.ORG Sat Jul 26 07:36:48 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 009A737B401; Sat, 26 Jul 2003 07:36:48 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id 3564143FBD; Sat, 26 Jul 2003 07:36:47 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6QEakax003333; Sat, 26 Jul 2003 10:36:46 -0400 (EDT) Date: Sat, 26 Jul 2003 10:36:46 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: David Xu In-Reply-To: <200307262145.02854.davidxu@FreeBSD.org> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: freebsd-threads@freebsd.org Subject: Re: libkse "wieght" X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 14:36:48 -0000 On Sat, 26 Jul 2003, David Xu wrote: > On Saturday 26 July 2003 21:00, Petri Helenius wrote: > > First of all, I=B4m very happy with the libkse since a few weeks ago, > > scheduling and signals seem to work for me and the application runs > > smoother than ever, although with only a handful of threads. > > > > I was wondering how "expensive" thread creation and termination designe= d to > > be with libkse, say should I just create and throw away tens or hundred= s of > > threads in a small time or try to "recycle" the worker threads I alread= y > > created?=20 >=20 > With newest libkse source code, I can create 5000 threads and then > pthread_join them in 0.6 seconds on my PIII 1Ghz machine. Although it is > cheap enough to create thread and throw it away, but caching some threads= is=20 > still a good idea. Libkse caches up to 100 threads for you, and throws away any more than that to free(). There is still a bit of set up to do with a libkse cached thread (makecontext(), add the thread to the run queue, etc), but it should be faster than creating a thread from scratch. I think you would have to modify the library to not cache threads (set MAX_CACHED_THREADS to 0 in lipthread/thread/thr_kern.c) in order to really benchmark the difference between the library caching threads and not caching threads. --=20 Dan Eischen From owner-freebsd-threads@FreeBSD.ORG Sat Jul 26 13:52:18 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id 17E5B37B401; Sat, 26 Jul 2003 13:52:18 -0700 (PDT) Received: from silver.he.iki.fi (silver.he.iki.fi [193.64.42.241]) by mx1.FreeBSD.org (Postfix) with ESMTP id C1A2B43F75; Sat, 26 Jul 2003 13:52:16 -0700 (PDT) (envelope-from pete@he.iki.fi) Received: from PETEX31 (h81.vuokselantie10.fi [193.64.42.129]) by silver.he.iki.fi (8.12.9/8.11.4) with SMTP id h6QKqEsL042743; Sat, 26 Jul 2003 23:52:15 +0300 (EEST) (envelope-from pete@he.iki.fi) Message-ID: <002701c353b7$cb29a310$812a40c1@PETEX31> From: "Petri Helenius" To: , "David Xu" References: Date: Sat, 26 Jul 2003 23:52:13 +0300 MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: 8bit X-Priority: 3 X-MSMail-Priority: Normal X-Mailer: Microsoft Outlook Express 6.00.2800.1158 X-MimeOLE: Produced By Microsoft MimeOLE V6.00.2800.1165 cc: freebsd-threads@freebsd.org Subject: Re: libkse "wieght" X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 20:52:18 -0000 >Libkse caches up to 100 threads for you, and throws away any more than >that to free(). There is still a bit of set up to do with a libkse >cached thread (makecontext(), add the thread to the run queue, etc), >but it should be faster than creating a thread from scratch. This sounds great, would probably catch most of the cases I´m thinking about without having to reach outside the cache that often. On a related note, looking at the code it seems to me that getaddrinfo holds a mutex while it´s waiting for get lookup to complete so if there is a stalled DNS lookup all other threads wait on the mutex? Is this observation correct? Pete From owner-freebsd-threads@FreeBSD.ORG Sat Jul 26 15:51:43 2003 Return-Path: Delivered-To: freebsd-threads@freebsd.org Received: from mx1.FreeBSD.org (mx1.freebsd.org [216.136.204.125]) by hub.freebsd.org (Postfix) with ESMTP id ABDB837B401; Sat, 26 Jul 2003 15:51:43 -0700 (PDT) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mx1.FreeBSD.org (Postfix) with ESMTP id AD06743FA3; Sat, 26 Jul 2003 15:51:42 -0700 (PDT) (envelope-from eischen@vigrid.com) Received: from mail.pcnet.com (mail.pcnet.com [204.213.232.4]) by mail.pcnet.com (8.12.8/8.12.1) with ESMTP id h6QMpfax018349; Sat, 26 Jul 2003 18:51:41 -0400 (EDT) Date: Sat, 26 Jul 2003 18:51:41 -0400 (EDT) From: Daniel Eischen X-Sender: eischen@pcnet5.pcnet.com To: Petri Helenius In-Reply-To: <002701c353b7$cb29a310$812a40c1@PETEX31> Message-ID: MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=X-UNKNOWN Content-Transfer-Encoding: QUOTED-PRINTABLE cc: David Xu cc: freebsd-threads@freebsd.org Subject: Re: libkse "wieght" X-BeenThere: freebsd-threads@freebsd.org X-Mailman-Version: 2.1.1 Precedence: list Reply-To: deischen@freebsd.org List-Id: Threading on FreeBSD List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 26 Jul 2003 22:51:44 -0000 On Sat, 26 Jul 2003, Petri Helenius wrote: >=20 > >Libkse caches up to 100 threads for you, and throws away any more than > >that to free(). There is still a bit of set up to do with a libkse > >cached thread (makecontext(), add the thread to the run queue, etc), > >but it should be faster than creating a thread from scratch. >=20 > This sounds great, would probably catch most of the cases I=B4m thinking > about without having to reach outside the cache that often. >=20 > On a related note, looking at the code it seems to me that getaddrinfo > holds a mutex while it=B4s waiting for get lookup to complete so if there > is a stalled DNS lookup all other threads wait on the mutex? Is this > observation correct? Probably. That's currently beyond my focus right now ;-) --=20 Dan Eischen