Date: Wed, 23 Jul 2003 13:16:08 -0700 (PDT) From: Julian Elischer <julian@elischer.org> To: threads@freebsd.org Cc: Julian Elischer <julian@elischer.org> Subject: Porting KSE to a new architecture.. kernel part Message-ID: <Pine.BSF.4.21.0307231224220.60197-100000@InterJet.elischer.org>
next in thread | raw e-mail | index | archive | help
David, Dan, Marcel, peter Here is my first hack at a small doc to help $ARCH developers enable KSE threading on their architecture. comments? additions? There are several small functions that need to be rewritten to allow KSE threading to run on a new architecture. Firstly it is important to understand the extra interractions that KSE threading has between the userland and the kernel. When a thread blocks in the kernel, the (virtual) cpu is reallocated to another newly created context (thread) and it is allowed to proceed back to userland and runs a known routine with a known pointer as its argument. This is called an "upcall" and there need to be machine specific functions to set it up. The blocking thread is allowed to continue at a later time but since the userland has already been returned to, It must not return there. Instead, when it reaches the user boundary, its context is stored back into the userland thread storage for that thread, in exactly the same way that it would look as if it had returned from the kernel, and then called 'yield'. The format of the saved context is "struct mcontext" and it should already be defined for each architecture. **************** Implementation ****************** ## The UPCALL ## There are two Machine independent functions that do the upcall, (It's done in two stages) and each calls a machine dependent part to do such things as may need to be done for each architecture. The machine INdependent functions are: thread_schedule_upcall() and thread_userret() Thread_schedule_upcall() calls cpu_set_upcall() and thread_userret() calls cpu_set_upcall_kse() Thread_schedule_upcall() sets up a new 'unused' Thread and requires that cpu_set_upcall() set up its context so that that thread, when run will return towards the user boundary. It does this in almost the same manner that fork() sets up the child thread to do the same thing. As a result, cpu_set_upcall() is almost the same in logic as cpu_fork(). It even sets up the context so that the next code run by the new thread is in fact fork_trampoline() and the (new) thread proceeds towards the userland boundary pretty much believing that it has just done a fork(). The main difference is that the flag TDP_UPCALLING is set in the thread's flags. In i386 this is about 14 lines of C and in ia64 it is about 18 lines of C. When the thread reaches the user boundary it runs userret() which in turn calls thread_userret() as mentionned above. Thread_userret() notices that TDP_UPCALLING is set and take special actions before allowing it to proceed to userland. One of these actions is to load the userland context with a PC (instruction pointer) for the upcall function for that particular (virtual) cpu, as well as setting the stack pointer to the stack associated with it. It also fiddles the arguments on the stack that the upcall function will see, setting the (single) argument to the address of that Virtual CPU's mailbox. This is all done by the function cpu_set_upcall_kse(). In i386 it is 4 lines of C and in ia64 it is about 14 lines of C. ## The context export to userland ** The blocking thread needs to be able to store its context back to userland when it has completed its kernel task, and an Machine dependent function is required for that. The basic userret() calls thread_userret() and that in turn calls thread_export_context(). Thread_export_context() in turn uses the machiine dependent function get_mcontext to extract the saved context from the thread's stack (or wherever it is stored) and write it in the proscribed format (struct mcontext), which is then written out to the thread context storage area in userland. It is possible that this may be rewritten to avoid the intermadiate copy of the mcontext in kernel space and write it directly to userland, however this is not what is done at this time. get_mcontext() has code that is much like that used to write signals to userland and can largly be cribbed (in loginc if not in code) from there (sendsig()). ## Other ## KSE threads also need some small assistance with signals. a machine dependent function cpu_thread_siginfo() is needed to format the information being sent to the userland scheduler into the correct format for that architecture.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.21.0307231224220.60197-100000>