From owner-freebsd-arch Tue Nov 28 14:20:55 2000 Delivered-To: freebsd-arch@freebsd.org Received: from mail.interware.hu (mail.interware.hu [195.70.32.130]) by hub.freebsd.org (Postfix) with ESMTP id 9001E37B400 for ; Tue, 28 Nov 2000 14:20:47 -0800 (PST) Received: from timbuktu-06.budapest.interware.hu ([195.70.51.198] helo=elischer.org) by mail.interware.hu with esmtp (Exim 3.16 #1 (Debian)) id 140t7A-0004y2-00; Tue, 28 Nov 2000 23:20:40 +0100 Message-ID: <3A242FAF.313295F0@elischer.org> Date: Tue, 28 Nov 2000 14:20:31 -0800 From: Julian Elischer X-Mailer: Mozilla 4.7 [en] (X11; U; FreeBSD 5.0-CURRENT i386) X-Accept-Language: en, hu MIME-Version: 1.0 To: Daniel Eischen Cc: arch@FreeBSD.ORG Subject: Re: Thread-specific data and KSEs References: Content-Type: text/plain; charset=iso-8859-15 Content-Transfer-Encoding: 7bit Sender: owner-freebsd-arch@FreeBSD.ORG Precedence: bulk X-Loop: FreeBSD.ORG Daniel Eischen wrote: > > On Mon, 27 Nov 2000, Julian Elischer wrote: > > > > One thing I just realised: > > > > If we are using defered FP state saving and restoring in the kernel, then we > > will have troubles with that when switching threads in userland, since the > > handler for that is in the kernel. Of course we could set the place for it in > > the KSE mailbox and let the kernel save the information when it needs it. > > Our current threads library knows when to save and restore FP state; > it currently only happens when a signal is received (for i386, I think > alpha FP state is always saved both in jmp_buf and ucontext_t) That's comforting. I was looking at the ia64 specs.. that thing presents some interesting challenges in regards to the 'intelligent' stack it has. It will be very hard to play games with it's stack when it's cached inside the chip. I presume they have a scheme to allow such things as threads, but it looks a mess from here. > > I think we want to avoid saving and restoring FP state unless it's > necessary. That's probably only when a fault occurs or when the > KSE is preempted. I like the idea of having the kernel save the > FP state in the thread state storage area (ucontext_t?) in the > KSE mailbox thingy. The question is, what happens to the FPU context when you swap threads? should each thread have it's own FPU context? If there is one for the KSE, might that be not enough? especially if the KSE was pre-empted. If a thread is migrated to another KSE, having last been pre-empted, it becomes important that the FPU state go with it because it may have been part way through some calculations when that KSE was stopped. And what if the new KSE already has one that wsa stopped in thasame way? it looks to me like you need to have one per thread. It's not much different fromt eh point of view of the kernel. WHen you create a KSE you give it's mailbox. when you schedule a thread onto the KSE you set a pointer in that mailbox to the thread's context and state storage area. The kernel can easily follow that link when it pre-empts the KSE to store the General regs, the FPU regs etc. Theoretically it might only store the regs there in a syscall if it looks like the syscall will block. but the aim would be to make allthreads look the same when stopped so that the UTS can restart any one it chooses. > > Also, are we going to allow the kernel to follow links out of > the mailbox, or are we going to limit UTS<->kernel communication > to just this one page? I think it might be preferable to only > communicate via the mailbox and never have the kernel attempt > to read/write to other areas of KSE/thread storage. The kernel already has to follow links etc for (for example) the readv() syscall. it's not that big a step. If you allocate all the thread-context blocks together, the pages they are in will be pretty hot. There are great advantages to having the KSEs being able to follow links. For example it means that the kernel can ALWAYS deliver a linked list of ALL completed and 'ready-to-run' threads. It can set them up so that each one will look exactly as if it has just returned from the syscall. If you only deal with one structure, you have to consider what happens when you cannot fit all returning threads into the single structure. As the kernel takes control (as the syscall or trap is entered) it notes where the context block is and when and if it decide it needs to save context, it knows where to put it. The UTS is given everything on a plate, and it's almost easier to do it this way for the kernel too. It can store this address with the KSEC and use it without any fear of ever having a clash with some other returning syscall (for example). I can imagine where a syscall starts on one KSE and is completed on another. it makes sence for the context to travel with the thread/KSEC rather than the KSE, which may suddenly have 4 syscalls all coming back within the same upcall. (where do you save all that data?) > For instance, > we could place the pointer to the thread state storage area > in the mailbox. But that would require a copyin, and then a > copyout to another page that might be paged out. Since the thread storage is part of the thread control block that the UTS has just used to schedule the thread, it's unlikely to be paged out. Even less so if it shares a page with other thread control blocks And you could always protect it with madvise(). In any case what you suggest above is EXACTLY what jason and I were planning on doing. The kernel will define a structure in /sys/i386/include/kse.h (or somewhere) called something like struct user_process_context which you would include in your thread control block. it would include a link to other such blocks (so we can return a linked list of completed or pre-empted threads (KSECs) and a status word that says whether it is a completed syscall, or a preempted thread, or whatever, and a cookie that the kernel doesn't touch so you can extend it with whatever else you need. > The drawback > of only using the mailbox is that it requires an additional copy > by the UTS every time an upcall is made (to copy the thread state > from the mailbox to the storage area in the thread). You forget that a single upcall may want to return 37 completed syscalls or pre-empted threads. Using my scheme.. there is an upcall, and bingo the UTS has ALL the completed items at once. It sorts them onto the runnable queues, selects it's favourite and puts that address into it's mailbox, loads the context, and it's off an running the next thread. > > -- > Dan Eischen > > To Unsubscribe: send mail to majordomo@FreeBSD.org > with "unsubscribe freebsd-arch" in the body of the message -- __--_|\ Julian Elischer / \ julian@elischer.org ( OZ ) World tour 2000 ---> X_.---._/ presently in: Budapest v To Unsubscribe: send mail to majordomo@FreeBSD.org with "unsubscribe freebsd-arch" in the body of the message