Date: Fri, 26 Nov 1999 15:37:58 -0800 (PST) From: Julian Elischer <julian@whistle.com> To: arch@freebsd.org, "Daniel M. Eischen" <eischen@vigrid.com> Subject: Re: Threads stuff Message-ID: <Pine.BSF.4.10.9911261503050.544-100000@current1.whistle.com> In-Reply-To: <383F1109.48AB132D@vigrid.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Fri, 26 Nov 1999, Daniel M. Eischen wrote:
> Julian Elischer wrote:
> >
> > But I was thinking of what context the UTS had access to for the thread.
> > For example, could the UTS time-out an IO and return a 'failed' result
> > and restart a thread that had made an IO request, even though the IO
> > hadn't returned? If all the context that was needed to restart a thread
> > blocked in the kernel was available to the UTS then that saved thread info
> > must be pretty much the same as what a thread blocked in the userland must
> > be.
>
> For the most part, the UTS doesn't need the context for threads blocked
> on I/O. It just needs to know how to resume and cancel them. If we
> optimize for the common case, then we don't need to make the context
> of the blocked thread available to the UTS. For threads that are
> preempted, the UTS only needs the program counter.
>
Let me make myself more clear..
I wasn't saying we needed to do that, but that if the state that the UTS
has available to it about a thread that is blocked in the kernel, and
a thread that is blocked in userland is the same, then it simplifies
things and such tricks become possible. It depends on what state the
thread leaves behind when it dives into the kernel, and grabs a KSE stack.
Alternatively it could have that state saved to it's context when it
is declared "blocked" by the upcall. Either way, the UTS could have all
the context needed to restart a thread, regardless of whether the kernel
wants it to :-)
> If you wanted to be able to time-out an I/O from the UTS (not a common
> case, we don't do it now), you could then add a system call to return
> the context of a blocked thread based on its unique KSE ID. This could
> also be used to retrieve the context of a thread that was preempted in
> the unlikely chance it was in a critical region.
>
> For threads that block in the kernel, we also need to save the kernel
> register set and stack. This will be stored in a KSE and we'll switch
> to a different KSE to return back to userland. The UTS still needs
> some way of identifying the KSE to be resumed. In your new syscall
> gate, are you returning something back from the kernel that can be
> used to identify the KSE that blocked?
>
> > > I don't think we want to require a kernel call to resume a thread
> > > that is blocked in user space, right? And I really don't think we
> > > want KSEs for every thread in an application, right?
> >
> > no, though you are suggesting it in other email because you say you need
> > to block/unblock signals, and your {s,g}et_context calls would require it.
>
> In an MT application, I think we can ignore signal masks for individual
> threads. As long as we have a method that lets the UTS act as signal
> proxy anyways. Actually, we should be able to safely use _setjmp/_longjmp
> instead of setjmp/longjmp now in libc_r. The UTS wouldn't use {g,s}etcontext
> other than for initially creating the UTS event handlers.
I don't think we need them for that.. see older email example.
>
> > >
> > > So other than the above, threads blocked in user space and threads
> > > blocked in kernel space are exactly the same. There will just be
> > > a flag and perhaps KSE ID in the user thread struct to indicate
> > > whether it can be resumed with a _longjmp or a thread_resume(2).
> >
> > After a UTS has been notified that a thread has blocked in the kernel,
> > it should end up being identical to one that is blocked in user space,
> > except it is waiting on an event that the kernel must supply
> > (i.e. an IO completion block or something.)
> >
> > Until it is notified of the blocking it is 'running'
>
> Right.
>
> > > Hmm, I was under the impression it was on the USER stack. Isn't
> > > that why we have to copyin/copyout to access the trapframe from
> > > the kernel?
> >
> > I need my x86 book, but it's at work..
> >
> > /*
> > * Call gate entry for FreeBSD ELF Linux/NetBSD syscall (int 0x80)
> > */
> > SUPERALIGN_TEXT
> > IDTVEC(int0x80_syscall)
> > subl $8,%esp /* skip over tf_trapno and tf_err
> > */
>
> At this point, doesn't %esp point at the user stack? Or does the
> int 0x80 somehow change the segments to be kernel segments?
I need the book but I think we are now runing on kernel stack
>
> > pushal <----
> > pushl %ds <----
> > pushl %es <----
> > pushl %fs <----
> > movl $KDSEL,%eax /* switch to kernel segments */
>
> The above comment is what confuses me. This makes me think that the
> push{al,l} statements above work on the user stack.
>
> > we've already switched to the kernel stack.
> > and we save all the regs (pushal) abd then the segment registers.
> > the stack segment has already been changed I think.
>
> OK, so the int 0x80 trap does switch segment registers and we
> are already on the kernel stack by the time that int0x80_syscall
> is entered?
That is my belief
>
> >
> > Of course if I need my x86 book it might not work for alpha..
> > but if the syscall itself did: (this is my version of the 'syscall()' syscall.)
> > ENTRY(syscall)
> > pop %ecx /* rta */
> > pop %eax /* syscall number */
> > push %ecx
> > pushal <-----------
> > pushl %ds <-----------
> > pushl %es <-----------
> > pushl %fs <-----------
> > KERNCALL
> > call _UTS_syscall_end <----------- may decide to schedule something else.
>
> What information is passed to _UTS_syscall_end? In the case that the
> thread blocks, is event information passed?
In the case the syscall blocked, it came back via the UPCALL, not via here.
>
> > popal <-----------
> > popl %fs <-----------
> > popl %es <-----------
> > popl %ds <-----------
> > push %ecx /* need to push a word to keep stack frame intact
> > upon return; the word must be the return address. */
> > jb 1f
> > ret
> > 1:
> > PIC_PROLOGUE
> > jmp PIC_PLT(HIDENAME(cerror)) <-- not needed.. already in return status b
> >
> > then the same stuff would not be needed in the first code segment
> > and all that context would be available to the UTS immediatly without
> > needing it to cross the kernel boundary at all.
>
> Why does the UTS need any of this, though? It just needs notification
> that the thread blocked and some "thingy" that can be used to resume
> or cancel it?
When the syscall blocks, the saved information above is to be turned into
a full frame. The PC (eip) is added (pointing to where we want it to
resume when the IO completes) and the thread is set up looking as though
it's waiting on it's IO completion block. The UTS gets (actualy should already know) the KSE-ID.
On completion of the IO, the io completion block is updated
just as it would have been had the syscall not blocked, and
the KSE is hung off the subprocess. When the subprocess is next run,
an upcall is made to the UTS, with some ID to let it know which
completion block to look at. This is simply the same as is some
other thread had released a mutex. The whole KSE in fact need not be hung
off the subproc because all that is needed is the single mutex address
to report.
>
> > The only problem I see is we would have to be very careful about getting a pagefault
> > while saving it.. pagefaults on stack segments are allowed to block?
>
> Sounds OK to me. I think the "Adding SA to Mach 3.0" paper talked about
> how pagefaults were handled.
>
> > > I think I follow you, but diagrams would help :-) Would there be some
> > > sort of unique ID returned from a blocked syscall, so that the UTS could
> > > later resume/cancel the thread?
> >
> > the blockage notification is from the syscall you think you are doing.
>
> How do you resume the thread? Longjmp'ing to the blocked system call
> won't work because the kernel doesn't know if it's a new system call
> or one that is being resumed.
If the userland context of the thread looks just like the context of
any other thread, then you just run it.
When the thread is reported as blocked, you fiddle the context it left
behind to make it look like it's blocked on a mutex on the io completion
block. when the syscall completes, the result is the equivalent of
releasing that mutex. i.e. the thread continues.
The result of syscalls is always reported using an IO completion block,
so on continuing from the mutex it checks to see the status etc, just like
is always done.
If the syscall didn't block, then the mutex is never hit even. It's all done
with mirrors, (and a little creative stack surgery).
>
> Dan Eischen
> eischen@vigrid.com
>
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.BSF.4.10.9911261503050.544-100000>