Date: Thu, 19 Jun 2003 22:47:13 -0400 (EDT) From: Daniel Eischen <eischen@vigrid.com> To: Julian Elischer <julian@elischer.org> Cc: Marcel Moolenaar <marcel@xcllnt.net> Subject: Re: Implementing TLS: step 1 Message-ID: <Pine.GSO.4.10.10306192208420.4795-100000@pcnet5.pcnet.com> In-Reply-To: <Pine.BSF.4.21.0306191846200.41210-100000@InterJet.elischer.org>
next in thread | previous in thread | raw e-mail | index | archive | help
On Thu, 19 Jun 2003, Julian Elischer wrote: > > On Thu, 19 Jun 2003, Daniel Eischen wrote: > > > On Thu, 19 Jun 2003, Marcel Moolenaar wrote: > > > On Thu, Jun 19, 2003 at 04:12:50PM -0700, Julian Elischer wrote: > > > > > > > > > > > > BTW Marcel, I think that we can get from where we are now with kse to > > > > what we need by just a little massaging of what points to what. > > > > it is possible that the register %gs could be pointed to the thread > > > > structure directly and we could derive the KSE from that. it will just > > > > make the context switches a fraction more expensive if we need to > > > > change the segment register.. > > > > > > Yes. On i386 because the indirection through %gs:0x0 allows having %gs > > > point anything we like: the KSE. On ia64 there's room for a pointer at > > > offset 8 in the TLS that we can use to point to whatever we like. That > > > context switches involve an additional pointer fiddle to make sure we > > > always have the indirection to the KSE is something I don't worry about. > > > > Currently, the libkse TLS pointer has to point to the KSE mailbox. > > It is necessary to be able to set a word in the KSE mailbox in 1 > > instruction. The KSE mailbox "current thread pointer" must be > > set to NULL to prevent upcalls. Indirecting to get to the > > mailbox pointer in order to set it leaves open a race condition > > where the TLS changes out from under us before we set it. > > So if TP can't point to the KSE mailbox, then there needs to > > be other changes in the kernel and libkse. > > > > Julian, please clarify what I've said if it doesn't seem > > clear. > > > What you have said is true, but it desn't reflect on where %gs:0 points > becasue the kernel doesn't use %gs to find the user KSE mailbox. it just > "knows" . In the same what the UTS "just KNOWS" where the thread > mailbox and KSE mailboxes are. it doesn't need to get them via %gs:0 so > we can still do teh singl instruction stuff. We can set %gs befoer we > set the KSE->thread pointer, while we are in the UTS. No-one will follow > it until we start running as the thread because it is only used by code > using the __thread keyword. > > In other words, the KSE_mbox->thread_mbox link is completely separate > (orthogonal if you want) from the TLS pointer.. they are separate > issues. > The info is slightly redundant (only slightly) but they don't impact on > each other. > > I think we are just fine.. we can set teh forst entry in the KSE mailbox > to be a pointer to the TLS for teh thread we are switching in. > It will ONLY be read by the thread itself so we know that it must be > correct as we set it when we schedule it in.. > > A thread can never switch KSEs without doing so via an upcall, or That is exactly the problem. An upcall switching the current thread to another KSE while the UTS is in the middle of preventing upcalls. > otherwise involving the UTS, so we know that %gs:0 always points to a > pointer to the TLS.. Everythings cool. You're missing something. The UTS needs to prevent upcalls at times. The current method of doing that is by setting the KSE_mbox->thread_mbox (km_curthread) link to null. This has to be atomic in some way, because it is always possible for the thread to get swapped out (quantum, etc) and completed on another KSE. So code in the UTS that needs to do this: curthread = _get_curthread(); /* Prevent upcalls */ curkse = _get_curkse(); /* * Here is race condition. An upcall here can force the * thread to be resumed on another KSE. If that is the * case, then curkse is no longer valid and we're setting * km_curthread to NULL in the wrong KSE! */ atomic_store_ptr(&curkse->km_curthread, NULL); KSE_LOCK_ACQUIRE(curkse, &some_lock); ... Currently, the UTS can atomically clear km_curthread in 1 instruction, so there is no race condition and the above looks more like this: curthread = _get_curthread(); curkse = read_and_clear_kmbx(); /* 1 instruction */ KSE_LOCK_ACQUIRE(curkse, &some_lock); ... This isn't i386-specific. If TP is going to point to something other than the KSE mailbox, then you have to find another way to atomically stop upcalls without the KSE from changing out from under you. I think we can get around it, at least on ia64, just by clearing and restroing TP around the critical section, but the kernel needs to know the difference between TP being cleared and not. It can't just look in the KSE mailbox to see if km_curthread is NULL. So the above code would be: tp = read_and_clear_tp(); /* Some asm to do this. */ curkse = _get_curkse(tp); curthread = _get_curthread(tp); curkse->km_curthread = NULL; restore_tp(tp); > For that matter %gs:0 need not point to the mailbox at all, > since anyoen who needs to find it either has teh time to get it slowely > (via wherever it DOES point), or has other ways of finding it. Currently it _does_ have to point to the mailbox. David Xu has convinced me of that due to the above argument. It initially did _not_ point to the mailbox and that caused the exact problems I'm talking about. Trust me, we've already been down that route. -- Dan Eischen
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?Pine.GSO.4.10.10306192208420.4795-100000>