Date: Wed, 17 Jan 2007 14:43:14 -0800 (PST) From: Matthew Dillon <dillon@apollo.backplane.com> To: Ivan Voras <ivoras@fer.hr> Cc: freebsd-current@freebsd.org, freebsd-arch@freebsd.org Subject: Re: [PATCH] Mantaining turnstile aligned to 128 bytes in i386 CPUs Message-ID: <200701172243.l0HMhECY054399@apollo.backplane.com> References: <3bbf2fe10607250813w8ff9e34pc505bf290e71758@mail.gmail.com> <3bb f2fe10607281004o6727e976h19ee7e054876f914@mail.gmail.com> <3bbf2fe107011608 51r79b04464m2cbdbb7f644b22b6@mail.gmail.com> <20070116154258.568e1aaf@pleia des.nextvenue.com> <b1fa29170701161355lc021b90o35fa5f9acb5749d@mail.gmail.c om> <eoji7s$cit$2@sea.gmane.org> <b1fa29170701161425n7bcfe1e5m1b8c671caf375 8db@mail.gmail.com> <eojlnb$qje$1@sea.gmane.org> <3bbf2fe10701161525j6ad929 2y93502b8df0f67aa9@mail.gmail.com> <45AD6DFA.6030808@FreeBSD.org> <3bbf2fe1 0701161655p5e686b52n7340b3100ecfab93@mail.gmail.com> <200701172022.l0HKMYV8053837@apollo.backplane.com> <45AE8DDC.8030402@fer.hr>
next in thread | previous in thread | raw e-mail | index | archive | help
:Does the same hold true with kernel threads in FreeBSD (e.g. two threads
:using FPU)?
Preemption and pinning make the issue a bit more difficult for FreeBSD,
but the basic idea remains valid.
From the point of view of NPXTHREAD the situation is very simple:
* NPXTHREAD = NULL
nobody owns the FP, nobody is using the FP. If the kernel wants
to use the FP it just FNCLEX + CLTS and sets NPXTHREAD = curthread.
When it is finished, it undoes that sequence (NPXTHREAD = NULL,
set CR0_TS again).
PLUSES: FP state does not need to be saved or restored
ISSUES: due to cpu migration and preemption the setup and teardown
sequence must be done with the cpu pinned, inside a critical section.
But the actual use of the FP does not need to occur inside a
critical section or with the cpu pinned.
* NPXTHREAD = other_thread
Some other thread owns the FP, but it isn't our thread so we can
safely save the FP state for the other thread without worrying
about creating a situation where we thrash the T_DNA exception.
Save FP state, FNCLEX, CLTS, set NPXTHREAD = curthread.
When finished, NPXTHREAD = NULL, set CR0_TS, do *not* restore
the 'other' thread's FP state.
PLUSES: The FP state probably had to be saved anyway, it's no big
deal or at least it is not as big a deal as the NPXTHREAD = curthread
case.
ISSUES: Same as above.
* NPXTHREAD = curthread
The current thread (either userland or a pushed kernel FP context)
is using the FP. If the kernel decides it needs the FP it
must save the FP state, FNCLEX, CLTS, do its thing. When finished
it can decide to set NPXTHREAD = NULL and set CR0_TS, or it
can restore the previously saved state and leave NPXTHREAD = curthread.
PLUSES: Very few
ISSUES: Same as above, but here the kernel must decide whether it
is worth stealing the FP or not, because it might get into a
thrashing situation with the T_DNA exception under certain
userland loads.
Note that there are many cases where userland may use the FP unit
very occassionally. In such cases you *DO* want to be able to steal
it, so perhaps some heuristic is needed to determine the cost of
stealing the FP unit dynamically.
It is possible to abstract it even more... for example, one can set
CR0_TS when going from userland to the kernel and completely abstract
out the kernel's use of the FP unit at the cost of a higher entrance
fee to get in and out of the kernel. I decided NOT to do this in
DragonFly. If the DragonFly kernel wants to use the FP it has to
check and adjust the NPXTHREAD state.
But, to be absolutely clear here, it costs virtually *nothing* to use
the FP in the kernel for non-FP media instructions (i.e. movdq and
friends) if userland has not used the FP recently. You push a
temporary save area, set NPXTHREAD, FNCLEX, CLTS, use the FP, then
pop the save area pointer, set NPXTHREAD to NULL, and set CR0_TS, and
that's it. It may seem like a lot of steps but those are all
very fast instructions verses having to actually save and restore the
512 byte FP state. The biggest overhead would actually be the critical
section and cpu pinning required to properly transition the NPXTHREAD
state.
-Matt
Matthew Dillon
<dillon@backplane.com>
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200701172243.l0HMhECY054399>