Date: Thu, 8 May 2008 21:59:57 +1000 (EST) From: Bruce Evans <brde@optusnet.com.au> To: Bakul Shah <bakul@bitblocks.com> Cc: freebsd-emulation@freebsd.org Subject: Re: seems I finally found what upset kqemu on amd64 SMP... shared gdt! (please test patch :) Message-ID: <20080508195843.G17500@delplex.bde.org> In-Reply-To: <20080507162713.73A3A5B47@mail.bitblocks.com> References: <20080507162713.73A3A5B47@mail.bitblocks.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Wed, 7 May 2008, Bakul Shah wrote: > On Wed, 07 May 2008 15:54:56 +1000 Bruce Evans <brde@optusnet.com.au> wrote: >> On Tue, 6 May 2008, Bakul Shah wrote: >>> See for instance >>> >>> http://docs.freebsd.org/cgi/getmsg.cgi?fetch=100953+0+archive/2007/freebsd- >> emulation/20070415.freebsd-emulation >>> >>> This seems to have not caused any problem in practice. And >>> any way taking out the message doesn't change the essential >>> behavior (the invariant is still broken) but it can speed up >>> your emulation considerably. >> >> I should have changed it to a panic long ago. That would give the correct >> number of messages (1) :-). > > Too late now for you to go fundamentalist :-) > >> i386 still doesn't even print a message (perhaps it never did). The >> bug would probably never have existed in any FreeBSD version of kqemu if >> i386 had had enough invariant checking. > > It does (in isa/npx.c) and I've disabled it! Not too late to go fundamentalist for that :-). I wrote it correctly. It paniced, but it was broken to a printf in rev.1.131 when I wasn't watching closely enough (though the log claims that I reviewed the patch, ISTR looking mainly at the parts in machdep.c). The message in npx.c is actually about violation of an even more fundamental invariant -- the invariant that owning the FPU includes having the TS flag clear so that DNA traps cannot occur. The bug in kqemu seems to be mismanagement of the TS flag related to this. I forget if it is the host or the target TS flag that seems to be mismanaged. For the target, it would take a bug in the virtualization of the TS flag to break this invariant (assuming no related bugs in the target kernel). The message in amd64/machdep.c is about violation of the invariant that the kernel cannot cause DNA traps. Spurious DNA traps in the target kernel might be caused either by violation of the previous invariant (then we might get a spurious DNA trap as part of non-broken target kernel FPU handling, after we have properly acquired ownership of the FPU). Non-spurious but wrong DNA traps in the host kernel might be caused by not properly acquiring ownership of the FPU before using it. This bug would be easier to implement, but I now remember more of the previous discussion and doubt that it is the bug in kqemu. It was said that kqemu never uses the FPU directly (not even for fxsave/fxrstor?) The second bug was implemented 3 years ago in the i386 linux emulator in rev.1.136 of linux_sysvec.c, but the bug is not detected because there is not even a printf about it in the i386 trap.c :-( and it is not a bug at the level of npx.c. The implementation is simply to hack on the FPU directly to set the control word. This is also a layering violation, but the API intended to be used for this initialization (npxinit()) is no longer used for this and no longer works for this. Direct hacking on the FPU works almost correctly here. It mainly wastes time by defeating the lazy reinitialization of the FPU on exec. > I seem to recall it is not just qemu but also some ndis > drivers that trigger this fpudna/npxdna message? I think that is from just hacking on the FPU directly. Windows drivers could easily have a bug like that. > Didn't > someone (Attilio?) has ported dragonfly code to allow FPU > register use in kernel mode? Whatever happened to it? I objected to it, and Attilio found better things to work on. Parts of it give a cleaner implementation of allowing controlled use of the FPU in the kernel, but even it didn't allow arbitrary use (as might be necessary for bad Windows drivers), and there are no known useful uses for the FPU/SSE in the kernel except on old systems: - Copying through SSE2 can be up to twice as fast as copying through 32-bit integer registers, but not-so-old systems in 64-bit mode can use 64-bit integer registers. Also, it is hard to find a real application where the speedups possible from copying through SSE2 affects more than the noise, since it takes very large data for copying running at multi-GB/S to take very long, but very large data is too large to cache and copying through SSE2 makes no difference in the uncached case.) - some multimedia instructions might be useful in the kernel. But it is hard to find a useful use that executes enough of them to justify a FPU context switch just to use them. Bruce
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20080508195843.G17500>