Date: Sat, 17 Aug 2002 19:18:54 +0100 From: Ian Dowse <iedowse@maths.tcd.ie> To: arch@freebsd.org Subject: Solving the stack gap issue Message-ID: <200208171918.aa72556@salmon.maths.tcd.ie>
next in thread | raw e-mail | index | archive | help
Many emulated Linux system calls use the stack gap to store paths
and structures that need to be converted before calling the native
system call. This has the well-known problem that shared address
space threads can corrupt each others stack gap data if they perform
system calls concurrently. Especially on SMP boxes this makes many
Linux applications unusable on FreeBSD.
A few approaches have been suggested:
- Lock access to the stack gap, so that only one thread at a time
can use it.
- Use a different address region for each thread.
- Avoid the need for the stack gap by providing kernel-callable versions
of all syscalls.
The first option would allow a single thread to hog the stack gap,
so most applications would probably deadlock instantly. The second
one sounds relatively straightforward, but so far nobody has found
a simple way to allocate and control the per-thread memory.
I have attempted to implement the third approach. It requires more
extensive changes than the others, but it has the advantage of
aiming to remove the stack gap hack instead of just adding another
bad-aid to it. That said, it does add some overhead to normal system
calls, so it may be that some ugliness is necessary to balance this
tradeoff.
The basic change is that many system calls have a version that is
called with the FreeBSD ABI syscall arguments, e.g.
The first option would allow a single thread to hog the stack gap,
so most applications would probably deadlock instantly. The second
one sounds relatively straightforward, but so far nobody has found
a simple way to allocate and control the per-thread memory.
I have attempted to implement the third approach. It requires more
extensive changes than the others, but it has the advantage of
aiming to remove the stack gap hack instead of just adding another
bad-aid to it. That said, it does add some overhead to normal system
calls, so it may be that some ugliness is necessary to balance this
tradeoff.
The basic change is that many system calls now have a version that
is called with the FreeBSD ABI syscall arguments, e.g.
int
open(struct thread *td, struct open_args *uap)
{
return sys_open(td, SCARG(uap, path), UIO_USERSPACE,
SCARG(uap, flags), SCARG(uap, mode));
}
and a version that contains all of the code for implementing open(2)
and is internal to the kernel, e.g.:
int sys_open(struct thread *td, char *path, enum uio_seg pathseg,
int flags, int mode);
In this case it is expected that we may need support for both
userspace and kernel-space paths, and since a path is a simple
string, it makes sense for the caller to specify the address space
from which the path should be read. For other functions, it seems
more appropriate to have the wrapper do the copyin() itself.
Anyway, there's a patch against -current at:
http://www.maths.tcd.ie/~iedowse/FreeBSD/stackgap.diff
This removes 80-90% of the stack gap uses in the i386 Linux emulation
code. I haven't done more than a basic level of testing though. I'm
also not particularly attached to any of the approaches taken here
- it is really just a proof of concept for this approach. Any
comments welcome!
Ian
To Unsubscribe: send mail to majordomo@FreeBSD.org
with "unsubscribe freebsd-arch" in the body of the message
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi? <200208171918.aa72556>