Date: Wed, 14 Jan 2009 02:01:04 GMT From: Gary Byers <gb@clozure.com> To: freebsd-gnats-submit@FreeBSD.org Subject: amd64/130526: fsbase issues for i386 processes running on 7.1-RELEASE/amd64 Message-ID: <200901140201.n0E214RK006767@www.freebsd.org> Resent-Message-ID: <200901140210.n0E2A2E4011773@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help
>Number: 130526
>Category: amd64
>Synopsis: fsbase issues for i386 processes running on 7.1-RELEASE/amd64
>Confidential: no
>Severity: critical
>Priority: low
>Responsible: freebsd-amd64
>State: open
>Quarter:
>Keywords:
>Date-Required:
>Class: sw-bug
>Submitter-Id: current-users
>Arrival-Date: Wed Jan 14 02:10:01 UTC 2009
>Closed-Date:
>Last-Modified:
>Originator: Gary Byers
>Release: 7.1-RELEASE/amd64
>Organization:
Clozure Associates
>Environment:
FreeBSD boddhi.abq.clozure.com 7.1-RELEASE FreeBSD 7.1-RELEASE #0: Thu Jan 1 08:58:24 UTC 2009 root@driscoll.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64
>Description:
After using i386_set_fsbase() to make the %fs segment register point to a specified linear address, 32-bit processes running on 7.1-RELEASE amd64 can't reliably use %fs to access memory.
Exactly when the process loses the ability to use %fs is unclear, but the enclosed example program seems to show that that ability is lost after a call to sleep().
I'm not familiar enough with x8664 system-level architecture to be fully comfortable in describing the symptom, but my limited understanding suggests that the symptom is consistent with the fsbase MSR not being restored correctly.
>How-To-Repeat:
The attached shell archive contains source to a small C program which seems to demonstrate the problem. (It likely needs to be compiled on a 32-bit FreeBSD system.)
The program establishes a signal handler for SIGBUS and SIGSEGV; the handler simply prints some context information and exits. The program then allocates a 100-byte pointer ("p") via malloc() and uses the pointer returned as an argument to i386_set_fsbase(); this should have the effect of making the %fs segment register address the pointer, so the byte or word at %fs:0 is equivalent to p[0].
The program then executes 100 iterations of a loop which stores a 32-bit value at the malloc'ed pointer's address, reads the 32-bit word at %fs:0, sleeps for 1 second, and reads the word at %fs:0 again. After each read of %fs:0, the value read relative to %fs is compared to the value stored at the pointer address; if the values differ, the code calls i386_set_fsbase() to see the adderss it returns by reference matches the pointer address, prints these values, and exits.
The program should therefore do little of interest and take about 100 seconds to do so. When run on a 7.1 amd64 release kernel, the program segfaults, usually after sleeping on the first iteration. (So the first attempt to reference memory at %fs:0 succeeded as expected; attempting to read the same value after calling sleep() fails.) This may indicate that the fsbase MSR is not restored correctly after context switch and/or syscall return.
>Fix:
Unknown.
A few days ago, I submitted an invalid bug report (130355) which claimed that this symptom was caused by failure to set %fs to the proper selector on return from i386_set_fsbase() on 7.1/amd64. No amd64 kernel changes the %fs selector in the implementation of i386_set_fsbase; 64-bit kernels do arrange that the "fsbase" MSR is set appropriately. It seems that this works correctly (the test program can use %fs before sleeping on the first iteration), but that the fsbase MSR isn't set correctly (or something like that ...) on syscall return or after context switch, in at least some cases.
Patch attached with submission follows:
# This is a shell archive. Save it in a file, remove anything before
# this line, and then unpack it by entering "sh file". Note, it may
# create directories; files and directories will be owned by you and
# have default permissions.
#
# This archive contains:
#
# i386_set_fsbase_test.c
#
echo x - i386_set_fsbase_test.c
sed 's/^X//' >i386_set_fsbase_test.c << 'e02ffb4f5ccf0ea67f579a83c1cccce4'
X/* This program should be compiled in a 32-bit i386 FreeBSD environment. */
X
X#include <stdio.h>
X#include <machine/segments.h>
X#include <machine/sysarch.h>
X#include <stdlib.h>
X#include <unistd.h>
X#include <sys/signal.h>
X#include <stdbool.h>
X
X
Xvoid
Xinstall_signal_handler(int signo, void * handler)
X{
X struct sigaction sa;
X
X sa.sa_sigaction = (void *)handler;
X sigfillset(&sa.sa_mask);
X sa.sa_flags = SA_SIGINFO;
X
X sigaction(signo, &sa, NULL);
X}
X
Xvoid *p, *check = NULL;
Xint i;
Xbool after_sleep = false;
X
Xvoid
Xsignal_handler(int signo, siginfo_t *info, void *context)
X{
X i386_get_fsbase(&check);
X
X fprintf(stderr, "terminating with signal %d on iteration %d, %s sleep, address = 0x%x, p = 0x%x, fsbase = 0x%x\n",
X signo,
X i,
X after_sleep ? "after" : "before",
X (uintptr_t)info->si_addr,
X (uintptr_t)p, (uintptr_t)check);
X exit(5);
X
X}
X
X/* return the contents of the first 32-bit word addressed by %fs */
Xunsigned long
Xread_fs_contents()
X{
X unsigned long res;
X
X __asm__ volatile ("movl %%fs:0,%0" : "=r" (res));
X return res;
X}
X
X
Xmain()
X{
X int status;
X
X install_signal_handler(SIGBUS, signal_handler);
X install_signal_handler(SIGSEGV, signal_handler);
X p = malloc(100);
X status = i386_set_fsbase(p);
X if (status != 0) {
X perror("i386_set_fsbase");
X exit(1);
X }
X /* it should now be true that %fs addresses the pointer 'p'; if we
X store a 32-bit value at 'p', we should be able to read that
X value back via read_fs_contents(), without getting any sort
X of bus fault.
X This seems to be true of all 32-bit FreeBSD kernels that I've
X tried it on and is true of the 64-bit 6.4-RELEASE and 7.0-RELEASE;
X it does not seem to be true of the 7.1/amd64-RELEASE kernel.
X */
X for (i = 0; i < 100; i++) {
X *((unsigned long *)p) = i;
X after_sleep = false;
X if (read_fs_contents() != i) {
X i386_get_fsbase(&check);
X fprintf (stderr, "%%fs base may have changed, now 0x%x, should be 0x%x\n",(uintptr_t)check,(uintptr_t)p);
X exit(2);
X }
X sleep(1);
X after_sleep = true;
X if (read_fs_contents() != i) {
X i386_get_fsbase(&check);
X fprintf (stderr, "after sleep, %%fs base may have changed, now 0x%x, should be 0x%x\n",(uintptr_t)check,(uintptr_t)p);
X exit(3);
X }
X }
X exit(0);
X}
X
e02ffb4f5ccf0ea67f579a83c1cccce4
exit
>Release-Note:
>Audit-Trail:
>Unformatted:
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?200901140201.n0E214RK006767>