Skip site navigation (1)Skip section navigation (2) 
Date:      Mon, 16 Sep 2013 17:19:46 GMT
From:      Russ Cox <rsc@swtch.com>
To:        freebsd-gnats-submit@FreeBSD.org
Subject:   kern/182161: restarting SYSCALL system call on amd64 loses arguments
Message-ID:  <201309161719.r8GHJk22001111@oldred.freebsd.org>
Resent-Message-ID: <201309161720.r8GHK0lq027926@freefall.freebsd.org>
next in thread | raw e-mail | index | archive | help 

>Number:         182161
>Category:       kern
>Synopsis:       restarting SYSCALL system call on amd64 loses arguments
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    freebsd-bugs
>State:          open
>Quarter:        
>Keywords:       
>Date-Required:
>Class:          sw-bug
>Submitter-Id:   current-users
>Arrival-Date:   Mon Sep 16 17:20:00 UTC 2013
>Closed-Date:
>Last-Modified:
>Originator:     Russ Cox
>Release:        FreeBSD 9.1-RELEASE
>Organization:
Google
>Environment:
FreeBSD ptnw 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec  4 09:23:10 UTC 2012     root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC  amd64

>Description:
FreeBSD 9 (and other versions) appear to support invoking system calls using the SYSCALL instruction. However, that code path does not work for system call that will be restarted due to incoming signals (that is, due to signals with SA_RESTART set in their sigaction settings), because the fast_syscall path in amd64/amd64/exception.S only restores two of the six system call arguments. 

The specific problem we have been seeing is that if a SIGCHLD interrupts wait4 (and we've marked SIGCHLD SA_RESTART), then the wait4 loses its fourth argument, R10, which changes to a different value entirely. If the restarted wait4 succeeds, the copy into the rusage will return "bad address".

I encountered this using Go, which invokes system calls using SYSCALL (because it seemed to work, I guess). I have reproduced it with a simple C program.


The bug reproduces under ktrace, where it becomes easy to see. Watch thread 6264879. The first restart loses the argument (changes it to 0xa00200a0), and the second restart, actually finds a child and fails with errno 14.

..
 50239 6264879 a.out    CALL  wait4(0xc760,0x7fffff9fcfb4,0<><invalid>0,0x7fffff9fcf20)
 50239 6264879 a.out    RET   wait4 RESTART
 50239 6264879 a.out    CALL  wait4(0xc760,0x7fffff9fcfb4,0<><invalid>0,0xa00200a0)

 50239 6264384 a.out    RET   wait4 51039/0xc75f
 50239 6264384 a.out    CALL  sigprocmask(SIG_BLOCK,0x80082c8f0,0x8010078e8)
 50239 6264384 a.out    RET   sigprocmask 0
 50239 6264384 a.out    CALL  fork
 50239 6264384 a.out    RET   fork 51041/0xc761
 50239 6264384 a.out    CALL  sigprocmask(SIG_SETMASK,0x8010078e8,0)
 50239 6264384 a.out    RET   sigprocmask 0
 50239 6264384 a.out    CALL  wait4(0xc761,0x7fffffbfdfb4,0<><invalid>0,0x7fffffbfdf20)

 50239 6264879 a.out    RET   wait4 RESTART
 50239 6264879 a.out    PSIG  SIGCHLD caught handler=0x800825520 mask=0x0 code=0x1
 50239 6264879 a.out    CALL  sigprocmask(SIG_SETMASK,0x7fffff9fca5c,0)
 50239 6264879 a.out    RET   sigprocmask 0
 50239 6264879 a.out    CALL  sigreturn(0x7fffff9fc690)
 50239 6264879 a.out    RET   sigreturn JUSTRETURN
 50239 6264879 a.out    CALL  wait4(0xc760,0x7fffff9fcfb4,0<><invalid>0,0xa00200a0)
 50239 6264879 a.out    RET   wait4 -1 errno 14 Bad address
..

The INT $0x80 path does not have this bug - it restores all the registers correctly - so I will change the Go implementation of system calls on FreeBSD to use INT $0x80.
>How-To-Repeat:
Run the attached C program on an unloaded multicore system. It prints 'wait4 returned 14' on most runs. If it doesn't happen in the first few seconds, kill it and start again.

>Fix:
Restore the other four arguments at the end of fast_syscall.


Patch attached with submission follows:

#include <sys/resource.h>
#include <sys/time.h>
#include <sys/signal.h>
#include <pthread.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
#include <string.h>

static void handler(int);
static void* looper(void*);

int
main(void)
{
	int i;
	struct sigaction sa;
	pthread_cond_t cond;
	pthread_mutex_t mu;
	pthread_t t;

	memset(&sa, 0, sizeof sa);
	sa.sa_handler = handler;
	sa.sa_flags = SA_RESTART;
	memset(&sa.sa_mask, 0xff, sizeof sa.sa_mask);
	sigaction(SIGCHLD, &sa, 0);

	for(i=0; i<2; i++)
		pthread_create(&t, 0, looper, 0);

	pthread_mutex_init(&mu, 0);
	pthread_mutex_lock(&mu);
	pthread_cond_init(&cond, 0);
	for(;;)
		pthread_cond_wait(&cond, &mu);

	return 0;
}

static void
handler(int sig)
{
}

int
mywait4(int pid, int *stat, int options, struct rusage *rusage)
{
	int result;

	asm("movq %%rcx, %%r10; syscall" 
		: "=a" (result)
		: "a" (7),
		  "D" (pid),
		  "S" (stat),
		  "d" (options),
		  "c" (rusage));
}

static void*
looper(void *v)
{
	int pid, stat, out;
	struct rusage rusage;

	for(;;) {
		if((pid = fork()) == 0)
			_exit(0);
		out = mywait4(pid, &stat, 0, &rusage);
		if(out != pid) {
			printf("wait4 returned %d\n", out);
		}
	}
}


>Release-Note:
>Audit-Trail:
>Unformatted:

Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?201309161719.r8GHJk22001111>