From owner-freebsd-bugs@FreeBSD.ORG Mon Sep 16 17:20:01 2013 Return-Path: Delivered-To: freebsd-bugs@smarthost.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [8.8.178.115]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id 14F7E7C2 for ; Mon, 16 Sep 2013 17:20:01 +0000 (UTC) (envelope-from gnats@FreeBSD.org) Received: from freefall.freebsd.org (freefall.freebsd.org [IPv6:2001:1900:2254:206c::16:87]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id E784B2D3B for ; Mon, 16 Sep 2013 17:20:00 +0000 (UTC) Received: from freefall.freebsd.org (localhost [127.0.0.1]) by freefall.freebsd.org (8.14.7/8.14.7) with ESMTP id r8GHK0ew027931 for ; Mon, 16 Sep 2013 17:20:00 GMT (envelope-from gnats@freefall.freebsd.org) Received: (from gnats@localhost) by freefall.freebsd.org (8.14.7/8.14.7/Submit) id r8GHK0lq027926; Mon, 16 Sep 2013 17:20:00 GMT (envelope-from gnats) Resent-Date: Mon, 16 Sep 2013 17:20:00 GMT Resent-Message-Id: <201309161720.r8GHK0lq027926@freefall.freebsd.org> Resent-From: FreeBSD-gnats-submit@FreeBSD.org (GNATS Filer) Resent-To: freebsd-bugs@FreeBSD.org Resent-Reply-To: FreeBSD-gnats-submit@FreeBSD.org, Russ Cox Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1 with cipher ADH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTP id B098E7B6 for ; Mon, 16 Sep 2013 17:19:46 +0000 (UTC) (envelope-from nobody@FreeBSD.org) Received: from oldred.freebsd.org (oldred.freebsd.org [8.8.178.121]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.freebsd.org (Postfix) with ESMTPS id 9DAB72D34 for ; Mon, 16 Sep 2013 17:19:46 +0000 (UTC) Received: from oldred.freebsd.org ([127.0.1.6]) by oldred.freebsd.org (8.14.5/8.14.7) with ESMTP id r8GHJkaj001114 for ; Mon, 16 Sep 2013 17:19:46 GMT (envelope-from nobody@oldred.freebsd.org) Received: (from nobody@localhost) by oldred.freebsd.org (8.14.5/8.14.5/Submit) id r8GHJk22001111; Mon, 16 Sep 2013 17:19:46 GMT (envelope-from nobody) Message-Id: <201309161719.r8GHJk22001111@oldred.freebsd.org> Date: Mon, 16 Sep 2013 17:19:46 GMT From: Russ Cox To: freebsd-gnats-submit@FreeBSD.org X-Send-Pr-Version: www-3.1 Subject: kern/182161: restarting SYSCALL system call on amd64 loses arguments X-BeenThere: freebsd-bugs@freebsd.org X-Mailman-Version: 2.1.14 Precedence: list List-Id: Bug reports List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 16 Sep 2013 17:20:01 -0000 >Number: 182161 >Category: kern >Synopsis: restarting SYSCALL system call on amd64 loses arguments >Confidential: no >Severity: non-critical >Priority: low >Responsible: freebsd-bugs >State: open >Quarter: >Keywords: >Date-Required: >Class: sw-bug >Submitter-Id: current-users >Arrival-Date: Mon Sep 16 17:20:00 UTC 2013 >Closed-Date: >Last-Modified: >Originator: Russ Cox >Release: FreeBSD 9.1-RELEASE >Organization: Google >Environment: FreeBSD ptnw 9.1-RELEASE FreeBSD 9.1-RELEASE #0 r243825: Tue Dec 4 09:23:10 UTC 2012 root@farrell.cse.buffalo.edu:/usr/obj/usr/src/sys/GENERIC amd64 >Description: FreeBSD 9 (and other versions) appear to support invoking system calls using the SYSCALL instruction. However, that code path does not work for system call that will be restarted due to incoming signals (that is, due to signals with SA_RESTART set in their sigaction settings), because the fast_syscall path in amd64/amd64/exception.S only restores two of the six system call arguments. The specific problem we have been seeing is that if a SIGCHLD interrupts wait4 (and we've marked SIGCHLD SA_RESTART), then the wait4 loses its fourth argument, R10, which changes to a different value entirely. If the restarted wait4 succeeds, the copy into the rusage will return "bad address". I encountered this using Go, which invokes system calls using SYSCALL (because it seemed to work, I guess). I have reproduced it with a simple C program. The bug reproduces under ktrace, where it becomes easy to see. Watch thread 6264879. The first restart loses the argument (changes it to 0xa00200a0), and the second restart, actually finds a child and fails with errno 14. .. 50239 6264879 a.out CALL wait4(0xc760,0x7fffff9fcfb4,0<>0,0x7fffff9fcf20) 50239 6264879 a.out RET wait4 RESTART 50239 6264879 a.out CALL wait4(0xc760,0x7fffff9fcfb4,0<>0,0xa00200a0) 50239 6264384 a.out RET wait4 51039/0xc75f 50239 6264384 a.out CALL sigprocmask(SIG_BLOCK,0x80082c8f0,0x8010078e8) 50239 6264384 a.out RET sigprocmask 0 50239 6264384 a.out CALL fork 50239 6264384 a.out RET fork 51041/0xc761 50239 6264384 a.out CALL sigprocmask(SIG_SETMASK,0x8010078e8,0) 50239 6264384 a.out RET sigprocmask 0 50239 6264384 a.out CALL wait4(0xc761,0x7fffffbfdfb4,0<>0,0x7fffffbfdf20) 50239 6264879 a.out RET wait4 RESTART 50239 6264879 a.out PSIG SIGCHLD caught handler=0x800825520 mask=0x0 code=0x1 50239 6264879 a.out CALL sigprocmask(SIG_SETMASK,0x7fffff9fca5c,0) 50239 6264879 a.out RET sigprocmask 0 50239 6264879 a.out CALL sigreturn(0x7fffff9fc690) 50239 6264879 a.out RET sigreturn JUSTRETURN 50239 6264879 a.out CALL wait4(0xc760,0x7fffff9fcfb4,0<>0,0xa00200a0) 50239 6264879 a.out RET wait4 -1 errno 14 Bad address .. The INT $0x80 path does not have this bug - it restores all the registers correctly - so I will change the Go implementation of system calls on FreeBSD to use INT $0x80. >How-To-Repeat: Run the attached C program on an unloaded multicore system. It prints 'wait4 returned 14' on most runs. If it doesn't happen in the first few seconds, kill it and start again. >Fix: Restore the other four arguments at the end of fast_syscall. Patch attached with submission follows: #include #include #include #include #include #include #include #include static void handler(int); static void* looper(void*); int main(void) { int i; struct sigaction sa; pthread_cond_t cond; pthread_mutex_t mu; pthread_t t; memset(&sa, 0, sizeof sa); sa.sa_handler = handler; sa.sa_flags = SA_RESTART; memset(&sa.sa_mask, 0xff, sizeof sa.sa_mask); sigaction(SIGCHLD, &sa, 0); for(i=0; i<2; i++) pthread_create(&t, 0, looper, 0); pthread_mutex_init(&mu, 0); pthread_mutex_lock(&mu); pthread_cond_init(&cond, 0); for(;;) pthread_cond_wait(&cond, &mu); return 0; } static void handler(int sig) { } int mywait4(int pid, int *stat, int options, struct rusage *rusage) { int result; asm("movq %%rcx, %%r10; syscall" : "=a" (result) : "a" (7), "D" (pid), "S" (stat), "d" (options), "c" (rusage)); } static void* looper(void *v) { int pid, stat, out; struct rusage rusage; for(;;) { if((pid = fork()) == 0) _exit(0); out = mywait4(pid, &stat, 0, &rusage); if(out != pid) { printf("wait4 returned %d\n", out); } } } >Release-Note: >Audit-Trail: >Unformatted: