From owner-svn-src-all@freebsd.org Tue Apr 24 12:50:23 2018 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id F3824FA326D; Tue, 24 Apr 2018 12:50:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id A5FC37AC4B; Tue, 24 Apr 2018 12:50:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id A0D221346F; Tue, 24 Apr 2018 12:50:22 +0000 (UTC) (envelope-from kib@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id w3OCoMA3096611; Tue, 24 Apr 2018 12:50:22 GMT (envelope-from kib@FreeBSD.org) Received: (from kib@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id w3OCoLkY096603; Tue, 24 Apr 2018 12:50:21 GMT (envelope-from kib@FreeBSD.org) Message-Id: <201804241250.w3OCoLkY096603@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: kib set sender to kib@FreeBSD.org using -f From: Konstantin Belousov Date: Tue, 24 Apr 2018 12:50:21 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r332930 - in head/sys: compat/linux conf i386/i386 i386/include i386/linux modules/linux X-SVN-Group: head X-SVN-Commit-Author: kib X-SVN-Commit-Paths: in head/sys: compat/linux conf i386/i386 i386/include i386/linux modules/linux X-SVN-Commit-Revision: 332930 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.25 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 24 Apr 2018 12:50:23 -0000 Author: kib Date: Tue Apr 24 12:50:21 2018 New Revision: 332930 URL: https://svnweb.freebsd.org/changeset/base/332930 Log: Fix futexes on i386 after the 4/4G split. Use proper method to access userspace. For now, only the slow copyout path is implemented. Reported and tested by: tijl (previous version) Sponsored by: The FreeBSD Foundation Added: head/sys/i386/linux/linux_copyout.c - copied, changed from r332929, head/sys/i386/linux/linux_support.s Deleted: head/sys/i386/linux/linux_support.s Modified: head/sys/compat/linux/linux_futex.c head/sys/compat/linux/linux_futex.h head/sys/conf/files.i386 head/sys/i386/i386/copyout.c head/sys/i386/include/md_var.h head/sys/modules/linux/Makefile Modified: head/sys/compat/linux/linux_futex.c ============================================================================== --- head/sys/compat/linux/linux_futex.c Tue Apr 24 12:17:28 2018 (r332929) +++ head/sys/compat/linux/linux_futex.c Tue Apr 24 12:50:21 2018 (r332930) @@ -273,14 +273,6 @@ static int handle_futex_death(struct linux_emuldata *, static int fetch_robust_entry(struct linux_robust_list **, struct linux_robust_list **, unsigned int *); -/* support.s */ -int futex_xchgl(int oparg, uint32_t *uaddr, int *oldval); -int futex_addl(int oparg, uint32_t *uaddr, int *oldval); -int futex_orl(int oparg, uint32_t *uaddr, int *oldval); -int futex_andl(int oparg, uint32_t *uaddr, int *oldval); -int futex_xorl(int oparg, uint32_t *uaddr, int *oldval); - - static int futex_copyin_timeout(int op, struct l_timespec *luts, int clockrt, struct timespec *ts) Modified: head/sys/compat/linux/linux_futex.h ============================================================================== --- head/sys/compat/linux/linux_futex.h Tue Apr 24 12:17:28 2018 (r332929) +++ head/sys/compat/linux/linux_futex.h Tue Apr 24 12:50:21 2018 (r332930) @@ -78,6 +78,11 @@ extern struct mtx futex_mtx; #define FUTEX_TID_MASK 0x3fffffff #define FUTEX_BITSET_MATCH_ANY 0xffffffff +int futex_xchgl(int oparg, uint32_t *uaddr, int *oldval); +int futex_addl(int oparg, uint32_t *uaddr, int *oldval); +int futex_orl(int oparg, uint32_t *uaddr, int *oldval); +int futex_andl(int oparg, uint32_t *uaddr, int *oldval); +int futex_xorl(int oparg, uint32_t *uaddr, int *oldval); void release_futexes(struct thread *, struct linux_emuldata *); Modified: head/sys/conf/files.i386 ============================================================================== --- head/sys/conf/files.i386 Tue Apr 24 12:17:28 2018 (r332929) +++ head/sys/conf/files.i386 Tue Apr 24 12:50:21 2018 (r332930) @@ -536,11 +536,10 @@ i386/ibcs2/ibcs2_xenix.c optional ibcs2 i386/ibcs2/ibcs2_xenix_sysent.c optional ibcs2 i386/ibcs2/imgact_coff.c optional ibcs2 i386/linux/imgact_linux.c optional compat_linux +i386/linux/linux_copyout.c optional compat_linux i386/linux/linux_dummy.c optional compat_linux i386/linux/linux_machdep.c optional compat_linux i386/linux/linux_ptrace.c optional compat_linux -i386/linux/linux_support.s optional compat_linux \ - dependency "linux_assym.h" i386/linux/linux_sysent.c optional compat_linux i386/linux/linux_sysvec.c optional compat_linux i386/pci/pci_cfgreg.c optional pci Modified: head/sys/i386/i386/copyout.c ============================================================================== --- head/sys/i386/i386/copyout.c Tue Apr 24 12:17:28 2018 (r332929) +++ head/sys/i386/i386/copyout.c Tue Apr 24 12:50:21 2018 (r332930) @@ -97,7 +97,7 @@ copyout_init_tramp(void) (uintptr_t)suword_fast + setidt_disp); } -static int +int cp_slow0(vm_offset_t uva, size_t len, bool write, void (*f)(vm_offset_t, void *), void *arg) { Modified: head/sys/i386/include/md_var.h ============================================================================== --- head/sys/i386/include/md_var.h Tue Apr 24 12:17:28 2018 (r332929) +++ head/sys/i386/include/md_var.h Tue Apr 24 12:50:21 2018 (r332930) @@ -55,6 +55,8 @@ struct segment_descriptor; union savefpu; void bcopyb(const void *from, void *to, size_t len); +int cp_slow0(vm_offset_t uva, size_t len, bool write, + void (*f)(vm_offset_t, void *), void *arg); void cpu_switch_load_gs(void) __asm(__STRING(cpu_switch_load_gs)); void copyout_init_tramp(void); void doreti_iret(void) __asm(__STRING(doreti_iret)); Copied and modified: head/sys/i386/linux/linux_copyout.c (from r332929, head/sys/i386/linux/linux_support.s) ============================================================================== --- head/sys/i386/linux/linux_support.s Tue Apr 24 12:17:28 2018 (r332929, copy source) +++ head/sys/i386/linux/linux_copyout.c Tue Apr 24 12:50:21 2018 (r332930) @@ -1,9 +1,12 @@ /*- * SPDX-License-Identifier: BSD-2-Clause-FreeBSD * - * Copyright (c) 2006,2007 Konstantin Belousov + * Copyright (c) 2018 The FreeBSD Foundation * All rights reserved. * + * This software was developed by Konstantin Belousov + * under sponsorship from the FreeBSD Foundation. + * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions * are met: @@ -16,7 +19,7 @@ * THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE * IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE - * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE + * ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE * FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL * DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS * OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) @@ -24,103 +27,152 @@ * LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY * OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF * SUCH DAMAGE. - * - * $FreeBSD$ */ -#include "linux_assym.h" /* system definitions */ -#include /* miscellaneous asm macros */ +#include +__FBSDID("$FreeBSD$"); -#include "assym.inc" +#include "opt_compat.h" -futex_fault_decx: - movl PCPU(CURPCB),%ecx -futex_fault: - movl $0,PCB_ONFAULT(%ecx) - movl $-EFAULT,%eax - ret +#include +#include +#include +#include +#include +#include +#include +#include +#include -ENTRY(futex_xchgl) - movl PCPU(CURPCB),%ecx - movl $futex_fault,PCB_ONFAULT(%ecx) - movl 4(%esp),%eax - movl 8(%esp),%edx - cmpl $VM_MAXUSER_ADDRESS-4,%edx - ja futex_fault - xchgl %eax,(%edx) - movl 12(%esp),%edx - movl %eax,(%edx) - xorl %eax,%eax - movl %eax,PCB_ONFAULT(%ecx) - ret +#include +#include -ENTRY(futex_addl) - movl PCPU(CURPCB),%ecx - movl $futex_fault,PCB_ONFAULT(%ecx) - movl 4(%esp),%eax - movl 8(%esp),%edx - cmpl $VM_MAXUSER_ADDRESS-4,%edx - ja futex_fault -#ifdef SMP - lock -#endif - xaddl %eax,(%edx) - movl 12(%esp),%edx - movl %eax,(%edx) - xorl %eax,%eax - movl %eax,PCB_ONFAULT(%ecx) - ret +#include +#include -ENTRY(futex_orl) - movl PCPU(CURPCB),%ecx - movl $futex_fault_decx,PCB_ONFAULT(%ecx) - movl 8(%esp),%edx - cmpl $VM_MAXUSER_ADDRESS-4,%edx - ja futex_fault - movl (%edx),%eax -1: movl %eax,%ecx - orl 4(%esp),%ecx -#ifdef SMP - lock -#endif - cmpxchgl %ecx,(%edx) - jnz 1b -futex_tail: - movl 12(%esp),%edx - movl %eax,(%edx) - xorl %eax,%eax - movl PCPU(CURPCB),%ecx - movl %eax,PCB_ONFAULT(%ecx) - ret +struct futex_st0 { + int oparg; + int *oldval; +}; -ENTRY(futex_andl) - movl PCPU(CURPCB),%ecx - movl $futex_fault_decx,PCB_ONFAULT(%ecx) - movl 8(%esp),%edx - cmpl $VM_MAXUSER_ADDRESS-4,%edx - ja futex_fault - movl (%edx),%eax -1: movl %eax,%ecx - andl 4(%esp),%ecx -#ifdef SMP - lock -#endif - cmpxchgl %ecx,(%edx) - jnz 1b - jmp futex_tail +static void +futex_xchgl_slow0(vm_offset_t kva, void *arg) +{ + struct futex_st0 *st; -ENTRY(futex_xorl) - movl PCPU(CURPCB),%ecx - movl $futex_fault_decx,PCB_ONFAULT(%ecx) - movl 8(%esp),%edx - cmpl $VM_MAXUSER_ADDRESS-4,%edx - ja futex_fault - movl (%edx),%eax -1: movl %eax,%ecx - xorl 4(%esp),%ecx -#ifdef SMP - lock -#endif - cmpxchgl %ecx,(%edx) - jnz 1b - jmp futex_tail + st = arg; + *st->oldval = atomic_swap_int((int *)kva, st->oparg); +} + +int +futex_xchgl(int oparg, uint32_t *uaddr, int *oldval) +{ + struct futex_st0 st; + + st.oparg = oparg; + st.oldval = oldval; + if (cp_slow0((vm_offset_t)uaddr, sizeof(uint32_t), true, + futex_xchgl_slow0, &st) != 0) + return (-EFAULT); + return (0); +} + +static void +futex_addl_slow0(vm_offset_t kva, void *arg) +{ + struct futex_st0 *st; + + st = arg; + *st->oldval = atomic_fetchadd_int((int *)kva, st->oparg); +} + +int +futex_addl(int oparg, uint32_t *uaddr, int *oldval) +{ + struct futex_st0 st; + + st.oparg = oparg; + st.oldval = oldval; + if (cp_slow0((vm_offset_t)uaddr, sizeof(uint32_t), true, + futex_addl_slow0, &st) != 0) + return (-EFAULT); + return (0); +} + +static void +futex_orl_slow0(vm_offset_t kva, void *arg) +{ + struct futex_st0 *st; + int old; + + st = arg; + old = *(int *)kva; + while (!atomic_fcmpset_int((int *)kva, &old, old | st->oparg)) + ; + *st->oldval = old; +} + +int +futex_orl(int oparg, uint32_t *uaddr, int *oldval) +{ + struct futex_st0 st; + + st.oparg = oparg; + st.oldval = oldval; + if (cp_slow0((vm_offset_t)uaddr, sizeof(uint32_t), true, + futex_orl_slow0, &st) != 0) + return (-EFAULT); + return (0); +} + +static void +futex_andl_slow0(vm_offset_t kva, void *arg) +{ + struct futex_st0 *st; + int old; + + st = arg; + old = *(int *)kva; + while (!atomic_fcmpset_int((int *)kva, &old, old & st->oparg)) + ; + *st->oldval = old; +} + +int +futex_andl(int oparg, uint32_t *uaddr, int *oldval) +{ + struct futex_st0 st; + + st.oparg = oparg; + st.oldval = oldval; + if (cp_slow0((vm_offset_t)uaddr, sizeof(uint32_t), true, + futex_andl_slow0, &st) != 0) + return (-EFAULT); + return (0); +} + +static void +futex_xorl_slow0(vm_offset_t kva, void *arg) +{ + struct futex_st0 *st; + int old; + + st = arg; + old = *(int *)kva; + while (!atomic_fcmpset_int((int *)kva, &old, old ^ st->oparg)) + ; + *st->oldval = old; +} + +int +futex_xorl(int oparg, uint32_t *uaddr, int *oldval) +{ + struct futex_st0 st; + + st.oparg = oparg; + st.oldval = oldval; + if (cp_slow0((vm_offset_t)uaddr, sizeof(uint32_t), true, + futex_xorl_slow0, &st) != 0) + return (-EFAULT); + return (0); +} Modified: head/sys/modules/linux/Makefile ============================================================================== --- head/sys/modules/linux/Makefile Tue Apr 24 12:17:28 2018 (r332929) +++ head/sys/modules/linux/Makefile Tue Apr 24 12:50:21 2018 (r332930) @@ -17,8 +17,12 @@ SRCS= linux_fork.c linux${SFX}_dummy.c linux_file.c li linux${SFX}_sysvec.c linux_uid16.c linux_time.c \ linux_timer.c linux_vdso.c \ opt_inet6.h opt_compat.h opt_posix.h opt_usb.h vnode_if.h \ - device_if.h bus_if.h \ - linux${SFX}_support.s + device_if.h bus_if.h +.if ${MACHINE_CPUARCH} == "amd64" +SRCS+= linux${SFX}_support.s +.else +SRCS+= linux_copyout.c +.endif DPSRCS= assym.inc linux${SFX}_genassym.c # XXX: for assym.inc @@ -56,11 +60,11 @@ linux${SFX}_locore.o: linux${SFX}_assym.h assym.inc -Wl,-soname=${VDSO}.so.1,--eh-frame-hdr,-warn-common \ ${.IMPSRC} -o ${.TARGET} +.if ${MACHINE_CPUARCH} == "amd64" linux${SFX}_support.o: linux${SFX}_assym.h assym.inc ${CC} -c -x assembler-with-cpp -DLOCORE ${CFLAGS} \ ${.IMPSRC} -o ${.TARGET} -.if ${MACHINE_CPUARCH} == "amd64" ${VDSO}.so: linux${SFX}_locore.o ${OBJCOPY} --input-target binary --output-target elf64-x86-64-freebsd \ --binary-architecture i386 linux${SFX}_locore.o ${.TARGET}