From owner-svn-src-all@freebsd.org Thu Aug 13 14:21:06 2020 Return-Path: Delivered-To: svn-src-all@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 0E6CA37B24E; Thu, 13 Aug 2020 14:21:06 +0000 (UTC) (envelope-from mhorne@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 4BS7z56YSqz4M9k; Thu, 13 Aug 2020 14:21:05 +0000 (UTC) (envelope-from mhorne@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id C53801E748; Thu, 13 Aug 2020 14:21:05 +0000 (UTC) (envelope-from mhorne@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 07DEL547003304; Thu, 13 Aug 2020 14:21:05 GMT (envelope-from mhorne@FreeBSD.org) Received: (from mhorne@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 07DEL5K5003303; Thu, 13 Aug 2020 14:21:05 GMT (envelope-from mhorne@FreeBSD.org) Message-Id: <202008131421.07DEL5K5003303@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: mhorne set sender to mhorne@FreeBSD.org using -f From: Mitchell Horne Date: Thu, 13 Aug 2020 14:21:05 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r364193 - head/sys/riscv/riscv X-SVN-Group: head X-SVN-Commit-Author: mhorne X-SVN-Commit-Paths: head/sys/riscv/riscv X-SVN-Commit-Revision: 364193 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 13 Aug 2020 14:21:06 -0000 Author: mhorne Date: Thu Aug 13 14:21:05 2020 New Revision: 364193 URL: https://svnweb.freebsd.org/changeset/base/364193 Log: Enable interrupts while handling traps I observed hangs post-r362977 in QEMU with -smp 2, in which one thread would acquire write access to an rm_lock (sysctllock) and get stuck waiting in smp_rendezvous_cpus while the other CPU was servicing a trap. The other thread was waiting for read access to the same lock, thus causing deadlock. It's clear that this is just one symptom of a larger problem. The general expectation of MI kernel code is that interrupts are enabled. Violating this assumption will at best create some additional latency, but otherwise might cause locking or other unforeseen issues. All other architectures do so for some subset of trap values, but this somehow got missed in the RISC-V port. Enable interrupts now during kernel page faults and for all user trap types. The code in exception.S already knows to disable interrupts while handling the return from exception, so there are no changes required there. Reviewed by: jhb, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D26017 Modified: head/sys/riscv/riscv/trap.c Modified: head/sys/riscv/riscv/trap.c ============================================================================== --- head/sys/riscv/riscv/trap.c Thu Aug 13 14:17:36 2020 (r364192) +++ head/sys/riscv/riscv/trap.c Thu Aug 13 14:21:05 2020 (r364193) @@ -198,14 +198,22 @@ data_abort(struct trapframe *frame, int usermode) "Kernel page fault") != 0) goto fatal; - if (usermode) + if (usermode) { map = &td->td_proc->p_vmspace->vm_map; - else if (stval >= VM_MAX_USER_ADDRESS) - map = kernel_map; - else { - if (pcb->pcb_onfault == 0) - goto fatal; - map = &td->td_proc->p_vmspace->vm_map; + } else { + /* + * Enable interrupts for the duration of the page fault. For + * user faults this was done already in do_trap_user(). + */ + intr_enable(); + + if (stval >= VM_MAX_USER_ADDRESS) { + map = kernel_map; + } else { + if (pcb->pcb_onfault == 0) + goto fatal; + map = &td->td_proc->p_vmspace->vm_map; + } } va = trunc_page(stval); @@ -324,6 +332,7 @@ do_trap_user(struct trapframe *frame) riscv_cpu_intr(frame); return; } + intr_enable(); CTR3(KTR_TRAP, "do_trap_user: curthread: %p, sepc: %lx, frame: %p", curthread, frame->tf_sepc, frame);