From owner-svn-src-head@freebsd.org Thu May 21 03:33:22 2020 Return-Path: Delivered-To: svn-src-head@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id 1E4882FF2B4; Thu, 21 May 2020 03:33:22 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256 client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 49SFZT727xz4GXD; Thu, 21 May 2020 03:33:21 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id EC03626C78; Thu, 21 May 2020 03:33:21 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id 04L3XLrp023938; Thu, 21 May 2020 03:33:21 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id 04L3XKSK023931; Thu, 21 May 2020 03:33:20 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <202005210333.04L3XKSK023931@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Thu, 21 May 2020 03:33:20 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r361317 - in head/sys/powerpc: aim include powerpc X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: in head/sys/powerpc: aim include powerpc X-SVN-Commit-Revision: 361317 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.33 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 21 May 2020 03:33:22 -0000 Author: jhibbits Date: Thu May 21 03:33:20 2020 New Revision: 361317 URL: https://svnweb.freebsd.org/changeset/base/361317 Log: powerpc: Handle machine checks caused by D-ERAT multihit Instead of crashing the user process when a D-ERAT multihit is detected, try to flush the ERAT, and continue. This machine check indicates a likely PMAP invalidation shortcoming that will need to be addressed, but it's recoverable, so just recover. The recovery is pmap-specific to flush the ERAT, so add a pmap function to do so, currently only implemented by the POWER9 radix pmap. Modified: head/sys/powerpc/aim/aim_machdep.c head/sys/powerpc/aim/mmu_radix.c head/sys/powerpc/include/pmap.h head/sys/powerpc/powerpc/mmu_if.m head/sys/powerpc/powerpc/pmap_dispatch.c Modified: head/sys/powerpc/aim/aim_machdep.c ============================================================================== --- head/sys/powerpc/aim/aim_machdep.c Thu May 21 02:10:45 2020 (r361316) +++ head/sys/powerpc/aim/aim_machdep.c Thu May 21 03:33:20 2020 (r361317) @@ -539,6 +539,10 @@ cpu_machine_check(struct thread *td, struct trapframe /* SLB multi-hit is recoverable. */ if ((frame->cpu.aim.dsisr & DSISR_MC_SLB_MULTIHIT) != 0) return (0); + if ((frame->cpu.aim.dsisr & DSISR_MC_DERAT_MULTIHIT) != 0) { + pmap_tlbie_all(); + return (0); + } /* TODO: Add other machine check recovery procedures. */ } else { if ((frame->srr1 & SRR1_MCHK_IFETCH_M) == SRR1_MCHK_IFETCH_SLBMH) Modified: head/sys/powerpc/aim/mmu_radix.c ============================================================================== --- head/sys/powerpc/aim/mmu_radix.c Thu May 21 02:10:45 2020 (r361316) +++ head/sys/powerpc/aim/mmu_radix.c Thu May 21 03:33:20 2020 (r361317) @@ -482,6 +482,7 @@ static void mmu_radix_dumpsys_map(mmu_t mmu, vm_paddr_ void **va); static void mmu_radix_scan_init(mmu_t mmu); static void mmu_radix_cpu_bootstrap(mmu_t, int ap); +static void mmu_radix_tlbie_all(mmu_t); static mmu_method_t mmu_radix_methods[] = { MMUMETHOD(mmu_bootstrap, mmu_radix_bootstrap), @@ -543,6 +544,8 @@ static mmu_method_t mmu_radix_methods[] = { MMUMETHOD(mmu_change_attr, mmu_radix_change_attr), MMUMETHOD(mmu_map_user_ptr, mmu_radix_map_user_ptr), MMUMETHOD(mmu_decode_kernel_ptr, mmu_radix_decode_kernel_ptr), + + MMUMETHOD(mmu_tlbie_all, mmu_radix_tlbie_all), { 0, 0 } }; @@ -772,6 +775,13 @@ mmu_radix_tlbiel_flush(int scope) tlbiel_flush_isa3(POWER9_TLB_SETS_RADIX, is); __asm __volatile(PPC_INVALIDATE_ERAT "; isync" : : :"memory"); +} + +static void +mmu_radix_tlbie_all(mmu_t __unused mmu) +{ + /* TODO: LPID invalidate */ + mmu_radix_tlbiel_flush(TLB_INVAL_SCOPE_GLOBAL); } static void Modified: head/sys/powerpc/include/pmap.h ============================================================================== --- head/sys/powerpc/include/pmap.h Thu May 21 02:10:45 2020 (r361316) +++ head/sys/powerpc/include/pmap.h Thu May 21 03:33:20 2020 (r361317) @@ -341,6 +341,7 @@ vm_offset_t pmap_early_io_map(vm_paddr_t pa, vm_size_t void pmap_early_io_unmap(vm_offset_t va, vm_size_t size); void pmap_track_page(pmap_t pmap, vm_offset_t va); void pmap_page_print_mappings(vm_page_t m); +void pmap_tlbie_all(void); static inline int pmap_vmspace_copy(pmap_t dst_pmap __unused, pmap_t src_pmap __unused) Modified: head/sys/powerpc/powerpc/mmu_if.m ============================================================================== --- head/sys/powerpc/powerpc/mmu_if.m Thu May 21 02:10:45 2020 (r361316) +++ head/sys/powerpc/powerpc/mmu_if.m Thu May 21 03:33:20 2020 (r361317) @@ -1100,3 +1100,11 @@ METHOD boolean_t ps_enabled { mmu_t _mmu; pmap_t _pmap; } DEFAULT mmu_null_ps_enabled; + + +/** + * @brief Flush the TLB (used by machine check handler). + */ +METHOD void tlbie_all { + mmu_t _mmu; +}; Modified: head/sys/powerpc/powerpc/pmap_dispatch.c ============================================================================== --- head/sys/powerpc/powerpc/pmap_dispatch.c Thu May 21 02:10:45 2020 (r361316) +++ head/sys/powerpc/powerpc/pmap_dispatch.c Thu May 21 03:33:20 2020 (r361317) @@ -631,6 +631,13 @@ pmap_ps_enabled(pmap_t pmap) return (MMU_PS_ENABLED(mmu_obj, pmap)); } +void +pmap_tlbie_all(void) +{ + CTR1(KTR_PMAP, "%s()", __func__); + return (MMU_TLBIE_ALL(mmu_obj)); +} + /* * MMU install routines. Highest priority wins, equal priority also * overrides allowing last-set to win.