From owner-svn-src-head@freebsd.org Fri Mar 22 22:14:16 2019 Return-Path: Delivered-To: svn-src-head@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D3510155329A; Fri, 22 Mar 2019 22:14:15 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from mxrelay.nyi.freebsd.org (mxrelay.nyi.freebsd.org [IPv6:2610:1c1:1:606c::19:3]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) server-signature RSA-PSS (4096 bits) client-signature RSA-PSS (4096 bits) client-digest SHA256) (Client CN "mxrelay.nyi.freebsd.org", Issuer "Let's Encrypt Authority X3" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 7742395E3C; Fri, 22 Mar 2019 22:14:15 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mxrelay.nyi.freebsd.org (Postfix) with ESMTPS id 560CA26A01; Fri, 22 Mar 2019 22:14:15 +0000 (UTC) (envelope-from jhibbits@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id x2MMEFXh002471; Fri, 22 Mar 2019 22:14:15 GMT (envelope-from jhibbits@FreeBSD.org) Received: (from jhibbits@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id x2MMEEQT002468; Fri, 22 Mar 2019 22:14:14 GMT (envelope-from jhibbits@FreeBSD.org) Message-Id: <201903222214.x2MMEEQT002468@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhibbits set sender to jhibbits@FreeBSD.org using -f From: Justin Hibbits Date: Fri, 22 Mar 2019 22:14:14 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r345426 - in head/sys: conf powerpc/aim X-SVN-Group: head X-SVN-Commit-Author: jhibbits X-SVN-Commit-Paths: in head/sys: conf powerpc/aim X-SVN-Commit-Revision: 345426 X-SVN-Commit-Repository: base MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Rspamd-Queue-Id: 7742395E3C X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org X-Spamd-Result: default: False [-2.96 / 15.00]; local_wl_from(0.00)[FreeBSD.org]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; NEURAL_HAM_SHORT(-0.96)[-0.957,0]; ASN(0.00)[asn:11403, ipnet:2610:1c1:1::/48, country:US]; NEURAL_HAM_LONG(-1.00)[-1.000,0] X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 22 Mar 2019 22:14:16 -0000 Author: jhibbits Date: Fri Mar 22 22:14:14 2019 New Revision: 345426 URL: https://svnweb.freebsd.org/changeset/base/345426 Log: powerpc: Re-merge isa3 HPT with moea64 native HPT r345402 fixed the bug that led to the split of the ISA 3.0 HPT handling from the existing manager. The cause of the bug was gcc moving the register holding VPN to a different register (not r0), which triggered bizarre behaviors. With the fix, things work, so they can be re-merged. No performance lost with the merge. Deleted: head/sys/powerpc/aim/isa3_hashtb.c Modified: head/sys/conf/files.powerpc head/sys/powerpc/aim/aim_machdep.c head/sys/powerpc/aim/moea64_native.c Modified: head/sys/conf/files.powerpc ============================================================================== --- head/sys/conf/files.powerpc Fri Mar 22 21:31:21 2019 (r345425) +++ head/sys/conf/files.powerpc Fri Mar 22 22:14:14 2019 (r345426) @@ -103,7 +103,6 @@ libkern/qdivrem.c optional powerpc | powerpcspe libkern/ucmpdi2.c optional powerpc | powerpcspe libkern/udivdi3.c optional powerpc | powerpcspe libkern/umoddi3.c optional powerpc | powerpcspe -powerpc/aim/isa3_hashtb.c optional aim powerpc64 powerpc/aim/locore.S optional aim no-obj powerpc/aim/aim_machdep.c optional aim powerpc/aim/mmu_oea.c optional aim powerpc Modified: head/sys/powerpc/aim/aim_machdep.c ============================================================================== --- head/sys/powerpc/aim/aim_machdep.c Fri Mar 22 21:31:21 2019 (r345425) +++ head/sys/powerpc/aim/aim_machdep.c Fri Mar 22 22:14:14 2019 (r345426) @@ -421,9 +421,7 @@ aim_cpu_init(vm_offset_t toc) * in case the platform module had a better idea of what we * should do. */ - if (cpu_features2 & PPC_FEATURE2_ARCH_3_00) - pmap_mmu_install(MMU_TYPE_P9H, BUS_PROBE_GENERIC); - else if (cpu_features & PPC_FEATURE_64) + if (cpu_features & PPC_FEATURE_64) pmap_mmu_install(MMU_TYPE_G5, BUS_PROBE_GENERIC); else pmap_mmu_install(MMU_TYPE_OEA, BUS_PROBE_GENERIC); Modified: head/sys/powerpc/aim/moea64_native.c ============================================================================== --- head/sys/powerpc/aim/moea64_native.c Fri Mar 22 21:31:21 2019 (r345425) +++ head/sys/powerpc/aim/moea64_native.c Fri Mar 22 22:14:14 2019 (r345426) @@ -134,7 +134,8 @@ __FBSDID("$FreeBSD$"); /* POWER9 only permits a 64k partition table size. */ #define PART_SIZE 0x10000 -static int moea64_crop_tlbie; +static bool moea64_crop_tlbie; +static bool moea64_need_lock; static __inline void TLBIE(uint64_t vpn) { @@ -149,14 +150,26 @@ TLBIE(uint64_t vpn) { vpn <<= ADDR_PIDX_SHFT; /* Hobo spinlock: we need stronger guarantees than mutexes provide */ - while (!atomic_cmpset_int(&tlbie_lock, 0, 1)); - isync(); /* Flush instruction queue once lock acquired */ + if (moea64_need_lock) { + while (!atomic_cmpset_int(&tlbie_lock, 0, 1)); + isync(); /* Flush instruction queue once lock acquired */ + } if (moea64_crop_tlbie) vpn &= ~(0xffffULL << 48); #ifdef __powerpc64__ - __asm __volatile("li 0, 0; tlbie %0" :: "r"(vpn) : "0","memory"); + /* + * Explicitly clobber r0. The tlbie instruction has two forms: an old + * one used by PowerISA 2.03 and prior, and a newer one used by PowerISA + * 2.06 (maybe 2.05?) and later. We need to support both, and it just + * so happens that since we use 4k pages we can simply zero out r0, and + * clobber it, and the assembler will interpret the single-operand form + * of tlbie as having RB set, and everything else as 0. The RS operand + * in the newer form is in the same position as the L(page size) bit of + * the old form, so a slong as RS is 0, we're good on both sides. + */ + __asm __volatile("li 0, 0 \n tlbie %0" :: "r"(vpn) : "r0", "memory"); __asm __volatile("eieio; tlbsync; ptesync" ::: "memory"); #else vpn_hi = (uint32_t)(vpn >> 32); @@ -183,7 +196,8 @@ TLBIE(uint64_t vpn) { #endif /* No barriers or special ops -- taken care of by ptesync above */ - tlbie_lock = 0; + if (moea64_need_lock) + tlbie_lock = 0; } #define DISABLE_TRANS(msr) msr = mfmsr(); mtmsr(msr & ~PSL_DR) @@ -195,6 +209,8 @@ TLBIE(uint64_t vpn) { static volatile struct lpte *moea64_pteg_table; static struct rwlock moea64_eviction_lock; +static volatile struct pate *moea64_part_table; + /* * PTE calls. */ @@ -409,9 +425,14 @@ moea64_cpu_bootstrap_native(mmu_t mmup, int ap) * Install page table */ - __asm __volatile ("ptesync; mtsdr1 %0; isync" - :: "r"(((uintptr_t)moea64_pteg_table & ~DMAP_BASE_ADDRESS) - | (uintptr_t)(flsl(moea64_pteg_mask >> 11)))); + if (cpu_features2 & PPC_FEATURE2_ARCH_3_00) + mtspr(SPR_PTCR, + ((uintptr_t)moea64_part_table & ~DMAP_BASE_ADDRESS) | + flsl((PART_SIZE >> 12) - 1)); + else + __asm __volatile ("ptesync; mtsdr1 %0; isync" + :: "r"(((uintptr_t)moea64_pteg_table & ~DMAP_BASE_ADDRESS) + | (uintptr_t)(flsl(moea64_pteg_mask >> 11)))); tlbia(); } @@ -427,13 +448,18 @@ moea64_bootstrap_native(mmu_t mmup, vm_offset_t kernel moea64_early_bootstrap(mmup, kernelstart, kernelend); switch (mfpvr() >> 16) { + case IBMPOWER9: + moea64_need_lock = false; + break; case IBMPOWER4: case IBMPOWER4PLUS: case IBM970: case IBM970FX: case IBM970GX: case IBM970MP: - moea64_crop_tlbie = true; + moea64_crop_tlbie = true; + default: + moea64_need_lock = true; } /* * Allocate PTEG table. @@ -463,9 +489,23 @@ moea64_bootstrap_native(mmu_t mmup, vm_offset_t kernel if (hw_direct_map) moea64_pteg_table = (struct lpte *)PHYS_TO_DMAP((vm_offset_t)moea64_pteg_table); + /* Allocate partition table (ISA 3.0). */ + if (cpu_features2 & PPC_FEATURE2_ARCH_3_00) { + moea64_part_table = + (struct pate *)moea64_bootstrap_alloc(PART_SIZE, PART_SIZE); + if (hw_direct_map) + moea64_part_table = + (struct pate *)PHYS_TO_DMAP((vm_offset_t)moea64_part_table); + } DISABLE_TRANS(msr); bzero(__DEVOLATILE(void *, moea64_pteg_table), moea64_pteg_count * sizeof(struct lpteg)); + if (cpu_features2 & PPC_FEATURE2_ARCH_3_00) { + bzero(__DEVOLATILE(void *, moea64_part_table), PART_SIZE); + moea64_part_table[0].pagetab = + (DMAP_TO_PHYS((vm_offset_t)moea64_pteg_table)) | + (uintptr_t)(flsl((moea64_pteg_count - 1) >> 11)); + } ENABLE_TRANS(msr); CTR1(KTR_PMAP, "moea64_bootstrap: PTEG table at %p", moea64_pteg_table); @@ -512,7 +552,7 @@ tlbia(void) TLBSYNC(); - for (; i < 0x200000; i += 0x00001000) { + for (; i < 0x400000; i += 0x00001000) { #ifdef __powerpc64__ __asm __volatile("tlbiel %0" :: "r"(i)); #else