From owner-svn-src-all@freebsd.org Wed Oct 26 18:47:48 2016 Return-Path: Delivered-To: svn-src-all@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 5BD22C22382; Wed, 26 Oct 2016 18:47:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org (repo.freebsd.org [IPv6:2610:1c1:1:6068::e6a:0]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 13F873E3; Wed, 26 Oct 2016 18:47:48 +0000 (UTC) (envelope-from jhb@FreeBSD.org) Received: from repo.freebsd.org ([127.0.1.37]) by repo.freebsd.org (8.15.2/8.15.2) with ESMTP id u9QIllgO064602; Wed, 26 Oct 2016 18:47:47 GMT (envelope-from jhb@FreeBSD.org) Received: (from jhb@localhost) by repo.freebsd.org (8.15.2/8.15.2/Submit) id u9QIllRU064601; Wed, 26 Oct 2016 18:47:47 GMT (envelope-from jhb@FreeBSD.org) Message-Id: <201610261847.u9QIllRU064601@repo.freebsd.org> X-Authentication-Warning: repo.freebsd.org: jhb set sender to jhb@FreeBSD.org using -f From: John Baldwin Date: Wed, 26 Oct 2016 18:47:47 +0000 (UTC) To: src-committers@freebsd.org, svn-src-all@freebsd.org, svn-src-head@freebsd.org Subject: svn commit: r307975 - head/sys/i386/i386 X-SVN-Group: head MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-BeenThere: svn-src-all@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: "SVN commit messages for the entire src tree \(except for " user" and " projects" \)" List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 26 Oct 2016 18:47:48 -0000 Author: jhb Date: Wed Oct 26 18:47:47 2016 New Revision: 307975 URL: https://svnweb.freebsd.org/changeset/base/307975 Log: Enable EFER_NXE properly on APs. EFER_NXE is set in the EFER MSR by initializecpu() and must be set on all CPUs in the system. When PG_NX support was added to PAE on i386, the block to enable EFER_NXE was placed in a section of initializecpu() that only runs if 'cpu == CPU_686'. During early boot, locore does an initial pass to set cpu that sets it to CPU_686 on all CPUs later than a Pentium. Later, printcpuinfo() adjusts the 'cpu' variable on PII and later CPUs to one of CPU_PII, CPU_PIII, or CPU_P4. However, printcpuinfo() is called after initializecpu() on the BSP, so the BSP would enable EFER_NXE and pg_nx. The APs execute initializecpu() much later after printcpuinfo() has run. The end result on a modern CPU was that cpu was set to CPU_PIII when the APs invoked initializecpu(), so they did not enable EFER_NXE. As a result, the APs would fault when trying to access any pages marked with PG_NX set. When booting a 2 CPU PAE kernel in bhyve this manifested as a hang before single user mode. The attempt to execute /bin/init tried to copy out the exec strings (argv, etc.) to a non-executable mapping while running on the AP. The instruction kept faulting due to invalid bits in the PTE in an infinite loop. Fix this by moving the code to enable EFER_NXE out of the switch statement on 'cpu' and always doing it if 'amd_feature' supports AMDID_NX. MFC after: 2 weeks Modified: head/sys/i386/i386/initcpu.c Modified: head/sys/i386/i386/initcpu.c ============================================================================== --- head/sys/i386/i386/initcpu.c Wed Oct 26 18:20:06 2016 (r307974) +++ head/sys/i386/i386/initcpu.c Wed Oct 26 18:47:47 2016 (r307975) @@ -753,16 +753,6 @@ initializecpu(void) init_transmeta(); break; } -#if defined(PAE) || defined(PAE_TABLES) - if ((amd_feature & AMDID_NX) != 0) { - uint64_t msr; - - msr = rdmsr(MSR_EFER) | EFER_NXE; - wrmsr(MSR_EFER, msr); - pg_nx = PG_NX; - elf32_nxstack = 1; - } -#endif break; #endif default: @@ -774,6 +764,16 @@ initializecpu(void) cpu_fxsr = hw_instruction_sse = 1; } #endif +#if defined(PAE) || defined(PAE_TABLES) + if ((amd_feature & AMDID_NX) != 0) { + uint64_t msr; + + msr = rdmsr(MSR_EFER) | EFER_NXE; + wrmsr(MSR_EFER, msr); + pg_nx = PG_NX; + elf32_nxstack = 1; + } +#endif } void