From owner-freebsd-stable@freebsd.org Wed Sep 14 21:24:31 2016 Return-Path: Delivered-To: freebsd-stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 7C8BCB96667 for ; Wed, 14 Sep 2016 21:24:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id 66E55179D for ; Wed, 14 Sep 2016 21:24:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: by mailman.ysv.freebsd.org (Postfix) id 62C5DB96665; Wed, 14 Sep 2016 21:24:31 +0000 (UTC) Delivered-To: stable@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 62673B96664 for ; Wed, 14 Sep 2016 21:24:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from zxy.spb.ru (zxy.spb.ru [195.70.199.98]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 232DF179B; Wed, 14 Sep 2016 21:24:31 +0000 (UTC) (envelope-from slw@zxy.spb.ru) Received: from slw by zxy.spb.ru with local (Exim 4.86 (FreeBSD)) (envelope-from ) id 1bkHfO-000DIH-S6; Thu, 15 Sep 2016 00:24:26 +0300 Date: Thu, 15 Sep 2016 00:24:26 +0300 From: Slawa Olhovchenkov To: Konstantin Belousov Cc: Andriy Gapon , stable@FreeBSD.org Subject: Re: X2APIC support Message-ID: <20160914212426.GI2840@zxy.spb.ru> References: <37f5cebc-3fa1-9e95-5123-f3d8daa3130a@FreeBSD.org> <20160913152240.GE38409@kib.kiev.ua> <20160914113634.GF38409@kib.kiev.ua> <763df55a-4b69-9f7a-1042-0f631a729881@FreeBSD.org> <20160914123313.GC2840@zxy.spb.ru> <966c58d6-db64-2879-6709-0aa596bec24b@FreeBSD.org> <20160914124950.GE2840@zxy.spb.ru> <50cf91d9-072d-76fe-768c-8a6a9515bd93@FreeBSD.org> <20160914160802.GH38409@kib.kiev.ua> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160914160802.GH38409@kib.kiev.ua> User-Agent: Mutt/1.5.24 (2015-08-30) X-SA-Exim-Connect-IP: X-SA-Exim-Mail-From: slw@zxy.spb.ru X-SA-Exim-Scanned: No (on zxy.spb.ru); SAEximRunCond expanded to false X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 14 Sep 2016 21:24:31 -0000 On Wed, Sep 14, 2016 at 07:08:02PM +0300, Konstantin Belousov wrote: > On Wed, Sep 14, 2016 at 05:02:21PM +0300, Andriy Gapon wrote: > > On 14/09/2016 15:49, Slawa Olhovchenkov wrote: > > > MSR_APICBASE = 0x fee00d00 > > > x2APIC is prohibited but turned on by BIOS > > > > Kostik, ^^^^^ > > Well, the following might work, but I have no good idea what to do > when BIOS does handoff with x2APIC enabled and directs us to not > enable it. Switching to xAPIC mode is not an option, I suspect. CPU: Intel(R) Xeon(R) CPU E5-2650 v4 @ 2.20GHz (2200.05-MHz K8-class CPU) Origin="GenuineIntel" Id=0x406f1 Family=0x6 Model=0x4f Stepping=1 Features=0xbfebfbff Features2=0x7ffefbff AMD Features=0x2c100800 AMD Features2=0x121 Structured Extended Features=0x21cbfbb XSAVE Features=0x1 VT-x: PAT,HLT,MTF,PAUSE,EPT,UG,VPID,VID,PostIntr TSC: P-state invariant, performance statistics real memory = 137438953472 (131072 MB) avail memory = 133407973376 (127227 MB) Event timer "LAPIC" quality 600 ACPI APIC Table: boot_cpu_id = 0 FreeBSD/SMP: Multiprocessor System Detected: 24 CPUs FreeBSD/SMP: 2 package(s) x 12 core(s) random: unblocking device. ioapic0 irqs 0-23 on motherboard ioapic1 irqs 24-47 on motherboard ioapic2 irqs 48-71 on motherboard random: entropy device external interface module_register_init: MOD_LOAD (vesa, 0xffffffff807bf2f0, 0) error 19 random: registering fast source Intel Secure Key RNG random: fast provider: "Intel Secure Key RNG" kbd1 at kbdmux0 netmap: loaded module vtvga0: on motherboard cryptosoft0: on motherboard acpi0: on motherboard acpi0: Power Button (fixed) [...] boot OK, w/o any APIC-related messages > diff --git a/sys/x86/acpica/madt.c b/sys/x86/acpica/madt.c > index 241a769..3a93fd6 100644 > --- a/sys/x86/acpica/madt.c > +++ b/sys/x86/acpica/madt.c > @@ -138,7 +138,6 @@ madt_setup_local(void) > > madt = pmap_mapbios(madt_physaddr, madt_length); > if ((cpu_feature2 & CPUID2_X2APIC) != 0) { > - x2apic_mode = 1; > reason = NULL; > > /* > @@ -150,21 +149,17 @@ madt_setup_local(void) > if (dmartbl_physaddr != 0) { > dmartbl = acpi_map_table(dmartbl_physaddr, > ACPI_SIG_DMAR); > - if ((dmartbl->Flags & ACPI_DMAR_X2APIC_OPT_OUT) != 0) { > - x2apic_mode = 0; > + if ((dmartbl->Flags & ACPI_DMAR_X2APIC_OPT_OUT) != 0) > reason = "by DMAR table"; > - } > acpi_unmap_table(dmartbl); > } > if (vm_guest == VM_GUEST_VMWARE) { > vmware_hvcall(VMW_HVCMD_GETVCPU_INFO, p); > if ((p[0] & VMW_VCPUINFO_VCPU_RESERVED) != 0 || > - (p[0] & VMW_VCPUINFO_LEGACY_X2APIC) == 0) { > - x2apic_mode = 0; > - reason = "inside VMWare without intr redirection"; > - } > + (p[0] & VMW_VCPUINFO_LEGACY_X2APIC) == 0) > + reason = > + "inside VMWare without intr redirection"; > } else if (vm_guest == VM_GUEST_XEN) { > - x2apic_mode = 0; > reason = "due to running under XEN"; > } else if (vm_guest == VM_GUEST_NO && > CPUID_TO_FAMILY(cpu_id) == 0x6 && > @@ -184,13 +179,21 @@ madt_setup_local(void) > if (!strcmp(hw_vendor, "LENOVO") || > !strcmp(hw_vendor, > "ASUSTeK Computer Inc.")) { > - x2apic_mode = 0; > reason = > "for a suspected SandyBridge BIOS bug"; > } > freeenv(hw_vendor); > } > } > + if (reason != NULL && lapic_is_x2apic()) { > + if (bootverbose) > + printf("x2APIC should be disabled %s but " > + "already enabled by BIOS; enabling.\n", > + reason); > + reason = NULL; > + } > + if (reason == NULL) > + x2apic_mode = 1; > TUNABLE_INT_FETCH("hw.x2apic_enable", &x2apic_mode); > if (!x2apic_mode && reason != NULL && bootverbose) > printf("x2APIC available but disabled %s\n", reason); > diff --git a/sys/x86/include/apicvar.h b/sys/x86/include/apicvar.h > index 1ddb69e..09c3a63 100644 > --- a/sys/x86/include/apicvar.h > +++ b/sys/x86/include/apicvar.h > @@ -206,6 +206,7 @@ struct apic_ops { > void (*create)(u_int, int); > void (*init)(vm_paddr_t); > void (*xapic_mode)(void); > + bool (*is_x2apic)(void); > void (*setup)(int); > void (*dump)(const char *); > void (*disable)(void); > @@ -268,6 +269,13 @@ lapic_xapic_mode(void) > apic_ops.xapic_mode(); > } > > +static inline bool > +lapic_is_x2apic(void) > +{ > + > + return (apic_ops.is_x2apic()); > +} > + > static inline void > lapic_setup(int boot) > { > diff --git a/sys/x86/x86/local_apic.c b/sys/x86/x86/local_apic.c > index cd774df..d9a3453 100644 > --- a/sys/x86/x86/local_apic.c > +++ b/sys/x86/x86/local_apic.c > @@ -269,6 +269,16 @@ native_lapic_enable_x2apic(void) > wrmsr(MSR_APICBASE, apic_base); > } > > +static bool > +native_lapic_is_x2apic(void) > +{ > + uint64_t apic_base; > + > + apic_base = rdmsr(MSR_APICBASE); > + return ((apic_base & (APICBASE_X2APIC | APICBASE_ENABLED)) == > + (APICBASE_X2APIC | APICBASE_ENABLED)); > +} > + > static void lapic_enable(void); > static void lapic_resume(struct pic *pic, bool suspend_cancelled); > static void lapic_timer_oneshot(struct lapic *); > @@ -329,6 +339,7 @@ struct apic_ops apic_ops = { > .create = native_lapic_create, > .init = native_lapic_init, > .xapic_mode = native_lapic_xapic_mode, > + .is_x2apic = native_lapic_is_x2apic, > .setup = native_lapic_setup, > .dump = native_lapic_dump, > .disable = native_lapic_disable, > diff --git a/sys/x86/xen/xen_apic.c b/sys/x86/xen/xen_apic.c > index 4d7a39b..45c3c18 100644 > --- a/sys/x86/xen/xen_apic.c > +++ b/sys/x86/xen/xen_apic.c > @@ -139,6 +139,13 @@ xen_pv_lapic_disable(void) > > } > > +static bool > +xen_pv_lapic_is_x2apic(void) > +{ > + > + return (false); > +} > + > static void > xen_pv_lapic_eoi(void) > { > @@ -351,6 +358,7 @@ struct apic_ops xen_apic_ops = { > .create = xen_pv_lapic_create, > .init = xen_pv_lapic_init, > .xapic_mode = xen_pv_lapic_disable, > + .is_x2apic = xen_pv_lapic_is_x2apic, > .setup = xen_pv_lapic_setup, > .dump = xen_pv_lapic_dump, > .disable = xen_pv_lapic_disable,