From owner-svn-src-head@FreeBSD.ORG Tue Mar 31 00:39:00 2015 Return-Path: Delivered-To: svn-src-head@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by hub.freebsd.org (Postfix) with ESMTPS id 186A6DDA; Tue, 31 Mar 2015 00:39:00 +0000 (UTC) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id AE7261DD; Tue, 31 Mar 2015 00:38:59 +0000 (UTC) Received: from tom.home (kostik@localhost [127.0.0.1]) by kib.kiev.ua (8.14.9/8.14.9) with ESMTP id t2V0cofX084079 (version=TLSv1/SSLv3 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 31 Mar 2015 03:38:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.9.2 kib.kiev.ua t2V0cofX084079 Received: (from kostik@localhost) by tom.home (8.14.9/8.14.9/Submit) id t2V0co2t084078; Tue, 31 Mar 2015 03:38:50 +0300 (EEST) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 31 Mar 2015 03:38:50 +0300 From: Konstantin Belousov To: John Baldwin Subject: Re: svn commit: r280866 - in head/sys: amd64/amd64 i386/i386 Message-ID: <20150331003850.GL2379@kib.kiev.ua> References: <201503302013.t2UKDNCo093442@svn.freebsd.org> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <201503302013.t2UKDNCo093442@svn.freebsd.org> User-Agent: Mutt/1.5.23 (2014-03-12) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.0 X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on tom.home Cc: svn-src-head@freebsd.org, svn-src-all@freebsd.org, src-committers@freebsd.org X-BeenThere: svn-src-head@freebsd.org X-Mailman-Version: 2.1.18-1 Precedence: list List-Id: SVN commit messages for the src tree for head/-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 31 Mar 2015 00:39:00 -0000 On Mon, Mar 30, 2015 at 08:13:23PM +0000, John Baldwin wrote: > Author: jhb > Date: Mon Mar 30 20:13:22 2015 > New Revision: 280866 > URL: https://svnweb.freebsd.org/changeset/base/280866 > > Log: > Wait 100 microseconds for a local APIC to dispatch each startup-related IPI > rather than 20. The MP 1.4 specification states in Appendix B.2: > > "A period of 20 microseconds should be sufficient for IPI dispatch to > complete under normal operating conditions". > > (Note that this appears to be separate from the 10 millisecond (INIT) and > 200 microsecond (STARTUP) waits after the IPIs are dispatched.) The > Intel SDM is silent on this issue as far as I can tell. > > At least some hardware requires 60 microseconds as noted in the PR, so > bump this to 100 to be on the safe side. > > PR: 197756 > Reported by: zaphod@berentweb.com > MFC after: 1 week > > Modified: > head/sys/amd64/amd64/mp_machdep.c > head/sys/i386/i386/mp_machdep.c > > Modified: head/sys/amd64/amd64/mp_machdep.c > ============================================================================== > --- head/sys/amd64/amd64/mp_machdep.c Mon Mar 30 20:01:41 2015 (r280865) > +++ head/sys/amd64/amd64/mp_machdep.c Mon Mar 30 20:13:22 2015 (r280866) > @@ -1084,7 +1084,7 @@ ipi_startup(int apic_id, int vector) > */ > lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_LEVEL | > APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_INIT, apic_id); > - lapic_ipi_wait(20); > + lapic_ipi_wait(100); > > /* Explicitly deassert the INIT IPI. */ > lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_LEVEL | > @@ -1104,7 +1104,7 @@ ipi_startup(int apic_id, int vector) > lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | > APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_STARTUP | > vector, apic_id); > - if (!lapic_ipi_wait(20)) > + if (!lapic_ipi_wait(100)) > panic("Failed to deliver first STARTUP IPI to APIC %d", > apic_id); > DELAY(200); /* wait ~200uS */ > @@ -1118,7 +1118,7 @@ ipi_startup(int apic_id, int vector) > lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | > APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_STARTUP | > vector, apic_id); > - if (!lapic_ipi_wait(20)) > + if (!lapic_ipi_wait(100)) > panic("Failed to deliver second STARTUP IPI to APIC %d", > apic_id); I initially thought that some x2APIC issues, which are still not understood, are related to changed timing, and esp. to the fact that lapic_ipi_wait() is nop for x2APIC mode. I have the patch that increased all DELAYs, in particular, the delays after the lapic_ipi_wait()s, by two times. But apparently, that did not helped, and it seems that there are sporadic reports of Linux having similar issues with x2APIC on similar mobile SandyBridge, which are proof-less charged to BIOS bugs. Mostly, my question is, should we increase DELAYS() in addition to lapic_ipi_wait() timeouts ?