From owner-freebsd-stable@FreeBSD.ORG Thu Aug 9 00:27:00 2012 Return-Path: Delivered-To: freebsd-stable@freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:4f8:fff6::34]) by hub.freebsd.org (Postfix) with ESMTP id 813F3106566B; Thu, 9 Aug 2012 00:27:00 +0000 (UTC) (envelope-from mnln.l4@gmail.com) Received: from mail-lpp01m010-f54.google.com (mail-lpp01m010-f54.google.com [209.85.215.54]) by mx1.freebsd.org (Postfix) with ESMTP id C8FDB8FC08; Thu, 9 Aug 2012 00:26:58 +0000 (UTC) Received: by lage12 with SMTP id e12so729311lag.13 for ; Wed, 08 Aug 2012 17:26:57 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :content-type; bh=bWovx7pTANCkskgR/DPjwvvV1qlojF95kktAJTru1YM=; b=J+kiKrRk9zUQ5yebOTn5xgR8CA1aJ3WqbP8h7/cUnoYJymfFkyOsLT2ggqBSmdfclG ymsYFS/5vUgkIhKrrmmsDSCjxCSknz2zhduJ+Q2c6+LNb0E+dqny6H5xBXTP+MPx/sTG duvadyHQ5W0roV6Y4a26pzz+oovkxZUXilbCp+Wg8ieqaCLp2Nn7C47guhxgKC7PeCRN hoF+KcFtM+pikmRl4xkzY8fKTL3Tq5feQqt1Qqs4vMYIZpBnRAR04mngs9LbWYBe1POL rPrmWNZqSc2nzUhuhqRo2Y40YY2i3e52TxCUQT4qzTToi71kGnPSY68OeLohz3ycLc1d tPnQ== MIME-Version: 1.0 Received: by 10.152.104.146 with SMTP id ge18mr2152975lab.7.1344472017161; Wed, 08 Aug 2012 17:26:57 -0700 (PDT) Received: by 10.112.29.132 with HTTP; Wed, 8 Aug 2012 17:26:56 -0700 (PDT) In-Reply-To: References: <201206190754.03765.jhb@freebsd.org> Date: Wed, 8 Aug 2012 17:26:56 -0700 Message-ID: From: "mnln.l4" To: John Baldwin , freebsd-stable@freebsd.org Content-Type: text/plain; charset=UTF-8 Cc: Subject: Re: Kernel panic at early boot time X-BeenThere: freebsd-stable@freebsd.org X-Mailman-Version: 2.1.5 Precedence: list List-Id: Production branch of FreeBSD source code List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Thu, 09 Aug 2012 00:27:00 -0000 I finally have some time to take a closer look at this issue. Yes, it is caused by SMI#. DragonflyBSD has tried to fix the similar problem (see http://gitweb.dragonflybsd.org/dragonfly.git/commitdiff/bb467734fc407e2c2de7f8314c63dd9f708f4df4) But Windows and Linux don't cause such problem on my machine. I compared MP initialization code between FreeBSD, Linux and NetBSD. I believe the problem is FreeBSD doesn't wait for 10ms between IPI_INIT assert and IPI_INIT deassert (though FreeBSD waits for 10ms after IPI_INIT deassert). After inserting 10ms wait time, the issue is solved. BTW, Intel's MP spec 1.4 doesn't explain very well either. Patch for AMD64 (similar change for i386 as well) Index: mp_machdep.c =================================================================== --- mp_machdep.c (revision 239120) +++ mp_machdep.c (working copy) @@ -993,6 +993,8 @@ lapic_ipi_raw(APIC_DEST_DESTFLD | APIC_TRIGMOD_EDGE | APIC_LEVEL_ASSERT | APIC_DESTMODE_PHY | APIC_DELMODE_INIT, apic_id); + DELAY(10000); /* wait ~10mS */ + /* wait for pending status end */ lapic_ipi_wait(-1); For comparison, 1. FreeBSD's ap_start() method: http://fxr.watson.org/fxr/source/amd64/amd64/mp_machdep.c?v=FREEBSD9#L977 2. Linux's wakeup_secondary_cpu_via_init() method: http://lxr.linux.no/linux+v3.5/arch/x86/kernel/smpboot.c#L527 3. NetBSD's x86_ipi_init() method: http://fxr.watson.org/fxr/source/arch/x86/x86/lapic.c?v=NETBSD5#L507, http://fxr.watson.org/fxr/source/arch/xen/x86/cpu.c?v=NETBSD5;im=bigexcerpts#L858 Best, On Fri, Jun 22, 2012 at 6:55 PM, mnln.l4 wrote: > Thanks for explaining the cause! > > > On Tue, Jun 19, 2012 at 4:54 AM, John Baldwin wrote: >> >> On Sunday, June 17, 2012 2:35:14 pm mnln.l4 wrote: >> > I get a kernel panic at early boot time on 9.0-stable (r237150), >> > GENERIC, >> > AMD64. >> > >> > Repro step: >> > 1. Boot, wait for welcome screen. >> > 2. Repeat pressing Enter key rapidly (so kernel is loading, don't stop >> > pressing Enter key). >> > 3. See the following message at early boot >> >> So don't do that. All your key presses are triggering SMI# events that >> are >> interfering with the AP's ability to respond to its startup IPI. There is >> nothing we can do about this in the OS. >> >> -- >> John Baldwin > >