From owner-freebsd-xen@freebsd.org Wed May 23 08:16:03 2018 Return-Path: Delivered-To: freebsd-xen@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 2A2F7EF4728 for ; Wed, 23 May 2018 08:16:03 +0000 (UTC) (envelope-from prvs=674474304=roger.pau@citrix.com) Received: from SMTP.EU.CITRIX.COM (smtp.eu.citrix.com [185.25.65.24]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client CN "mail.citrix.com", Issuer "DigiCert SHA2 Secure Server CA" (verified OK)) by mx1.freebsd.org (Postfix) with ESMTPS id 571857DD8A; Wed, 23 May 2018 08:16:02 +0000 (UTC) (envelope-from prvs=674474304=roger.pau@citrix.com) X-IronPort-AV: E=Sophos;i="5.49,432,1520899200"; d="scan'208";a="73570958" Date: Wed, 23 May 2018 10:15:47 +0200 From: Roger Pau =?utf-8?B?TW9ubsOp?= To: Pratyush Yadav CC: FreeBSD-Xen , Akshay Jaggi , Edward Napierala Subject: Re: Xen kernel fails to boot, d0v1 triple fault looks like the cuplrit Message-ID: <20180523081547.2vvthg42vmphvbex@MacBook-Pro-de-Roger.local> References: <20180519081030.qhzyjdrpwcekmcac@MacBook-Pro-de-Roger.local> <20180521090310.c46eexnwe4c7w62x@MacBook-Pro-de-Roger.local> MIME-Version: 1.0 Content-Type: text/plain; charset="iso-8859-1" Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: User-Agent: NeoMutt/20180323 X-ClientProxiedBy: AMSPEX02CAS02.citrite.net (10.69.22.113) To AMSPEX02CL02.citrite.net (10.69.22.126) X-BeenThere: freebsd-xen@freebsd.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Discussion of the freebsd port to xen - implementation and usage List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Wed, 23 May 2018 08:16:03 -0000 On Wed, May 23, 2018 at 12:57:59PM +0530, Pratyush Yadav wrote: > On Mon, May 21, 2018 at 2:33 PM, Roger Pau Monné > wrote: > > Please try to avoid top posting. > > Sorry, I didn't know. I Googled top posting just now, and realized it is > inconvenient for the reader. > > > Hm, it seems like dbg_stack is not properly allocated. Can you please > > try the above debug patch and paste the boot log? > > > > ---8<--- > > diff --git a/sys/amd64/amd64/mp_machdep.c b/sys/amd64/amd64/mp_machdep.c > > index 301461420874..8854242b4bf5 100644 > > --- a/sys/amd64/amd64/mp_machdep.c > > +++ b/sys/amd64/amd64/mp_machdep.c > > @@ -304,6 +304,7 @@ init_secondary(void) > > > > /* Save the per-cpu pointer for use by the DB# handler. */ > > np = ((struct nmi_pcpu *) &dbg_stack[PAGE_SIZE]) - 1; > > +printf("ID %d dbg_stack: %p per-cpu: %p\n", cpu, dbg_stack, np); > > np->np_pcpu = (register_t) pc; > > > > wrmsr(MSR_FSBASE, 0); /* User value */ > > @@ -403,6 +404,7 @@ native_start_all_aps(void) > > M_WAITOK | M_ZERO); > > dbg_stack = (char *)kmem_malloc(kernel_arena, PAGE_SIZE, > > M_WAITOK | M_ZERO); > > +printf("allocated dbg_stack: %p\n", dbg_stack); > > dpcpu = (void *)kmem_malloc(kernel_arena, DPCPU_SIZE, > > M_WAITOK | M_ZERO); > > > > I applied your patch. The boot stops at the same spot. Also, I can't see > any of the two print messages added in the patch. I booted from a Live CD > to look at the logs, but they are not stored. I checked dmesg.boot and > messages, and /var/log/xen/ none of them have the logs. They all have logs > of the previous boot. It's too early for the logs to be stored anywhere. The point where you get the crash is when the APs are started, which is way before FreeBSD starts proving for disk devices. Can you please try the patch below? Thanks, Roger. ---8<--- diff --git a/sys/x86/xen/pv.c b/sys/x86/xen/pv.c index 54184898e9bf..52391e2e7c08 100644 --- a/sys/x86/xen/pv.c +++ b/sys/x86/xen/pv.c @@ -113,6 +113,7 @@ static int xen_pv_start_all_aps(void); extern char *doublefault_stack; extern char *mce_stack; extern char *nmi_stack; +extern char *dbg_stack; #endif /* @@ -329,6 +330,8 @@ start_xen_ap(int cpu) (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO); nmi_stack = (char *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO); + dbg_stack = + (void *)kmem_malloc(kernel_arena, PAGE_SIZE, M_WAITOK | M_ZERO); dpcpu = (void *)kmem_malloc(kernel_arena, DPCPU_SIZE, M_WAITOK | M_ZERO);