From owner-freebsd-current@freebsd.org Tue Dec 13 15:01:49 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C470DC754DA for ; Tue, 13 Dec 2016 15:01:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 504FF1D7F for ; Tue, 13 Dec 2016 15:01:49 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBDF1dHJ085915 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 13 Dec 2016 17:01:39 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBDF1dHJ085915 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBDF1dr6085914; Tue, 13 Dec 2016 17:01:39 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 13 Dec 2016 17:01:39 +0200 From: Konstantin Belousov To: Slawa Olhovchenkov Cc: freebsd-current@freebsd.org Subject: Re: Enabling NUMA in BIOS stop booting FreeBSD Message-ID: <20161213150139.GZ54029@kib.kiev.ua> References: <20161212192619.GF90287@zxy.spb.ru> <584F22B1.9000903@Wilcox-Tech.com> <20161212233714.GA90401@zxy.spb.ru> <20161213110535.GS54029@kib.kiev.ua> <20161213111437.GH90287@zxy.spb.ru> <20161213112340.GT54029@kib.kiev.ua> <20161213124931.GI90287@zxy.spb.ru> <20161213135759.GY54029@kib.kiev.ua> <20161213141114.GJ90287@zxy.spb.ru> <20161213143401.GK90287@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161213143401.GK90287@zxy.spb.ru> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 15:01:49 -0000 On Tue, Dec 13, 2016 at 05:34:01PM +0300, Slawa Olhovchenkov wrote: > On Tue, Dec 13, 2016 at 05:11:14PM +0300, Slawa Olhovchenkov wrote: > > > On Tue, Dec 13, 2016 at 03:57:59PM +0200, Konstantin Belousov wrote: > > > > > On Tue, Dec 13, 2016 at 03:49:32PM +0300, Slawa Olhovchenkov wrote: > > > > > Boot with NUMA enabled and interleave off. > > > > > > > > Already with patched kernel > > > > > > > > > Patch kernel with the 'if (1 || ...)' patch. > > > > > Reboot, enter BIOS setup and enable interleave there. > > > > > Try to boot - does it boot ? > > > > > > > > No. > > > > > > > > > If it did not booted, power machine off for 10 minutes. > > > > > > > > OK > > > > > > > > > Power it on, try to boot (with the same patched kernel). > > > > > Does the machine boot now ? > > > > > > > > Don't boot. > > > > > > I am really puzzled. In other words, touching all memory causes the > > > msgbuf to not hang. > > > > yes > > > > > Can you try one more experiment ? > > > Take the patch below, apply it. > > > >From the config where interleave is disabled, install new kernel. > > > Reboot, enter BIOS setup and enable interleave. > > > Set late_console to zero in loader. > > > Do not enable memory test. > > > Boot the patched kernel. > > > Kernel must hang, according to your previous reports. > > > I want to see the console log. > > > > Hmm. I am [already] show output from ddb, and guess kernel will be > > hang at first wirte to *mbp, i.e. you don't see any in console log. > > > > OK, anyway I am try this pacth. > > KDB: debugger backends: ddb > KDB: current backend: ddb > SMAP type=01 base=0000000000000000 len=0000000000099c00 > SMAP type=02 base=0000000000099c00 len=0000000000006400 > SMAP type=02 base=00000000000e0000 len=0000000000020000 > SMAP type=01 base=0000000000100000 len=000000007906b000 > SMAP type=02 base=000000007916b000 len=0000000000936000 > SMAP type=04 base=0000000079aa1000 len=0000000000509000 > SMAP type=02 base=0000000079faa000 len=0000000002056000 > SMAP type=01 base=0000000100000000 len=0000001f80000000 > SMAP type=02 base=000000007c000000 len=0000000014000000 > SMAP type=02 base=00000000fed1c000 len=0000000000029000 > SMAP type=02 base=00000000ff000000 len=0000000001000000 > XXX1 > YYY1 > ZZZ1 Ok, please do exactly the same testing with the following patch. If the patched kernel boots succesfully, or if the patched kernel boots further, I will provide one more, last patch, to test. diff --git a/sys/kern/subr_msgbuf.c b/sys/kern/subr_msgbuf.c index f275aef3b4f..1be7a629f65 100644 --- a/sys/kern/subr_msgbuf.c +++ b/sys/kern/subr_msgbuf.c @@ -67,14 +67,19 @@ msgbuf_init(struct msgbuf *mbp, void *ptr, int size) mbp->msg_ptr = ptr; mbp->msg_size = size; mbp->msg_seqmod = SEQMOD(size); +printf("YYY1\n"); msgbuf_clear(mbp); +printf("YYY2\n"); mbp->msg_magic = MSG_MAGIC; mbp->msg_lastpri = -1; mbp->msg_flags = 0; +printf("YYY3\n"); bzero(&mbp->msg_lock, sizeof(mbp->msg_lock)); mtx_init(&mbp->msg_lock, "msgbuf", NULL, MTX_SPIN); +printf("YYY4\n"); } + /* * Reinitialize a message buffer, retaining its previous contents if * the size and checksum are correct. If the old contents cannot be @@ -85,8 +90,10 @@ msgbuf_reinit(struct msgbuf *mbp, void *ptr, int size) { u_int cksum; - if (mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) { + if (1 || mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) { +printf("XXX1\n"); msgbuf_init(mbp, ptr, size); +printf("XXX2\n"); return; } mbp->msg_seqmod = SEQMOD(size); @@ -117,10 +124,12 @@ void msgbuf_clear(struct msgbuf *mbp) { +printf("ZZZ1\n"); bzero(mbp->msg_ptr, mbp->msg_size); mbp->msg_wseq = 0; mbp->msg_rseq = 0; mbp->msg_cksum = 0; +printf("ZZZ2\n"); } /* diff --git a/sys/kern/subr_prf.c b/sys/kern/subr_prf.c index e78863830c7..435412d55ea 100644 --- a/sys/kern/subr_prf.c +++ b/sys/kern/subr_prf.c @@ -998,6 +998,8 @@ msgbufinit(void *ptr, int size) char *cp; static struct msgbuf *oldp = NULL; +printf("TTT1 %p %p %x\n", ptr, (char *)ptr + size - sizeof(*msgbufp), size); +bzero(ptr, size); size -= sizeof(*msgbufp); cp = (char *)ptr; msgbufp = (struct msgbuf *)(cp + size);