From owner-freebsd-current@freebsd.org Mon Dec 12 18:36:53 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id D039FC74F16 for ; Mon, 12 Dec 2016 18:36:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id B5DA514CB for ; Mon, 12 Dec 2016 18:36:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id B248EC74F15; Mon, 12 Dec 2016 18:36:53 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id B1F0EC74F14 for ; Mon, 12 Dec 2016 18:36:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3FE3B14CA for ; Mon, 12 Dec 2016 18:36:53 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBCIalNg004940 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Dec 2016 20:36:47 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBCIalNg004940 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBCIalJA004939; Mon, 12 Dec 2016 20:36:47 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 12 Dec 2016 20:36:47 +0200 From: Konstantin Belousov To: Slawa Olhovchenkov Cc: current@freebsd.org Subject: Re: Enabling NUMA in BIOS stop booting FreeBSD Message-ID: <20161212183647.GL54029@kib.kiev.ua> References: <20161211192656.GZ54029@kib.kiev.ua> <20161211194559.GJ31311@zxy.spb.ru> <20161211200654.GA54029@kib.kiev.ua> <20161211204709.GK31311@zxy.spb.ru> <20161212145418.GF54029@kib.kiev.ua> <20161212162152.GC90287@zxy.spb.ru> <20161212165457.GI54029@kib.kiev.ua> <20161212171634.GD90287@zxy.spb.ru> <20161212172418.GK54029@kib.kiev.ua> <20161212174311.GE90287@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161212174311.GE90287@zxy.spb.ru> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 18:36:54 -0000 On Mon, Dec 12, 2016 at 08:43:11PM +0300, Slawa Olhovchenkov wrote: > On Mon, Dec 12, 2016 at 07:24:18PM +0200, Konstantin Belousov wrote: > > > On Mon, Dec 12, 2016 at 08:16:34PM +0300, Slawa Olhovchenkov wrote: > > > On Mon, Dec 12, 2016 at 06:54:57PM +0200, Konstantin Belousov wrote: > > > > > > > On Mon, Dec 12, 2016 at 07:21:53PM +0300, Slawa Olhovchenkov wrote: > > > > > On Mon, Dec 12, 2016 at 04:54:18PM +0200, Konstantin Belousov wrote: > > > > > > > > > > > On Sun, Dec 11, 2016 at 11:47:09PM +0300, Slawa Olhovchenkov wrote: > > > > > > > Booting... > > > > > > > ESC[01;00H8+0x8+0xe9bdc] KDB: debugger backends: ddb > > > > > > > KDB: current backend: ddb > > > > > > > exit from kdb_init > > > > > > > KDB: enter: Boot flags requested debugger > > > > > > > [ thread pid 0 tid 0 ] > > > > > > - remove any video consoles from the HEAD kernel config, i.e. sc/vt and > > > > > > vga/efi, > > > > > > - do not use boot -d, > > > > > > - use serial console (IPMI SOL qualifies), > > > > > > - set late console to 0, > > > > > > and show me the verbose dmesg of such boot with the BIOS options > > > > > > which cause troubles. > > > > > > > > > > Booting... > > > > > KDB: debugger backends: ddb > > > > > KDB: current backend: ddb > > > > > SMAP type=01 base=0000000000000000 len=0000000000099c00 > > > > > SMAP type=02 base=0000000000099c00 len=0000000000006400 > > > > > SMAP type=02 base=00000000000e0000 len=0000000000020000 > > > > > SMAP type=01 base=0000000000100000 len=000000007906b000 > > > > > SMAP type=02 base=000000007916b000 len=0000000000936000 > > > > > SMAP type=04 base=0000000079aa1000 len=0000000000509000 > > > > > SMAP type=02 base=0000000079faa000 len=0000000002056000 > > > > > SMAP type=01 base=0000000100000000 len=0000001f80000000 > > > > > SMAP type=02 base=000000007c000000 len=0000000014000000 > > > > > SMAP type=02 base=00000000fed1c000 len=0000000000029000 > > > > > SMAP type=02 base=00000000ff000000 len=0000000001000000 > > > > > > > > > > This is all. No more. > > > > When you switch between variations of the NUMA enablement options, do > > > > you just reboot the machine or do you sometimes physically turn it off ? > > > > > > just reboot an 'power reset' via kvm client (memory preserved, i mean) > > > > > > > Try to enable memtest, with the hw.memtest.tests=1 loader variable. > > > > Does it change things ? > > > > > > System booted, dmesg is http://zxy.spb.ru/dmesg.numa > > I suspect now the reversed situation could take place, the non-interleaved > > option would cause hang. > > No, also booted, dmesg http://zxy.spb.ru/dmesg.numa-ninter I mean, it could hang if memory testing is not enabled. > > > My current guess is that memory content is preserved but swizzled by > > the cache line sized chunks. So that the msgbuf header, left after the > > previous boot, looks correct while the real buffer content is shuffled. > > > > Try the debugging patch below, which unconditionally disables import of > > previous buffer. To test, you would need to boot, then frob options in > > BIOS, reboot, again frob etc. > > still need test patch? if yes, with BIOS options? Yes, please test the patch. I explained the procedure above. > > > diff --git a/sys/kern/subr_msgbuf.c b/sys/kern/subr_msgbuf.c > > index f275aef3b4f..d45ef502204 100644 > > --- a/sys/kern/subr_msgbuf.c > > +++ b/sys/kern/subr_msgbuf.c > > @@ -85,7 +85,7 @@ msgbuf_reinit(struct msgbuf *mbp, void *ptr, int size) > > { > > u_int cksum; > > > > - if (mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) { > > + if (1 || mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) { > > msgbuf_init(mbp, ptr, size); > > return; > > }