From owner-freebsd-current@freebsd.org Mon Dec 12 17:24:27 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id DDE8AC73455 for ; Mon, 12 Dec 2016 17:24:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (mailman.ysv.freebsd.org [IPv6:2001:1900:2254:206a::50:5]) by mx1.freebsd.org (Postfix) with ESMTP id C4312F49 for ; Mon, 12 Dec 2016 17:24:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id C38E3C73454; Mon, 12 Dec 2016 17:24:27 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id C33CFC73452 for ; Mon, 12 Dec 2016 17:24:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 50EF8F48 for ; Mon, 12 Dec 2016 17:24:27 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBCHOIuM086509 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Mon, 12 Dec 2016 19:24:18 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBCHOIuM086509 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBCHOI20086508; Mon, 12 Dec 2016 19:24:18 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Mon, 12 Dec 2016 19:24:18 +0200 From: Konstantin Belousov To: Slawa Olhovchenkov Cc: current@freebsd.org Subject: Re: Enabling NUMA in BIOS stop booting FreeBSD Message-ID: <20161212172418.GK54029@kib.kiev.ua> References: <20161211182111.GH31311@zxy.spb.ru> <20161211191626.GI31311@zxy.spb.ru> <20161211192656.GZ54029@kib.kiev.ua> <20161211194559.GJ31311@zxy.spb.ru> <20161211200654.GA54029@kib.kiev.ua> <20161211204709.GK31311@zxy.spb.ru> <20161212145418.GF54029@kib.kiev.ua> <20161212162152.GC90287@zxy.spb.ru> <20161212165457.GI54029@kib.kiev.ua> <20161212171634.GD90287@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161212171634.GD90287@zxy.spb.ru> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 12 Dec 2016 17:24:28 -0000 On Mon, Dec 12, 2016 at 08:16:34PM +0300, Slawa Olhovchenkov wrote: > On Mon, Dec 12, 2016 at 06:54:57PM +0200, Konstantin Belousov wrote: > > > On Mon, Dec 12, 2016 at 07:21:53PM +0300, Slawa Olhovchenkov wrote: > > > On Mon, Dec 12, 2016 at 04:54:18PM +0200, Konstantin Belousov wrote: > > > > > > > On Sun, Dec 11, 2016 at 11:47:09PM +0300, Slawa Olhovchenkov wrote: > > > > > Booting... > > > > > ESC[01;00H8+0x8+0xe9bdc] KDB: debugger backends: ddb > > > > > KDB: current backend: ddb > > > > > exit from kdb_init > > > > > KDB: enter: Boot flags requested debugger > > > > > [ thread pid 0 tid 0 ] > > > > - remove any video consoles from the HEAD kernel config, i.e. sc/vt and > > > > vga/efi, > > > > - do not use boot -d, > > > > - use serial console (IPMI SOL qualifies), > > > > - set late console to 0, > > > > and show me the verbose dmesg of such boot with the BIOS options > > > > which cause troubles. > > > > > > Booting... > > > KDB: debugger backends: ddb > > > KDB: current backend: ddb > > > SMAP type=01 base=0000000000000000 len=0000000000099c00 > > > SMAP type=02 base=0000000000099c00 len=0000000000006400 > > > SMAP type=02 base=00000000000e0000 len=0000000000020000 > > > SMAP type=01 base=0000000000100000 len=000000007906b000 > > > SMAP type=02 base=000000007916b000 len=0000000000936000 > > > SMAP type=04 base=0000000079aa1000 len=0000000000509000 > > > SMAP type=02 base=0000000079faa000 len=0000000002056000 > > > SMAP type=01 base=0000000100000000 len=0000001f80000000 > > > SMAP type=02 base=000000007c000000 len=0000000014000000 > > > SMAP type=02 base=00000000fed1c000 len=0000000000029000 > > > SMAP type=02 base=00000000ff000000 len=0000000001000000 > > > > > > This is all. No more. > > When you switch between variations of the NUMA enablement options, do > > you just reboot the machine or do you sometimes physically turn it off ? > > just reboot an 'power reset' via kvm client (memory preserved, i mean) > > > Try to enable memtest, with the hw.memtest.tests=1 loader variable. > > Does it change things ? > > System booted, dmesg is http://zxy.spb.ru/dmesg.numa I suspect now the reversed situation could take place, the non-interleaved option would cause hang. My current guess is that memory content is preserved but swizzled by the cache line sized chunks. So that the msgbuf header, left after the previous boot, looks correct while the real buffer content is shuffled. Try the debugging patch below, which unconditionally disables import of previous buffer. To test, you would need to boot, then frob options in BIOS, reboot, again frob etc. diff --git a/sys/kern/subr_msgbuf.c b/sys/kern/subr_msgbuf.c index f275aef3b4f..d45ef502204 100644 --- a/sys/kern/subr_msgbuf.c +++ b/sys/kern/subr_msgbuf.c @@ -85,7 +85,7 @@ msgbuf_reinit(struct msgbuf *mbp, void *ptr, int size) { u_int cksum; - if (mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) { + if (1 || mbp->msg_magic != MSG_MAGIC || mbp->msg_size != size) { msgbuf_init(mbp, ptr, size); return; }