From owner-freebsd-current@freebsd.org Sun Dec 11 20:06:59 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id CAE3EC72F04 for ; Sun, 11 Dec 2016 20:06:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from mailman.ysv.freebsd.org (unknown [127.0.1.3]) by mx1.freebsd.org (Postfix) with ESMTP id B2BA71E07 for ; Sun, 11 Dec 2016 20:06:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: by mailman.ysv.freebsd.org (Postfix) id AF160C72F02; Sun, 11 Dec 2016 20:06:59 +0000 (UTC) Delivered-To: current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id AD0C1C72F01 for ; Sun, 11 Dec 2016 20:06:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 3E16B1E06 for ; Sun, 11 Dec 2016 20:06:59 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBBK6sGp071952 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Sun, 11 Dec 2016 22:06:54 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBBK6sGp071952 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBBK6sYG071951; Sun, 11 Dec 2016 22:06:54 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Sun, 11 Dec 2016 22:06:54 +0200 From: Konstantin Belousov To: Slawa Olhovchenkov Cc: current@freebsd.org Subject: Re: Enabling NUMA in BIOS stop booting FreeBSD Message-ID: <20161211200654.GA54029@kib.kiev.ua> References: <20161126092124.GM57876@zxy.spb.ru> <20161126155747.GF54029@kib.kiev.ua> <20161211182111.GH31311@zxy.spb.ru> <20161211191626.GI31311@zxy.spb.ru> <20161211192656.GZ54029@kib.kiev.ua> <20161211194559.GJ31311@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161211194559.GJ31311@zxy.spb.ru> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sun, 11 Dec 2016 20:06:59 -0000 On Sun, Dec 11, 2016 at 10:45:59PM +0300, Slawa Olhovchenkov wrote: > On Sun, Dec 11, 2016 at 09:26:56PM +0200, Konstantin Belousov wrote: > > > On Sun, Dec 11, 2016 at 10:16:26PM +0300, Slawa Olhovchenkov wrote: > > > On Sun, Dec 11, 2016 at 09:21:11PM +0300, Slawa Olhovchenkov wrote: > > > > > > > On Sat, Nov 26, 2016 at 05:57:47PM +0200, Konstantin Belousov wrote: > > > > > > > > > On Sat, Nov 26, 2016 at 12:21:24PM +0300, Slawa Olhovchenkov wrote: > > > > > > I am try to enable NUMA in bios and can't boot FreeBSD. > > > > > > Boot stoped after next messages: > > > > > > > > > > > > === > > > > > > Booting... > > > > > > KDB: debugger backends: ddb > > > > > > KDB: current backend: ddb > > > > > So at least the hammer_time() has a chance to initialize the console. > > > > > Do you have serial console ? Set the loader tunable debug.late_console > > > > > to 1 and see if any NMI reaction appear. > > > > > > > > > > > === > > > > > > > > > > > > This is verbose boot. > > > > > > No reaction to ~^B, NMI. > > > > > > > > > > > > Same for head and 10.3-RELEASE. > > > > > > > > > > > > Hardware is Supermicro X10DRi, Dual E5-2650v4, 256GB RAM. > > > > > Is there a BIOS option for 'on-chip cluster' or 'HPC computing' ? > > > > > What if you try to frob it ? > > > > > > > > > > > > > > > > > On slight different hardware > > > > > > (Supermicro X10DRi w/ old BIOS, Dual E5-2640v3, 128GB RAM) > > > > > > 10.3 boot ok w/ BIOS NUMA enabled. > > > > > > > > > > I think the only way to debug this is to add printf() lines to hammer_time() > > > > > to see where does it break. Note that amd64_kdb_init() call succeeded, > > > > > so you can start bisect the code from there. > > > > > > > > > > > > > Hang in next two lines: > > > > > > > > msgbufinit(msgbufp, msgbufsize); > > > > fpuinit(); > > > > Can you show the verbose dmesg up to the failure point ? > > In particular, the SMAP lines should be relevant. > > KDB: debugger backends: ddb > KDB: current backend: ddb > exit from kdb_init > KDB: enter: Boot flags requested debugger > [ thread pid 0 tid 0 ] > Stopped at 0xffffffff805361eb = kdb_enter+0x3b: movq > $0,0xffffffff80dcef20 = kdb_why > > No SMAP print, boot_verbose enabled. The log above shows that you used boot -d. What are the pristine boot messages, with debug.late_console set to 0, of course ? Hm, might be also show the output of the 'smap' and 'memmap' output from the loader. If any of them worked, could be useful to see the same output with the NUMA option disabled as well.