From owner-freebsd-current@freebsd.org Tue Dec 13 11:23:47 2016 Return-Path: Delivered-To: freebsd-current@mailman.ysv.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2001:1900:2254:206a::19:1]) by mailman.ysv.freebsd.org (Postfix) with ESMTP id 907DBC75753 for ; Tue, 13 Dec 2016 11:23:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from kib.kiev.ua (kib.kiev.ua [IPv6:2001:470:d5e7:1::1]) (using TLSv1 with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 379AE1F7 for ; Tue, 13 Dec 2016 11:23:47 +0000 (UTC) (envelope-from kostikbel@gmail.com) Received: from tom.home (kib@localhost [127.0.0.1]) by kib.kiev.ua (8.15.2/8.15.2) with ESMTPS id uBDBNgFh032895 (version=TLSv1 cipher=DHE-RSA-CAMELLIA256-SHA bits=256 verify=NO); Tue, 13 Dec 2016 13:23:42 +0200 (EET) (envelope-from kostikbel@gmail.com) DKIM-Filter: OpenDKIM Filter v2.10.3 kib.kiev.ua uBDBNgFh032895 Received: (from kostik@localhost) by tom.home (8.15.2/8.15.2/Submit) id uBDBNeBY032894; Tue, 13 Dec 2016 13:23:40 +0200 (EET) (envelope-from kostikbel@gmail.com) X-Authentication-Warning: tom.home: kostik set sender to kostikbel@gmail.com using -f Date: Tue, 13 Dec 2016 13:23:40 +0200 From: Konstantin Belousov To: Slawa Olhovchenkov Cc: "A. Wilcox" , freebsd-current@freebsd.org Subject: Re: Enabling NUMA in BIOS stop booting FreeBSD Message-ID: <20161213112340.GT54029@kib.kiev.ua> References: <20161212165457.GI54029@kib.kiev.ua> <20161212171634.GD90287@zxy.spb.ru> <20161212172418.GK54029@kib.kiev.ua> <20161212174311.GE90287@zxy.spb.ru> <20161212183647.GL54029@kib.kiev.ua> <20161212192619.GF90287@zxy.spb.ru> <584F22B1.9000903@Wilcox-Tech.com> <20161212233714.GA90401@zxy.spb.ru> <20161213110535.GS54029@kib.kiev.ua> <20161213111437.GH90287@zxy.spb.ru> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20161213111437.GH90287@zxy.spb.ru> User-Agent: Mutt/1.7.1 (2016-10-04) X-Spam-Status: No, score=-2.0 required=5.0 tests=ALL_TRUSTED,BAYES_00, DKIM_ADSP_CUSTOM_MED,FREEMAIL_FROM,NML_ADSP_CUSTOM_MED autolearn=no autolearn_force=no version=3.4.1 X-Spam-Checker-Version: SpamAssassin 3.4.1 (2015-04-28) on tom.home X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.23 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Tue, 13 Dec 2016 11:23:47 -0000 On Tue, Dec 13, 2016 at 02:14:37PM +0300, Slawa Olhovchenkov wrote: > On Tue, Dec 13, 2016 at 01:05:35PM +0200, Konstantin Belousov wrote: > > > On Tue, Dec 13, 2016 at 02:37:14AM +0300, Slawa Olhovchenkov wrote: > > > On Mon, Dec 12, 2016 at 04:20:33PM -0600, A. Wilcox wrote: > > > > > > > >>>> Try the debugging patch below, which unconditionally disables import of > > > > >>>> previous buffer. To test, you would need to boot, then frob options in > > > > >>>> BIOS, reboot, again frob etc. > > > > >>> > > > > >>> still need test patch? if yes, with BIOS options? > > > > >> Yes, please test the patch. I explained the procedure above. > > > > > > > > > > sorry, i don't know 'frob'. > > > > > what exactly options combination I need test and what about memory test? > > > > > > > > > > > > > > > > > The idea is that when rebooting, stale memory contents remain, but are > > > > corrupted due to interleave. > > > > > > > > "Frob" basically means "mess with". So apply patch, test kernel, > > > > reboot, change NUMA option, reboot again, see if it works, and so on. > > > > Basically repeat your test with the NUMA=on interleave=on, NUMA=off > > > > interleave=on, etc etc. > > > > > > NUMA=on interleave=off booted > > > NUMA=on interleave=on hang > > > > > > I think different combination whatever? > > > > Do you mean, that both patched kernel, and unpatched kernel with the > > memory test enabled, hang when NUMA and interleave options enabled ? > > Unpatched kernel boot with the memory test enabled when NUMA and > interleave options enabled -- I am already reported this. > > patched kernel with the memory test enabled boot too. > > i.e. memory test enabled allow boot in any situation. Then what about was the statement above ? You said that NUMA and interleave on caused hang. Was that on the patched kernel ? > > > Could you enable the options, power down the machine for 10-20 minutes, > > and try to boot ? > > For with kernel and bios options and boot options? > I am have two day befor server put in production for any expirements, > but please, be more clear in what combination need to test. Boot with NUMA enabled and interleave off. Patch kernel with the 'if (1 || ...)' patch. Reboot, enter BIOS setup and enable interleave there. Try to boot - does it boot ? If it did not booted, power machine off for 10 minutes. Power it on, try to boot (with the same patched kernel). Does the machine boot now ?