From owner-freebsd-current@freebsd.org Fri Sep 20 15:27:54 2019 Return-Path: Delivered-To: freebsd-current@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id D5BCDFFB49 for ; Fri, 20 Sep 2019 15:27:54 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from mail.ijs.si (mail.ijs.si [IPv6:2001:1470:ff80::25]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 46ZczY4lrLz4Jvy for ; Fri, 20 Sep 2019 15:27:53 +0000 (UTC) (envelope-from Mark.Martinec+freebsd@ijs.si) Received: from amavis-ori.ijs.si (localhost [IPv6:::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by mail.ijs.si (Postfix) with ESMTPS id 46ZczT0jPbzr4D for ; Fri, 20 Sep 2019 17:27:49 +0200 (CEST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=ijs.si; h= user-agent:message-id:references:in-reply-to:organization :subject:subject:from:from:date:date:content-transfer-encoding :content-type:content-type:mime-version:received:received :received:received; s=jakla4; t=1568993265; x=1571585266; bh=3xW pXTcRBN4wLwkYiue71UzGLEeLYTfK0Im4pdxNV1g=; b=fk2A9q6lpwpZvUZhwgl Qw1NG5KjGjDttUnEtnTDAgP6CppEOzXHQxfWYJDhSIzF5SoJnvu5A9vFkNJ1nnCL lbPOlp62Zhe1WyAQsu7Z+p30OU5j2GdZ/449cYoU088AFHn5rok1Z/utGTOF+2h+ 15n/UFQA6+SONFHWyR91sVqA= X-Virus-Scanned: amavisd-new at ijs.si Received: from mail.ijs.si ([IPv6:::1]) by amavis-ori.ijs.si (mail.ijs.si [IPv6:::1]) (amavisd-new, port 10026) with LMTP id g7gH-QCuig96 for ; Fri, 20 Sep 2019 17:27:45 +0200 (CEST) Received: from mildred.ijs.si (mailbox.ijs.si [IPv6:2001:1470:ff80::143:1]) by mail.ijs.si (Postfix) with ESMTP id 46ZczP2DjPzr4C for ; Fri, 20 Sep 2019 17:27:45 +0200 (CEST) Received: from nabiralnik.ijs.si (nabiralnik.ijs.si [IPv6:2001:1470:ff80::80:16]) by mildred.ijs.si (Postfix) with ESMTP id 46ZczN6y4czkp for ; Fri, 20 Sep 2019 17:27:44 +0200 (CEST) Received: from neli.ijs.si (2001:1470:ff80:88:21c:c0ff:feb1:8c91) by webmail.ijs.si with HTTP (HTTP/2.0 POST); Fri, 20 Sep 2019 17:27:44 +0200 MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII; format=flowed Content-Transfer-Encoding: 7bit Date: Fri, 20 Sep 2019 17:27:44 +0200 From: Mark Martinec To: freebsd-current@freebsd.org Subject: Re: Lockdown adaX numbers to allow booting ? Organization: Jozef Stefan Institute In-Reply-To: <20190920143531.GB38096@zxy.spb.ru> References: <20190919140219.GE2863@home.opsec.eu> <7E0AE025-596C-457E-BC40-41217857A3CD@me.com> <20190919155713.GG2863@home.opsec.eu> <3C855A39-BF79-4430-98CB-CB9174768E11@freebsd.org> <20190920143531.GB38096@zxy.spb.ru> Message-ID: <9ba9867986faac1ce5068b481669d5f8@ijs.si> X-Sender: Mark.Martinec+freebsd@ijs.si User-Agent: Roundcube Webmail/1.3.1 X-Rspamd-Queue-Id: 46ZczY4lrLz4Jvy X-Spamd-Bar: -- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=ijs.si header.s=jakla4 header.b=fk2A9q6l; dmarc=pass (policy=none) header.from=ijs.si; spf=pass (mx1.freebsd.org: domain of Mark.Martinec@ijs.si designates 2001:1470:ff80::25 as permitted sender) smtp.mailfrom=Mark.Martinec@ijs.si X-Spamd-Result: default: False [-2.46 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-1.000,0]; R_DKIM_ALLOW(-0.20)[ijs.si:s=jakla4]; RCVD_COUNT_FIVE(0.00)[6]; FROM_HAS_DN(0.00)[]; R_SPF_ALLOW(-0.20)[+ip6:2001:1470:ff80::25]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; PREVIOUSLY_DELIVERED(0.00)[freebsd-current@freebsd.org]; TO_DN_NONE(0.00)[]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; RCPT_COUNT_ONE(0.00)[1]; HAS_ORG_HEADER(0.00)[]; RCVD_TLS_LAST(0.00)[]; DKIM_TRACE(0.00)[ijs.si:+]; DMARC_POLICY_ALLOW(-0.50)[ijs.si,none]; RCVD_IN_DNSWL_NONE(0.00)[5.2.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.8.f.f.0.7.4.1.1.0.0.2.list.dnswl.org : 127.0.11.0]; IP_SCORE(-0.46)[asn: 2107(-2.30), country: SI(0.01)]; FROM_EQ_ENVFROM(0.00)[]; SUBJECT_ENDS_QUESTION(1.00)[]; MIME_TRACE(0.00)[0:+]; ASN(0.00)[asn:2107, ipnet:2001:1470::/32, country:SI]; TAGGED_FROM(0.00)[freebsd]; MID_RHS_MATCH_FROM(0.00)[]; DWL_DNSWL_NONE(0.00)[ijs.si.dwl.dnswl.org : 127.0.11.0] X-BeenThere: freebsd-current@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussions about the use of FreeBSD-current List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Fri, 20 Sep 2019 15:27:54 -0000 Kurt Jaeger writes: > The problem is that if all 10 disks are connected, the system > looses track from where it should boot and fails to boot (serial boot > log): > > Consoles: internal video/keyboard serial port > BTX loader 1.00 BTX version is 1.02 > Consoles: internal video/keyboard serial port > BIOS drive C: is disk0 > BIOS drive D: is disk1 > BIOS drive E: is disk2 > BIOS drive F: is disk3 > BIOS drive G: is disk4 > BIOS drive H: is disk5 > BIOS drive I: is disk6 > BIOS drive J: is disk7 > BIOS drive K: is disk8 > BIOS drive L: is disk9 > // > [...] > The solution right now is this to unplug all disks of the 'bck' pool, > reboot, and re-insert the data disks after the boot is finished. > [...] > No gpart on the bck pool, raw drives. This sounds very much like my experience: 2018-11-29, Boot loader stuck after first stage upgrading 11.2 to 12.0-RC2 https://lists.freebsd.org/pipermail/freebsd-stable/2018-November/090129.html https://lists.freebsd.org/pipermail/freebsd-stable/2018-December/090159.html I now have three SuperMicro machines which are unable to boot after upgrading 11.2 to 12.0. After unsuccessfully fiddling with boot loaders, I have reverted two back to 11.2 (which boots and works fine again), and the third one is now at 12.0 but needs the boot hack as described by Kurt, i.e. pull out half the disks (of the 'data' pool), boot the system, plug the disks back in and zfs mount the remaining pool. Considering that the 11.2 boots and works fine on these machines, I consider it a btx loader failure and not a BIOS issue. What is common with these three machines is that they have one pool on raw devices for historical reasons (not on gpt partitions). My guess is that the new loader gets confused by these raw disks. Mark