From owner-freebsd-fs@freebsd.org Mon Oct 28 23:54:51 2019 Return-Path: Delivered-To: freebsd-fs@mailman.nyi.freebsd.org Received: from mx1.freebsd.org (mx1.freebsd.org [IPv6:2610:1c1:1:606c::19:1]) by mailman.nyi.freebsd.org (Postfix) with ESMTP id C260D1604D4 for ; Mon, 28 Oct 2019 23:54:51 +0000 (UTC) (envelope-from johannes@jo-t.de) Received: from email.jo-t.de (seppel.jo-t.de [188.68.55.77]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (Client did not present a certificate) by mx1.freebsd.org (Postfix) with ESMTPS id 472BQy4cCCz4fKd; Mon, 28 Oct 2019 23:54:49 +0000 (UTC) (envelope-from johannes@jo-t.de) DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=jo-t.de; s=jot0; t=1572306882; bh=hkRW1856O0NolgvXtqNmzVuHCs3M4Qg4XZOyu9TwbhI=; h=Subject:To:References:From:Message-ID:Date:MIME-Version:From; b=6hUdOrERClAJ4/7DDsB5MEDQBveEQBng54kUpkopKP9b9L7IJPthoQf7i5phTIl4O aJVR6gwgjOPZXXRZ2YhlcygzZZoebVcIwj0UnNof2+vuRFeUhEpbr/6h3WOR0MToD2 JCAvnUliVraAbyA6obsqmA6OJVON+V9/FxH9h1nI76GSgho9Iq0TdH4wIPQlANLQJ3 lq7Z9CzmkcyHOKHd5yTzXBy2/Z1jrwFZMmpX2KZgieKhBaxRatJyTKh2NAXLQyrYt6 16uCc+5MYAQDhlzOZDk+ZSZdToD+5htZnx98TMyXzub8ksYhnLkLodh/GBZdO0N2cS OxnHGv4pDjeOA== Subject: Re: zpool attach/resilver trashes bsdlabel bootcode To: Andriy Gapon , freebsd-fs@freebsd.org References: From: Johannes Totz Message-ID: <2ca27aa3-bec8-db97-20fb-bfdc37ad7ef7@jo-t.de> Date: Mon, 28 Oct 2019 23:54:41 +0000 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-GB Content-Transfer-Encoding: 7bit X-Rspamd-Queue-Id: 472BQy4cCCz4fKd X-Spamd-Bar: --- Authentication-Results: mx1.freebsd.org; dkim=pass header.d=jo-t.de header.s=jot0 header.b=6hUdOrER; dmarc=pass (policy=none) header.from=jo-t.de; spf=pass (mx1.freebsd.org: domain of johannes@jo-t.de designates 188.68.55.77 as permitted sender) smtp.mailfrom=johannes@jo-t.de X-Spamd-Result: default: False [-3.12 / 15.00]; ARC_NA(0.00)[]; NEURAL_HAM_MEDIUM(-1.00)[-0.999,0]; R_DKIM_ALLOW(-0.20)[jo-t.de:s=jot0]; FROM_HAS_DN(0.00)[]; TO_DN_SOME(0.00)[]; R_SPF_ALLOW(-0.20)[+mx]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; NEURAL_HAM_LONG(-1.00)[-1.000,0]; DKIM_TRACE(0.00)[jo-t.de:+]; RCPT_COUNT_TWO(0.00)[2]; DMARC_POLICY_ALLOW(-0.50)[jo-t.de,none]; RCVD_COUNT_ZERO(0.00)[0]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; IP_SCORE(-0.12)[asn: 197540(-0.58), country: DE(-0.01)]; ASN(0.00)[asn:197540, ipnet:188.68.48.0/21, country:DE]; MID_RHS_MATCH_FROM(0.00)[] X-BeenThere: freebsd-fs@freebsd.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: Filesystems List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Mon, 28 Oct 2019 23:54:51 -0000 On 28/10/2019 13:06, Andriy Gapon wrote: > On 27/10/2019 22:02, Johannes Totz wrote: >> Hi there, >> >> how does a zpool attach and subsequent resilver deal with the bootcode that's >> inside a bsdlabel partition? >> I'm trying to create a bootable image (with mbr, slices, and partition) but >> every time I attach the partition to a pool the bootcode ends up garbled and the >> image is not bootable. >> >> For example: >>> /usr/bin/truncate -s 1025m /var/tmp/usbbootpoolimg.raw >>> /sbin/mdconfig -a -t vnode -u 3 -S 512 -y 16 -x 63 -f /var/tmp/usbbootpoolimg.raw >>> >>> /sbin/gpart create -s mbr /dev/md3 >>> /sbin/gpart add -s 1g -t freebsd /dev/md3 >>> /sbin/gpart set -a active -i 1 /dev/md3 >>> /sbin/gpart bootcode -b /boot/boot0 /dev/md3 >>> /sbin/gpart create -s bsd /dev/md3s1 >>> /sbin/gpart add -t freebsd-zfs /dev/md3s1 >>> >>> /sbin/bsdlabel -B /dev/md3s1 >>> /bin/dd if=/boot/zfsboot of=/var/tmp/zfsboot1 count=1 >>> /sbin/gpart bootcode -b /var/tmp/zfsboot1 /dev/md3s1 >>> /bin/dd if=/boot/zfsboot of=/dev/md3s1a skip=1 seek=1024 >> >> All good up to here. Image is bootable, but there is no pool yet in the image so >> the bootloader complains. Fine. >> Then: >> >>> /sbin/zpool attach bootpool gpt/boot2 /dev/md3s1a >> >> Now the freshly installed bootcode is trashed. If I try to boot this it makes it >> to the F1 boot prompt (the mbr part) but then hangs indefinitely. >> >> If I write the bootcode again, after the pool resilver is done then the image >> boots. >> >> To me, the bootcode is part of the partitioning scheme, not part of the pool >> that needs resilvering. >> Any ideas? > > You write the second part of zfsboot inside of ZFS managed area. > So, you should do that *after* ZFS initializes the partition. I had https://www.freebsd.org/cgi/man.cgi?query=zfsboot interpreted to mean that zfs wont touch the special boot area (even though it's under zfs control). Is that vdev initialisation the only time that the bootcode gets overwritten? Never on "normal" resilver? Cheers, Johannes