Date: Sun, 10 Nov 2019 22:16:26 -0330 From: Jonathan Anderson <jonathan.anderson@mun.ca> To: "freebsd-fs@FreeBSD.org" <freebsd-fs@freebsd.org> Subject: Broken ZFS boot on upgrade Message-ID: <CAP8WKbJWSHzhFCKijRVxydKEwgD_4NX2gmA-QVEVZPuotFCGvQ@mail.gmail.com>
next in thread | raw e-mail | index | archive | help
Hello friendly FreeBSD people, I’ve gone and done it: I upgraded a key ZFS-on-root machine from 11.2 to 12.0 and now I can't boot my ZFS-on-root pool. I wonder if the folks on this list might be able to help me figure out what's wrong and what I can do about it? It looks like the ZFS code in the bootloader can't find anything in my root directory (zroot/ROOT/default), even though a booted FreeBSD kernel can. If I boot a rescue image from USB I can mount everything in the pool (`zpool import -f -R /mnt zroot`) and see all of my data, but when I run `lszfs zroot/ROOT/default` from the loader prompt it gives me an empty result (so, e.g., no /boot). Booting fails with messages such as, "i/o error - all block copies unavailable". My pool consists of three mirrored vdevs, in which the first mirror uses GPT partitioning (for the boot partitions) and the other two mirrors use whole disks. I recall reading somewhere that the bootloader ZFS code doesn't like non-partition-based vdevs... is that true? If so, perhaps the issue is that my upgrade caused /boot to live on one of the newer whole-disk-based mirrors, hiding it from the bootloader's view? I've backed up all of my data (a 28h process) and am resigned to re-installing if need be, but I would like to understand where I went wrong to avoid making the same mistakes again. What would you recommend for someone building a new array-of-mirrors pool? I see that the Handbook says that certain things *can* be done, but it’s not clear to me whether they should or even must be done, e.g.: > partitions with bootcode and file systems needed for booting can be added. This allows booting from disks that are also members of a pool. There is no performance penalty on FreeBSD when using a partition rather than a whole disk The Handbook suggests that it's possible to break disks into multiple partitions that are added separately to a vdev, but is this a sensible thing to do? Is there any other ZFS lore that hasn't made it to the Handbook but that ought to be kept in mind from the outset? Thanks, Jon-- Assistant Professor Electrical and Computer Engineering Memorial University of Newfoundland https://www.engr.mun.ca/~anderson
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?CAP8WKbJWSHzhFCKijRVxydKEwgD_4NX2gmA-QVEVZPuotFCGvQ>