Date: Tue, 25 Oct 2016 11:32:57 +0200 From: Roger Pau =?iso-8859-1?Q?Monn=E9?= <roger.pau@citrix.com> To: <niek@bigfoot.com> Cc: <freebsd-xen@freebsd.org> Subject: Re: FreeBSD Xen DomU boot from drbd fails with error 19 Message-ID: <20161025093257.eu2b25hs7svv3g46@mac> In-Reply-To: <9c49bd4e-abfe-3d9a-70f5-12895586f389@bigfoot.com> References: <a3784827-5410-eabc-68d6-bd823f5dad06@bigfoot.com> <20161024170219.j5btql5s6f6xeylb@MacBook-Air-de-Roger.local> <9c49bd4e-abfe-3d9a-70f5-12895586f389@bigfoot.com>
next in thread | previous in thread | raw e-mail | index | archive | help
On Mon, Oct 24, 2016 at 08:50:01PM -0400, niek@bigfoot.com wrote: > On 24-10-2016 13:02, Roger Pau Monné wrote: > > On Sat, Oct 22, 2016 at 04:14:33PM -0400, niek@bigfoot.com wrote: > >> Running a FreeBSD 11.0-RELEASE HVM guest, using ufs, on Xen 4.4 (debian > >> jessie) on LVM with drbd 8.4. The FreeBSD guest boots fine from LVM > >> using 'disk' /dev/<vg>/<volume> in domU config. > >> > >> I have a number of Linux guests running on drbd (8.4). Booting FreeBSD > >> from the drbd device associated with drbd (with 'disk' /dev/drbdx) > >> fails. I know I can't use drbd:<resource> with HVM guest. > >> > >> Trying to mount root from ufs:/dev/xbd0p2 [rw]... > >> mountroot: waiting for device /dev/xbd0p2... > >> Mounting from ufs:/dev/xbd0p2 failed with error 19. > > > > Ops, hit send to soon. Can you also paste the output of `xenstore-ls -fp` > > executed from Dom0 when the guest is in this state? > > > > Thanks, Roger. > > > > Thanks for replying. > > Sorry for the long post. > There is one Linux PV guest on this host, not on drbd, but there are > configured drbd devices on this host that belong to linux PV guests on > another host also running debian jessie. They boot from drbd without > problems. > > To collect more data I created a fresh FreeBSD HVM guest on the same > host. Interestingly, this guest does boot from drbd. You'll find it as > domain 16 (vps13) in the attached output of xenstore-ls. > > The HVM guest that fails to mount root is domain 17 (vps10). > The error message is: > Trying to mount root from ufs:/dev/xbd0p2 [rw]... > GEOM: xbd0: corrupt or invalid GPT detected. > GEOM: xbd0: GPT rejected -- may not be recoverable. > > I should mention that there seems to be a conflict with the second GEOM > GPT table in FreeBSD and the drbd metadata both writing to the end of > the partition (we are using internal metadata). After creating metadata > (drbdadm create-md) the kernel complains about the secondary GPT table, > but that does not prevent it from mounting the fs. > > Inside the domu this can be resolved by issuing 'gpart recover', but > this in turn seems to destroy the drbd metadata, which is apparent by > the need to re-create md and re-sync at the next boot. > > Perhaps we should use external metadata in this case. I have no idea how drbd works, but it certainly shouldn't store internal metadata in the device presented to the guest, or else havoc is going to happen for sure. I would suggest that you try this "external metadata" mode. > As an aside, for your information, we tried to run the FreeBSD 11.0 > guest in PVH mode as well, having followed the instructions on > https://wiki.xen.org/wiki/FreeBSD_PVH. > > The guest failed to start with xl reporting en error: > xc: error: elf_xen_addr_calc_check: ERROR: ELF start or entries are out > of bounds.: Invalid kernel > Output of readelf -l /xenkernels/freebsd/kernel-11.0-RELEASE: > Elf file type is EXEC (Executable file) > Entry point 0xffffffff802ff000 > There are 6 program headers, starting at offset 64 That's expected, Xen 4.4 has some PVH support, but that's not enough, IIRC you should use Xen 4.6 or newer. Roger.
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?20161025093257.eu2b25hs7svv3g46>